By Pierre Dehombreux, Director of Information Technology at Whiteriver Unified School District
My name is Pierre and I am an IT director at Whiteriver School District in Arizona, U.S. I am a recent ransomware survivor, and I’d like to share my story and the lessons I learned with you.
Who We Are
Whiteriver School District comprises three elementary, one high and one general school. Our IT infrastructure is very dynamic. Every year, new children enroll and graduates leave. Teachers and other personnel are coming and going, too — the turnover is really high here. This lifecycle results in a never-ending stream of changes in our IT environment, which my small team of three has to control. We must constantly add and remove users, take care of our backups, make sure nobody touches sensitive files they’re not supposed to access, maintain a clean environment, and so on. We never get bored at school!
How We Detected an Anomaly
My teammates and I routinely review reports on file activity delivered by Netwrix Auditor, just to make sure there is nothing anomalous going on. We actively use our file shares to store a very large amount of information, including highly sensitive data — from attendance records to personal student information. Everyone working in the district needs access to some of this data, so we have to keep it safe. If we lost some of the files, the nurse might not be able to retrieve a student’s health records or find the emergency contact information if something bad happened. We know that any anomaly — such as somebody trying to modify many files at once — is a warning bell that something might be wrong.
We saw that one of the file share activity summaries, which is typically 20–30 pages long, reached 100 pages
One day in October, right after fall break, we saw that one of the file share activity summaries, which is typically 20–30 pages long, reached 100 pages. We immediately noticed a huge number of failed file modifications: A user was trying to modify hundreds of files in one of our most critical shares.
The report showed the user account name, so we didn’t have to be a Sherlock to find the user in our Active Directory, along with the building and the room she sat in.
Resolving the Ransomware Problem in 5 minutes
What happened next? I rushed to the room where that user works and saw one of our teachers in tears. “I got an email, clicked on the attachment, and now all my files are encrypted,” she cried.
She needed those files to prepare an expense report for the State of Arizona, which finances her vocational program. Without the files that had been encrypted, our school would have had to close the program and pay back all the money given for it — around $60,000 — as well as a large fine from the Arizona Auditor General. Just one click on a malicious email, and the whole school district was facing a pretty big financial hit.
I came to her computer and saw a pop-up window with a message from the attackers demanding a ransom for the decryption key. Honestly, I just laughed at it. All I had to do was disable the compromised account and restore the encrypted files from the offline backup. It took us not more than five minutes to recover from the ransomware attack and give our teacher back her peace of mind.
Lessons Learned: Make Good Anti-Ransomware Habits Stick
I cannot say that my life changed that day, but our approach to securing the data was certainly borne out by these events. I would be happy to share some basic rules that we established in our IT department that enabled us to beat that ransomware and that help us minimize the risk of ransomware damage in the future:
1. Gain visibility into your IT environment
We were able to track down this issue in minutes because we have visibility into what is happening in our IT environment, including activity across file shares. With deep visibility, you can quickly figure out if something illicit like crypto intrusion is happening in your shares.
2. Follow the least-privilege principle
It is essential to make sure that all file permissions are set up properly. In our case, ransomware managed to encrypt only a limited amount of data — just the files that one teacher had rights to modify because they were directly related to her job. The files she had read-only access to were safe; the malware was unable to modify them.
3. Regularly test users’ permissions
Because proper file permissions are of vital importance, it is essential to test them: Regularly log as a user and try to delete stuff or change files, as if you were a bad guy. Do the same after applying patches. I cannot count how many times I have downloaded a software upgrade and all the permissions were changed. One day, improper permissions could cost you a fortune.
4. Take regular backups and limit access to them
Users need access to their own documents and permissions to modify them. Let’s be honest, ransomware will encrypt that, and there is nothing you can do about it. But that’s not a big deal if you have a proper backup. Along with automated online backups, we do offline backups every morning; it takes my assistants just a couple of minutes.
Moreover, the IT staff are the only people in the whole district who can access backup files; they have special accounts for that purpose that do not have a connection to the internet. No internet means no emails. No emails means no temptation to click on the wrong thing and share the same fate as one of our teachers.
5. Set your spam filter
After what happened with the teacher, we experienced another ransomware attack coming from an email. Fortunately, it was stopped in time, thanks to properly configured spam email filter rules that hinder the delivery of potentially harmful documents.
However, one user who received the email did not take the filter seriously enough. She circumvented it by forwarding the email from her Spam folder to her personal email account, and then opened it on her laptop. As a result, all her personal files got encrypted. In her defense, the email seemed to be coming from a U.S. federal agency, so I can see why she did not trust the filter.
6. Raise awareness and encourage employees to notify about the problem
Finally, you should make your employees and colleagues understand that it never hurts to ask for help and admit their mistakes. The faster they report on the problem, the faster you can respond, and the less tears they shed.
Here is what an expert thinks…
The school district prevented the disaster because it had visibility into its IT environment, tested backups, and had restricted user access to file shares. But prevention is better than cure. Ensuring Windows is up-to-date is critical as ransomware often exploits vulnerabilities that have been patched by Microsoft. The principle of least privilege can also be applied to end-user devices. Users shouldn’t log in with administrator privileges and application control should be configured so that only trusted executables, apps, scripts, and Windows Installer files can run.
These simple tips will help you mitigate the risk of ransomware jeopardizing the security of your IT ecosystem and stay on the lookout for the next attack. Therefore, I advise all IT pros to take care of establishing healthy habits before ransomware takes care of them.
Director of Information Technology at Whiteriver Unified School. An IT professional with more than 19 years of experience. Pierre holds MCSE and CISSP certifications. In the Netwrix blog, he describes issues he has faced in his work, such as a ransomware attack, data theft and malicious insider activity, and shares the lessons he learned from them.
