Malware that encrypts and holds files at ransom can have devastating effects on personal documents, customer data and business operations. If you were designing software that could detect and impede such ransomware, what techniques would you use to safeguard the endpoint? Accomplishing this could involve observing the behavior of the programs running on the system to spot ransomware-like activity or employing deception to encourage ransomware to reveal itself.

Detecting the Presence of Ransomware

Consider how the behavior of file-encrypting ransomware might be different from the operation of normal programs. Ransomware probably reads and writes more files than typical processes on the system. In addition, the encrypted files it creates have patterns that are different from the majority of non-encrypted files. That means that our anti-ransomware tool could:

• Flag processes that read or write too many files too quickly. Our tool would define a threshold of normal file access activities, and react when it is exceeded. A variation of this approach is implemented by Malwarebytes Anti-Ransomware, which is based on Nathan Scott's CryptoMonitor tool. It counts how often untrusted processed have modified "a certain number of personal files, under a certain time." A similar method implemented by Adam Kramer's hande_monitor tool on the frequency with which processes create handles through which they can access files.

• Flag processes that change files' entropy values. Encrypted files tend to have a more uniform distribution of byte values than other files. Their contents are more uniform. Our tool could compare the file's entropy before and after the change. If a modified file has higher entropy, it might have gotten encrypted, indicating that the responsible process might be ransomware. However, the entropy of encrypted files is similar to that of compressed files; to keep down the rate of false positives, Patrick Wardle's RansomWhere? tool also accounts for "Chi Square distribution and Monte Carlo pi approximations" to determine when encrypted files are created.

In addition to observing the behavior of the processes, our tool could set generate files that act as traps, in the hopes that ransomware gets caught in them. For example, the tool could create a set of decoy files that a normal user isn't expected access. Any attempt to read or modify these files would be suspicious. CryptoMonitor implemented some variation of this technique.

A similar approach is also employed by the Yago Jesus' Anti Ransom utility, which creates a set of document files in a decoy folder and monitors it for changes. Along these lines, the folks behind the Free Forensics blog published sample PowerShell scripts that watch for changes to such file "canaries."

Reacting to Suspected Ransomware

When our tool spots the likely presence of ransomware, it could alert the user and then, if configured to do so, kill the offending process. Terminating the process could be risky, though, if ransomware operates by injecting its code into legitimate processes. This highlights the importance of coupling anti-ransomware functionality with techniques that look for other malicious behavior, such as code injection, to spot malware even before it exhibits ransomware characteristics.

If terminating the suspicious process (or thread) is impractical or undesirable, out tool could interfere with the potentially-malicious code by slowing down its interactions with the file system. This would give the endpoint's user time to decide whether to terminate suspected ransomware, turn off the system or respond in some other way.

Another way of impeding ransomware was proposed on the Free Forensics blog. This method involves setting up infinitely-recursive directories by taking advantage of certain features of the Windows file system. Ransomware that attempts to traverse the file system to locate the files it needs to encrypt will get stuck in such a sinkhole, giving the victim the opportunity to react to the infection.

Anti-Ransomware in the Real World

Any methods for detecting and impeding ransomware cannot be foolproof, as is the case with any anti-malware technology. For instance, ransomware authors could adjust their code to stay under the threshold anomalous file access. Many aspects of information security have such action-reaction dynamics, which might be the reasons why the majority of anti-malware vendors rarely reveal details about their inner-workings.

While the techniques outlined above might sounds reasonable and achievable in the context of a single endpoint, creating a product that can be deployed on many personal or enterprise systems drastically increases the endeavor's complexity. For example, the tool would need to have a practical way of dealing with false alarms where legitimate processes exhibit ransomware-like behavior or compressed files resemble encrypted ones.

My expectation is that commercial anti-malware vendors are creating or have already developed more sophisticated methods for dealing with ransomware, and have techniques that are more elaborate than what I described in this article. Those who remember the early days of spyware might recall standalone anti-spyware tools that were later merged into mainstream antivirus products. Similarly, anti-ransomware capabilities are becoming an essential feature of modern Internet security suites and anti-malware products.