Even if malware analysis is not your primary occupation, once in a while you may find yourself wondering about the nature of an unfamiliar malicious executable that crosses your desk. Starting your investigation with behavioral analysis—an observation of how the specimen interacts with the file system, the registry and the network—can rapidly produce useful results. Virtualization software such as VMware is incredibly helpful in this process.

Advantage of malware analysis with VMware

VMware allows for the simulation of multiple computers running simultaneously on a single physical system. There are several advantages to this approach for behavioral malware analysis, compared to a lab built using distinct physical infrastructure components:

• It's often beneficial to have several systems in the analysis lab, so that the malware can interact with components of the simulated Internet. With VMware, it's possible to build a multi-component laboratory without the hulk of multiple physical boxes.
• Being able to take a snapshot of the system's state before infecting it and taking periodic snapshots throughout the analysis saves time. This functionality provides an easy means of reverting to the desired system state almost instantaneously. VMware makes this simple with its integrated snapshot feature. VMware Workstation, a commercial product, allows multiple snapshots. VMware Server, which is a free product, supports only a single snapshot. VMware Player, also free, cannot take snapshots at all.
• VMware's host-only networking option is convenient for interconnecting virtual systems using a simulated network without additional hardware. This setup also makes it less likely that an analyst would be tempted to connect the laboratory environment to the production network. The host-only network allows any virtual system to see all traffic on the simulated network when listening in promiscuous mode. This makes monitoring the specimen's network interactions easy.

Getting started with VMware malware analysis

Preparing a VMware-based analysis laboratory is simple. You need a system with plenty of RAM and disk space that will act as the physical host. You also need the necessary software: VMware Workstation or Server, and the installation media for the OS you'll deploy in the lab.

VMware emulates the computer's hardware, so you must install the OS into each virtual host created using VMware's new Virtual Machine Wizard. Once the OS is set up, install the VMware Tools package, which optimizes the system for operating within VMware. Then install the appropriate malware analysis software.

I recommend having virtual machines with different operating systems in the lab, each representing the OS that malware is likely to target. This enables observation of malicious programs in their native environments. If using VMware Workstation, take snapshots of the virtual system at different points during the security update installation process to analyze malware at the desired patch level.

Keeping production systems safe

When dealing with malware, take precautions not to infect production systems. Such breaches can happen when handling malware improperly or when a specimen exploits a weakness in the VMware setup and escapes its sandbox. There have been several publicly announced vulnerabilities in VMware that, in theory, could allow malicious code from the virtual system to find its way onto the physical host.

Here are some suggestions for mitigating these risks:

• Keep up with security patches from VMware.
• Dedicate the physical host to the VMware-based lab; don't use the system for other purposes.
• Do not connect the physical laboratory system to your production network.
• Monitor the physical host with host-based intrusion detection (IDS) software, such as a file-integrity checker.
• Periodically re-image the physical host using cloning software, such as Norton Ghost. If this option is too slow, look to hardware modules, such as CoreRESTORE, for undoing changes to the system's state.

One of the challenges of using VMware for malware analysis is that malicious code can detect whether it is running within a virtual system, which indicates to the specimen that it is being analyzed. If you cannot modify the specimen's code to eliminate this functionality, you can reconfigure VMware to make it stealthier. Tom Liston and Ed Skoudis documented several VMware .vmx file settings you can insert to accomplish this. The biggest problem with these settings is that they may slow down the virtual system's performance. Also note that they're not supported by VMware.

Virtualization options and strategy

Of course, VMware is not the only option for virtualization software you can use for malware analysis. Common alternatives include Microsoft Virtual PC and Parallels Workstation.

Virtualization software provides a convenient and time-saving mechanism for building a malware analysis environment. Just be sure to establish the necessary controls to prevent malicious software from escaping your testing environment. With a fine-tuned lab, you will be well on your way toward making the most of your malware analysis skills.

For a follow up to this article, please see VMware Network Isolation for a Malware Analysis Lab.