This cheat sheet helps system administrators examine a suspicious system and decide whether to escalate to incident responders while preserving evidence. It covers what to check in the EDR or cloud console and on the host, what not to touch, how to hand off to responders, and the Windows and Linux commands for an initial look.

Security Incident Survey Cheat Sheet for System Administrators - illustration

This cheat sheet outlines how system administrators can examine a suspicious system, decide whether to escalate, and preserve evidence for responders. To print, use the one-page PDF version. You can also edit the Word version for your own needs. If you’re the incident handler taking over a qualified incident, see the companion Initial Security Incident Questionnaire for Responders.

Decide Whether to Escalate

  • Use the least privileged account that can run these checks, never a domain or cloud admin account.
  • Start with the security tools that already monitor the system, such as an EDR or cloud security console, since they show its activity without changing it.
  • Review system, security, and application logs, and escalate if someone cleared logs or disabled auditing.
  • List network connections and listening ports, and note sessions to unfamiliar addresses.
  • List accounts, roles, and group memberships, including cloud identities, and look for any that appeared recently or gained admin rights.
  • List processes, services, scheduled tasks, and startup programs, and look for entries that don’t belong.
  • In the cloud console, look for new access keys, app permissions, or virtual machines that nobody can explain.
  • If you find nothing but the system still feels wrong, escalate anyway, since rootkits hide from local tools.

Preserve Evidence While You Decide

  • Avoid scans, installs, and other actions that touch many files, which can alter timestamps and overwrite deleted data.
  • Don’t reset passwords, delete files, patch, or reboot on your own, since you could destroy evidence or warn the attacker.
  • To stop an attack in progress, ask the security team to isolate the host through EDR. If nobody answers and data is being encrypted, disconnect it from the network, never from power.
  • On a system that controls a physical process, involve the engineers who run it before you isolate or rebuild it, since you could halt the process.
  • Report a ransom note to the responders privately, without photographing or forwarding it, since anyone with the ID inside can open the attacker’s negotiation site.
  • Note each command you ran, what you saw, and when, with the time zone, in a dedicated notebook.

Examine a Windows System

Task Command
Event logs eventvwr, Get-WinEvent -LogName Security -MaxEvents 500
Key event IDs 4624/4625 logons, 4720 new account, 4698 new task (if audited), 1102 Security log cleared, 7045 new service (System)
Connections netstat -nao, Get-NetTCPConnection
DNS cache ipconfig /displaydns
Users and admins net user, net localgroup administrators
User sessions query user
Processes gcim Win32_Process | fl ProcessId,ParentProcessId,CommandLine
Services, tasks, startup Get-Service, Get-ScheduledTask, gcim Win32_StartupCommand

Examine a Linux System

Task Command
Log files ls /var/log, journalctl -S yesterday
Logins and failures last, who, lastb, wtmpdb last
Connections ss -tunap
Users and sudo cat /etc/passwd, getent group sudo wheel
Processes ps -ww -eo pid,ppid,user,lstart,cmd
Shell history cat /root/.*history /home/*/.*history
Cron jobs and timers cat /etc/crontab, ls /etc/cron.* /var/spool/cron/*, systemctl list-timers
Enabled services systemctl list-unit-files --type=service --state=enabled
SSH keys and preloaded libraries cat /root/.ssh/authorized_keys /home/*/.ssh/authorized_keys /etc/ld.so.preload

Hand Off to the Response Team

  • Escalate to your incident response contact and your manager as soon as a compromise looks likely, without waiting for certainty.
  • Share with responders what you saw, when, and where, the commands you ran, what you changed, the alert that started your review, and the account names, file hashes, IP addresses, and domains.
  • Share details only with the responders, in a way the attacker can’t read, such as by phone, and keep incident data out of public AI chatbots.
  • Responders will verify the incident, triage it with decision-makers, then scope, contain, eradicate, and recover in cycles until a decision-maker closes it.

Other Incident Response Resources

Post-Scriptum

Special thanks for feedback to Lorna Hutcheson, Patrick Nolan, Raul Siles, Ed Skoudis, Donald Smith, Koon Yaw Tan, Gerard White, and Bojan Zdrnja. If you have suggestions for improving this cheat sheet, please let me know. This cheat sheet is distributed under the Creative Commons Attribution 4.0 International License. Take a look at my other security cheat sheets.

About the Author

Lenny Zeltser is a cybersecurity executive with deep technical roots, product management experience, and a business mindset. He has built security products and programs from early stage to enterprise scale. He is also a Faculty Fellow at SANS Institute and the creator of REMnux, a popular Linux toolkit for malware analysis. Lenny shares his perspectives on security leadership and technology at zeltser.com.