Approaches to Malware Analysis

Overview

There are three ways to study a malware sample, and each answers a different question at a different depth. Most investigations use them in sequence, stopping when the question at hand is answered.

Approach	Question Answered	Execution	Key Tools
Basic Static	"What is this file?"	None	FLOSS, PEStudio, DIE, Capa
Basic Dynamic	"What does it do?"	Manual, isolated VM	Procmon, Wireshark, FakeNet-NG
Automated Sandbox	"What does it do at scale?"	Automated, instrumented VM	CAPE, Any.Run, Joe Sandbox

Refer to REMA eBook 2026, Figure 1.6 for the three-approach overview diagram.

Basic Static Analysis

Static analysis examines the binary without executing it. Nothing runs. The analyst inspects the file's structure, metadata, and embedded strings on disk. It is the safest starting point for any investigation.

The six-step triage sequence

1. Hash it — Compute MD5 and SHA-256. Query VirusTotal and MalwareBazaar. If the hash is already known with 55/70 detections, you have your answer in thirty seconds.
2. Pull strings — Run FLOSS (Mandiant). Look for IP addresses, domain names, registry paths, file paths, ransom note text, debug strings. FLOSS decodes stack strings and tight XOR loops that the basic strings tool misses.
3. Inspect the PE header — Use PEStudio or PE-Bear. Check the compilation timestamp, imported DLLs and API functions, section names, entropy values, and digital signature.
4. Detect packers — Run Detect It Easy (DIE). If it identifies UPX, ASPack, or Themida, the binary must be unpacked before deeper analysis.
5. Map capabilities — Run Capa (Mandiant). Instead of raw import names, Capa reports behavioural capabilities: "can delete shadow copies," "communicates via HTTP," "captures keystrokes."
6. Match YARA rules — Scan against community and custom rules for known families (LockBit, Conti, Emotet).

Limitation

If the binary is packed or encrypted, strings are gibberish, imports are stripped, and Capa cannot see past the packer stub. When static analysis hits a wall, move to dynamic.

Basic Dynamic Analysis

Running the malware in a controlled, isolated environment and observing what it does at runtime. Unlike static analysis, dynamic analysis sees what the malware actually does, including behaviour hidden by packing or encryption.

Cardinal rule: Never run malware on your host machine. Never on a system connected to a production network. Never in a VM with shared folders or clipboard sharing enabled. Snapshot the VM before execution. Revert after every run.

Three monitoring dimensions

Dimension	Tools	What to Watch For
Process activity	Process Hacker, Process Explorer	Child processes (cmd.exe, powershell.exe), thread injection into explorer.exe, mutex creation
File and registry	Procmon, Regshot	Files dropped in %TEMP%, DLLs in System32, Run keys for persistence, ransom notes
Network traffic	Wireshark, FakeNet-NG	DNS queries to suspicious domains, HTTP POST exfiltration, TOR traffic, C2 beaconing

What five minutes of dynamic analysis reveals

Detonate a Ryuk sample in an isolated VM with Procmon, Regshot, and FakeNet-NG running:

1. Procmon logs cmd.exe /c vssadmin delete shadows /all /quiet within the first ten seconds — confirms ransomware immediately.
2. Regshot diff shows a new Run key pointing to the dropped payload in %APPDATA%.
3. FakeNet-NG captures a DNS query for the C2 domain and an HTTP POST containing the per-machine AES key encrypted with the attacker's RSA public key.
4. File system monitoring shows mass .ryuk extension appending and a RyukReadMe.html note in every affected directory.

Total time: under five minutes.

Automated Sandbox Analysis

Manual dynamic analysis is thorough but slow. Automated sandboxes handle the volume problem: a SOC processing 500 alerts per day cannot manually detonate each one.

Sandbox	Type	Capability
Cuckoo / CAPE	Open-source	VM-based execution, API hooking, PCAP capture, JSON/HTML report. CAPE adds automatic payload extraction.
Any.Run	Cloud	Interactive: the analyst controls the VM through a browser in real time.
Joe Sandbox	Commercial	Deep behavioural analysis, evasion detection, automated MITRE ATT&CK mapping.
Hybrid Analysis	Free online	CrowdStrike Falcon engine. Good for quick triage.

Sandbox blind spots

Malware that sleeps for ten minutes before executing will outlast the sandbox timeout. Malware that checks for VMware MAC addresses, low RAM, or the CPUID hypervisor bit will detect the sandbox and refuse to detonate. These evasion techniques and their bypasses are covered in Unit 6.

Comparing the Three Approaches

	Basic Static	Basic Dynamic	Automated Sandbox
Time per sample	5–15 min	15–60 min	2–5 min
Sees past packing	No	Yes	Yes
Handles volume	Moderate	Low	High
Blind spot	Packed/encrypted	Time-bombs, env checks	VM-aware evasion

The Analysis Pipeline

The three approaches form a sequential pipeline. A known Emotet dropper with 60/70 VirusTotal detections can be blocked at stage 1. A novel, packed, evasive implant requires all four stages spread across days of work.

Static Triage → Dynamic Analysis → Code Analysis → Report & Defend
"What is it?"   "What does it do?"  "How does it   "How do we
                                     work?"          stop it?"