CloudTrail Forensics, Evidence Collection & Chain of Custody

Why CloudTrail Is the Forensic Gold Standard

CloudTrail is the primary audit and forensic data source for AWS investigations. Every API call — every RunInstances, every GetObject, every AssumeRole — generates a CloudTrail event. Unlike traditional forensics where logs may not exist or may be local to a compromised host, CloudTrail is generated by AWS infrastructure independently of the target instance.

CloudTrail Event Anatomy

A CloudTrail event contains the following forensically significant fields:

Field	Forensic Significance
eventTime	When the action occurred (UTC)
eventName	What API action was called (GetCallerIdentity, PutObject)
eventSource	Which AWS service processed the call (s3.amazonaws.com)
awsRegion	Region where the action was processed
sourceIPAddress	Source IP of the API caller — critical for attribution
userAgent	Tool used (aws-cli/2.9, Boto3/1.28, browser console)
userIdentity	Identity of the caller (IAM user ARN, role ARN, account ID)
requestParameters	Input parameters to the API call
responseElements	What the API returned (confirms success or reveals created resources)
errorCode	If the call failed, why (AccessDenied, NoSuchBucket)
errorMessage	Human-readable error detail

High-Value Event Patterns

These CloudTrail event sequences indicate malicious activity and should trigger immediate investigation:

Pattern	Events	Indication
Credential use from new geography	Any event with unusual sourceIPAddress	Credential theft and external use
Account reconnaissance	ListBuckets, DescribeInstances, ListUsers, ListRoles in rapid sequence	Attacker mapping environment
Defence evasion	DeleteTrail, StopLogging, DisableGuardDuty	Attacker removing visibility
Privilege escalation	CreatePolicyVersion, AttachRolePolicy, CreateRole followed by AssumeRole	Building admin access
Data staging	PutBucketReplication, CreateBucket in new region, then mass CopyObject	Preparing data exfiltration
Persistence	CreateUser + CreateAccessKey outside business hours	Creating backdoor account

Querying Logs at Scale with Athena

CloudTrail logs are stored in S3 as JSON files. For large environments, searching them manually is impractical. Amazon Athena provides serverless SQL queries over S3-stored logs.

Setting Up Athena for CloudTrail

CREATE EXTERNAL TABLE cloudtrail_logs (
    eventVersion STRING,
    eventTime STRING,
    eventName STRING,
    awsRegion STRING,
    sourceIPAddress STRING,
    userAgent STRING,
    eventSource STRING,
    userIdentity STRUCT<type:STRING, principalId:STRING, arn:STRING, accountId:STRING>
)
PARTITIONED BY (region STRING, year STRING, month STRING, day STRING)
ROW FORMAT SERDE 'com.amazon.emr.hive.serde.CloudTrailSerde'
STORED AS INPUTFORMAT 'com.amazon.emr.cloudtrail.CloudTrailInputFormat'
LOCATION 's3://my-cloudtrail-bucket/AWSLogs/123456789012/CloudTrail/';

Forensic Queries

Find all API calls from a specific IP in the last 30 days:

SELECT eventTime, eventName, eventSource, userIdentity.arn, requestParameters
FROM cloudtrail_logs
WHERE sourceIPAddress = '185.220.101.42'
  AND eventTime > '2026-06-01'
ORDER BY eventTime;

Find all DeleteTrail and StopLogging events:

SELECT eventTime, userIdentity.arn, sourceIPAddress
FROM cloudtrail_logs
WHERE eventName IN ('DeleteTrail', 'StopLogging', 'UpdateTrail')
ORDER BY eventTime DESC;

Log Integrity Validation

CloudTrail includes a digest file delivered hourly to S3 alongside log files. The digest file contains SHA-256 hashes of all log files from the previous hour and is signed with CloudTrail private keys.

To validate log integrity:

aws cloudtrail validate-logs \
  --trail-arn arn:aws:cloudtrail:ap-south-1:123456789012:trail/my-trail \
  --start-time 2026-07-01T00:00:00Z \
  --end-time 2026-07-02T00:00:00Z

If a log file was deleted or modified, validation reports a failure. This mechanism provides cryptographic assurance that logs presented as evidence have not been tampered with since CloudTrail wrote them.

Chain of Custody for Cloud Evidence

Documentation Requirements

For cloud evidence to be admissible, each piece must be documented with:

1. What was collected (specific log file names, S3 URIs, time ranges)
2. When it was collected (UTC timestamp)
3. How it was collected (API calls used, tools, authentication method)
4. Who collected it (identity, organisation, contact)
5. Hash value of collected data (SHA-256 of log files or API response)
6. Provider attestation where available (AWS Certificate of Authenticity)

Preservation Checklist (Run in This Order)

1. Isolate compromised EC2 instances (change security group to deny all egress)
2. Create EBS snapshots of all attached volumes before termination
3. Enable GuardDuty if not already active (it will begin collecting from this point)
4. Download and hash current CloudTrail log files for the suspected time window
5. Preserve VPC Flow Logs for the relevant time window
6. Document all API calls made during preservation (meta-evidence)
7. If Lambda or containers were compromised: collect environment variables, execution logs, and container image digests

Key Takeaway

CloudTrail is not just an audit log — it is the primary forensic data source for any AWS incident. Understanding event anatomy, recognising malicious patterns, querying at scale with Athena, validating integrity with digest files, and maintaining rigorous chain of custody documentation are the skills that distinguish a cloud forensic practitioner.