The local ledger¶
Splunk may be unreachable. The Collector may be down. make clean wipes data/. None of that is allowed to cost you the record.
Every finding is written to a durable local ledger before any network export, so the local copy is the one thing that always exists.
python3 -m shadowclaw --ledger # readable summary
python3 -m shadowclaw --ledger verify # check the hash chain
python3 -m shadowclaw --ledger path # where the files are
python3 -m shadowclaw --ledger --since 7d # filter by age
python3 -m shadowclaw --ledger export --format html --out report.html
make ledger, make ledger-verify and make ledger-report wrap the common ones.
Where it lives¶
System-wide when running as root, per-user otherwise.
Root is the deployment mode that sees every user's sockets, so its records are a machine-wide account of the host. An unprivileged run sees the whole process table but attributes sockets only for its own user, so its ledger is a partial record and would be misleading written to the same place.
Directory 0700, files 0600. Deliberately outside the repository, so cleaning the working tree cannot destroy the evidence.
findings.jsonl, ktp-risk-factors.jsonl and ktp-envelope.jsonl land in the same directory and follow the same split, so there is one answer to "where did it go" rather than four. A relative findings_path is anchored there too; only an absolute one moves it, and "" switches it off.
On Linux the paths come from the platform layer rather than macOS literals — /var/lib/shadowclaw and ~/.local/share/shadowclaw. See Multi-platform seam.
Two files, on purpose¶
File 1
`ledger.db`
SQLite, and the queryable record. Three tables: meta, events, episodes.
- Open with the
sqlite3macOS ships - Or DB Browser, Excel, pandas
- No ShadowClaw required
File 2
`ledger.jsonl`
The raw append-only stream, one line per observation.
- Greppable and tail-able
- No tooling needed
- Easy to forward to a SIEM later
Not evidence
The `episodes` rollup
Mutable and not chained. A convenience that can be rebuilt from events.
- Only the immutable log is evidence
- Folds repeat sightings into one row
- Ends after
ledger_episode_gap_seconds
sqlite3 ~/Library/Application\ Support/ShadowClaw/ledger.db \
"SELECT ts, process_name, pid, severity, endpoints
FROM events ORDER BY seq DESC LIMIT 20;"
Episodes¶
Repeat sightings of the same activity fold into one episode carrying first seen, last seen, observation count, and peak risk — so a process talking to Anthropic for an hour is one row in a report, not sixty.
An episode ends after an hour of quiet (ledger_episode_gap_seconds, default 3600.0), so a long-lived agent stays a readable timeline rather than one unbounded row.
Schema¶
Immutable, append-only, hash-chained. One row per observation.
| Column group | Contents |
|---|---|
| Identity | seq, ts, host, episode_id, finding_id |
| Process | pid, process_name, exe_path, cmdline (redacted), user |
| Score | risk_score, severity |
| Evidence | signals, endpoints, providers, categories, attribution_source |
| Resources | cpu_percent, rss_mb |
| Full record | payload |
| Chain | prev_hash, hash |
Mutable rollup per process session.
| Column | Meaning |
|---|---|
episode_id |
Session identity. |
first_seen, last_seen |
Window bounds. |
count |
Observations folded in. |
peak_risk |
Highest score seen. |
signals, endpoints, providers |
Merged across the episode. |
| Key | Meaning |
|---|---|
product, version, author |
Identity of the writer. |
schema_version |
Ledger format version. |
attribution_digest |
SHA-256 over the authorship identity block. |
genesis_hash, checkpoint_hash |
Chain roots. |
Tamper evidence¶
The events table is immutable and hash-chained: every row commits to a SHA-256 over its own contents plus the previous row's hash, and the chain is rooted in the authorship digest.
Editing a score, deleting a row, or reordering the log all break it — and --ledger verify names the exact sequence number and which of the two happened:
python3 -m shadowclaw --ledger verify
An insider quietly deleting the evidence of their own shadow AI use is the threat this exists for.
Tamper-evident, not tamper-proof
Anyone with the privileges to run the sensor can destroy the whole file. What they cannot do is make a selective, quiet edit. Ship the ledger off-box to close the rest of the gap.
Retention¶
| Setting | Default | Effect |
|---|---|---|
ledger_retention_days |
0.0 |
0 keeps everything. |
Pruning checkpoints the chain, so retention and tamper-evidence stay compatible — verification resumes from the checkpoint rather than failing on a gap it cannot explain.
Reports¶
python3 -m shadowclaw --ledger export --format csv --out findings.csv
python3 -m shadowclaw --ledger export --format json --out findings.json
python3 -m shadowclaw --ledger export --format html --out report.html
| Format | Good for |
|---|---|
| Terminal summary | A look at the last hour. |
events |
Raw JSON, newest first, for scripting. |
| CSV | Spreadsheets and ad-hoc pivots. |
| JSON | Programmatic consumption. |
| HTML | A self-contained report to hand to someone. |
Watching it live¶
python3 -m shadowclaw --ledger watch
python3 -m shadowclaw --ledger watch --since 1h --heartbeat 30
python3 -m shadowclaw --ledger watch --no-follow
watch follows ledger.jsonl and renders one plain-English line per change. It backfills 5 minutes by default, prints a still-active line for long-running episodes every --heartbeat seconds, and does not contend with the writer — a viewer is never mistaken for a competing sensor.
Two sensors, one directory¶
Two sensors at the same privilege level share a ledger directory and record every episode twice. BEGIN IMMEDIATE keeps the hash chain intact, so the damage is logical duplication rather than corruption — which is worse in one respect, because nothing fails and the counts simply read high.
A pid-bearing sensor.owner file makes the second one say so at startup. See Startup checks.
Two sensors in different directories are not a conflict, so the root/per-user split still separates a privileged run from an unprivileged one.
The KTP files are separate on purpose¶
ktp-risk-factors.jsonl and ktp-envelope.jsonl sit beside the ledger but are not rows in the hash-chained events table.
That table records conclusions about processes. A periodic environmental measurement is not one, and neither is a per-action supervision receipt. Both files carry a flat record — no nested object and no array, evidence included — because Grafana parses the line with | json, which reaches neither.
Neither file rotates. ktp-envelope.jsonl grows with agent activity rather than with uptime, so ship it off-box or rotate it with newsyslog on a busy host.