Architecture¶

ShadowClaw is one detection core fed by platform-specific acquisition. Thirty modules, no third-party imports, and a hard rule about which of them are allowed to know what operating system they are on.

Modules30

Third-party imports0

Test methods983

Test files33

The shape of the whole thing¶

flowchart TB
  subgraph ACQ["Acquisition — platform-specific"]
    P1["procprobe.py<br/>ps + lsof"]
    P2["dnssniffer.py<br/>tcpdump :53"]
    P3["esf.py<br/>eslogger"]
    P4["configwatch.py<br/>agent config polling"]
  end

  subgraph CORE["Detection core — platform-neutral"]
    R["resolver.py<br/>IP → provider"]
    S["scoring.py<br/>Plane A + B correlation"]
    T["tactics.py<br/>event → tactic"]
    A["agentchain.py<br/>lineage + sessions"]
    C["catalog.py<br/>providers + runtimes"]
  end

  subgraph OUT["Output"]
    L["ledger.py<br/>hash-chained SQLite"]
    O["otlp.py + otlpevent.py<br/>OTLP/HTTP JSON"]
    K["ktp/<br/>risk factors + envelope"]
  end

  SEN["sensor.py<br/>poll loop"]

  P1 --> SEN
  P2 --> R
  P3 --> SEN
  P4 --> SEN
  R --> S
  C --> R
  C --> S
  SEN --> S
  SEN --> T
  T --> A
  S --> L
  A --> L
  L --> O
  SEN --> K
  K --> O

The rule: acquisition varies, detection does not¶

Every platform-specific thing the sensor does is shaped the same way — invoke a system tool or read a kernel interface, parse the output into a neutral record. ps, lsof, eslogger and tcpdump all follow that pattern, and the records they produce carry nothing macOS-specific in their fields.

That is why porting is a seam and not a rewrite. It is also why the following list is fixed:

Must not vary by platform	Why
`ProcessSnapshot`, `Connection`, `Listener` record types	Everything downstream is written against them.
`scoring.py`, `agentchain.py`, the chain logic in `tactics.py`	This is the part with value in it. Forking it means every improvement is written three times and drifts twice.
The ledger format, including the hash chain	Evidence has to be verifiable by one implementation.
`otlp.py` and `otlpevent.py` wire formats	A finding from Linux and a finding from a Mac must be the same object, or every consumer downstream needs to know which produced it.
Everything under `shadowclaw/ktp/`	The Risk Factor contract is a spec, not an implementation detail.

tests/test_tactics_seam.py reads the source of tactics.py, scoring.py and agentchain.py with docstrings and comments stripped, and fails the build if any of them names an operating system or asks which one it is on. The rule is enforced, not documented.

Module map¶

OrchestrationAcquisitionDetectionOutputPlatforms & KTP

Module	Responsibility
`__main__.py`	Argument parsing, config resolution, self-test, catalog display, ledger views, sensor lifecycle.
`sensor.py`	The polling loop and the output fan-out.
`settings.py`	Dataclass defaults, JSON loading, config search order, path resolution.
`authorship.py`	Product version, canonical attribution, SHA-256 digest, telemetry identity.
`preflight.py`	Advisory startup checks: OTLP reachability, ESF admission, competing ledger writer.

Module	Responsibility
`procprobe.py`	macOS `ps` and `lsof` parsing into neutral records.
`dnssniffer.py`	Optional root `tcpdump` DNS answer capture.
`esf.py`	`eslogger` subscriptions, bounded queue, JSON normalisation, credential prefilter.
`configwatch.py`	Unprivileged mtime/size/hash polling of agent configuration files.

Module	Responsibility
`catalog.py`	Hosted providers, local runtimes, candidate-runtime regexes, hostname heuristics.
`resolver.py`	IP scope and IP-to-host attribution via DNS capture, catalog cache, or PTR.
`scoring.py`	Plane A/B per-process correlation, confidence weighting, de-duplication, banding.
`tactics.py`	Pure Plane C tactic classification and ATT&CK mapping. No I/O.
`agentchain.py`	Process lineage, agent attribution, session windows, ordered chain scoring.
`redact.py`	Command-line and secret scrubbing.
`verify.py`	The deterministic synthetic detection matrix.

Module	Responsibility
`ledger.py`	SQLite events and episodes, raw JSONL mirror, hash chain, retention checkpoints.
`ledgerview.py`	Terminal summary plus CSV, JSON, and HTML reports.
`watchview.py`	Live plain-English ledger tail.
`otlp.py`	Direct OTLP/HTTP protobuf-JSON encoding over `urllib`. Logs, gauges, counters.
`otlpevent.py`	Flat JSON event schemas and OTLP attribute mapping.

Module	Responsibility
`platforms/__init__.py`	Platform registry, capability model, locations, plane vocabulary.
`platforms/darwin.py`	macOS capabilities, locations, adapters.
`platforms/linux.py`	`/proc` process and socket acquisition; declares Plane C via `linuxevents`.
`linuxevents.py`	Linux Plane C: `cn_proc` process events, composed with the `fanotify` file plane.
`linuxfanotify.py`	`fanotify` via `ctypes` — credential/persistence file events, root-gated.
`platforms/indicators.py`	Per-platform credential, persistence, identity, and privilege evidence.
`ktp/stress.py`	Stress-term resolution. Unobserved means maximum stress.
`ktp/riskfactors.py`	Maps poll observations to the four KTP factors.
`ktp/aggregation.py`	Aggregation entries that preserve empty and silent windows.
`ktp/envelope.py`	Per-action autonomy demand, environmental capacity, supervision floor.

No third-party dependencies, enforced¶

The sensor speaks OTLP/HTTP JSON directly over urllib rather than importing the OpenTelemetry SDK, parses JSON with json, and reads the process table with subprocess and string handling. There is no psutil, no requests, no pyyaml — which is also why the config format is JSON.

tests/test_ktp_zero_dependency.py fails the build if anything under shadowclaw/ imports a third-party module. It checks twice: by walking the source, and by importing the sensor in a fresh interpreter to see what actually loaded.

That rule is why it runs on the stock macOS python3 with no virtualenv, and it is also why eBPF was never a candidate for planes A and B.

Blindness is a value, not a comment¶

The load-bearing idea in shadowclaw/platforms/ is that a platform declares what it cannot see.

Capability is a dataclass with rules enforced in __post_init__: an available plane must name its mechanism, and an unavailable one must give a reason. A backend may not claim a plane it returns no source for, pinned by a test.

requires_root and requires_grant are separate fields because they fail differently. Root is a decision a script can make. A grant — Full Disk Access on macOS, a kernel facility on Linux — is a human at a settings pane or a reboot. Telling someone to run as root when the real answer is a grant costs them an afternoon.

The same discipline runs through KTP, where an unobserved term reads as maximum stress rather than zero, so a sensor that cannot see is never mistaken for a calm host. See Risk Factors.

Official-build provenance¶

shadowclaw/authorship.py is the single source of truth for product identity, covered by a SHA-256 digest over the identity block. Every source file carries the authorship header, every OTLP record carries shadowclaw.author on its resource attributes, and every finding carries a detected_by block.

Editing the identity block changes the computed digest. The tool then warns on every start and tags its telemetry shadowclaw.attribution.intact=false — an official-build provenance indicator — while keeping running. Bricking a security control over an edited string would turn a provenance check into a denial of service against the operator. The indicator is product metadata, not a license-compliance signal. See Authorship.

Next¶

Data flowOne poll, start to finish, in order.

Multi-platform seamLinux, Windows, and the one decision left open.

Browser telemetryThe gap TLS-in-the-browser creates.

Event schemasThe wire formats that must not vary.