Linux user guide¶
Install the public Linux archive, prove that the sensor can collect on this host, and follow a detection from the local ledger to Grafana or Splunk. This guide uses the ShadowClaw 1.5.1 tarball published beside the other platform downloads.
Operator edition · 22 October 2026 · ShadowClaw 1.5.1
1. Before you install¶
Use a Linux host with /usr/bin/python3 version 3.9 or later, Bash, and root or sudo access. The standard service path requires systemd running as PID 1 and systemd 240 or later. Python is supplied and maintained by the operating system; the archive does not bundle it or need third-party Python packages.
Plane A reads process and compute information, Plane B attributes network activity, and Plane C collects host events through the kernel process connector and fanotify. Run on the actual host to validate host-wide coverage. A container's namespace, available capabilities and kernel interfaces determine what it can see; installing inside a container does not establish coverage of the host.
The package documentation records smoke tests on Debian 12 and Rocky 9 on arm64. Treat other distributions and kernel configurations as requiring endpoint acceptance. ShadowClaw observes and records; it does not block. This public guide exposes operational settings, not private detection rules, scoring weights or reporting thresholds.
2. Download and verify¶
Download the Linux tarball and SHA256SUMS into a new working folder:
mkdir -p ~/shadowclaw-install-1.5.1
cd ~/shadowclaw-install-1.5.1
curl -fLO https://downloads.shadowclaw.tech/1.5.1/shadowclaw-1.5.1-linux.tar.gz
curl -fLO https://downloads.shadowclaw.tech/1.5.1/SHA256SUMS
sha256sum shadowclaw-1.5.1-linux.tar.gz
cat SHA256SUMS
Compare all 64 hexadecimal characters with the manifest entry for this exact tarball. Do not continue on a mismatch. The archive is distributed by file, not through a signed apt or yum repository; a checksum comparison checks the downloaded bytes.
3. Install and enable the three planes¶
The installer places the CLI at /usr/bin/shadowclaw, the payload at /usr/lib/shadowclaw, and policy at /etc/shadowclaw/shadowclaw.json. It enables and starts shadowclaw.service when systemd is running. The initial policy leaves full host-event collection off; enable it explicitly for all three planes:
sudo cp /etc/shadowclaw/shadowclaw.json \
/etc/shadowclaw/shadowclaw.json.before-guide
sudo /usr/bin/python3 - <<'PY'
import json
from pathlib import Path
p = Path('/etc/shadowclaw/shadowclaw.json')
settings = json.loads(p.read_text())
settings['enable_esf'] = True
p.write_text(json.dumps(settings, indent=2) + '\n')
PY
sudo systemctl restart shadowclaw.service
enable_esf is the cross-platform setting name. On Linux it selects the Linux host-event sources; it does not require Apple's Endpoint Security framework. Full collection depends on the kernel interfaces being available and permitted. A process-only or degraded source must stay marked partial in your acceptance record.
If systemd is not PID 1, the installer places files but does not start a service. Run the foreground command in section 5 under your chosen supervisor and repeat the same coverage checks. Do not assume a successful file installation implies monitoring is active.
4. Verify installation and service status¶
/usr/bin/shadowclaw --version
/usr/bin/shadowclaw --about
/usr/bin/shadowclaw --verify
sudo /usr/bin/shadowclaw --self-test
sudo systemctl is-enabled shadowclaw.service
sudo systemctl status shadowclaw.service --no-pager
sudo tail -n 80 /var/log/shadowclaw/sensor.log
sudo tail -n 80 /var/log/shadowclaw/sensor.err.log
Expect version 1.5.1, all matrix scenarios passing, and an enabled, active service with a PID. Inspect per-plane self-test results and the live startup banner. Confirm the loaded policy path and host-event subscriptions. journalctl -u shadowclaw.service helps with service-manager failures; the unit appends sensor output to the two files above.
The service writes the machine ledger at /var/lib/shadowclaw, protected for root. An unprivileged sensor writes ~/.local/share/shadowclaw. Use sudo to read the service's ledger and avoid confusing a user's history with the machine history. Do not relax the service's filesystem permissions to make the ledger readable to everyone.
5. Test Planes A, B and C¶
What each check establishes¶
| Check | Evidence to keep | What it establishes |
|---|---|---|
--version and --about |
Version and attribution output | The installed command runs the expected payload. |
--verify |
All scenarios pass; command exits successfully | Synthetic inputs exercise the detector logic for Planes A, B, and C, including benign cases. It does not create live ledger findings. |
--self-test |
Per-plane lines, host-plane status, process and socket counts | The current execution context can reach the platform acquisition mechanisms. Read every plane line, even when the final line says PASS. |
| A bounded live run | Startup banner and verbose poll lines | The sensor actually collects processes, connections and host events on this endpoint. |
| A controlled AI session | Time, host, process, provider and matching ledger entry | The real workload reached a recorded finding. |
| Dashboard and destination search | The matching host, time and event | The record reached each configured presentation or central collection path. |
Exercise the three planes¶
Use an endpoint you administer and an approved test account. Before starting, record the host name, time, AI application, provider and expected activity. Use test prompts and disposable files. Keep credentials out of commands and transcripts.
- Plane A - process and compute. During the bounded run below, open your approved local model runtime, if one is available, and run an ordinary inference request. Confirm that the live sample has processes and that verbose polls continue. Look for the runtime in the self-test output and for any corresponding finding. A busy process alone is not a promised detection;
--verifysupplies the repeatable logic check when no model runtime is installed. - Plane B - network. Use an approved AI client to make a real request to its provider while the sensor is running. Record which application and provider you used. Match the process, provider or endpoint, and time in the ledger. A generic website fetch is not an equivalent test of AI-client attribution.
- Plane C - host activity. While the bounded sensor runs, open another terminal and launch a harmless child process with the command shown below. Inspect the host-plane startup line and verbose
esf=event counts. The nameesfis used in the CLI and telemetry across all three platforms. Nonzero counts demonstrate host-event intake; they do not certify every subscription or tactic. Keep anyDEGRADEDorUNAVAILABLEresult as an unresolved acceptance item.
A harmless shell command is not expected to become an agent-tactic finding. The synthetic matrix tests the detection decisions without creating accounts, changing persistence, or touching credentials. To validate a real host-plane alert, use your organization's approved lab exercise and correlate its resulting shadowclaw.agent.activity and finding events. Do not declare that alert path verified solely because the service is running.
Acceptance record¶
Record a separate outcome for each plane: verified, degraded, failed, or not exercised. Retain the version, host, execution identity, test time window, matrix result, per-plane status, ledger verification result, and matching destination event. An empty ledger on an idle host is possible; it is not evidence of full coverage. If a declared workload produces no expected finding, keep that test unresolved and use the troubleshooting section rather than changing detection settings to manufacture a pass.
Run a bounded diagnostic session¶
Stop the service before the foreground run so there is only one writer. This creates a deliberate monitoring interruption in your test window:
sudo systemctl stop shadowclaw.service
sudo /usr/bin/shadowclaw \
--config /etc/shadowclaw/shadowclaw.json \
--ledger-dir /var/lib/shadowclaw \
--esf --verbose --duration 120
While it runs, use the approved AI workload and launch a harmless child process from a second terminal:
Review processes=, connections= and esf= in successive verbose polls. Confirm the startup banner describes the host sources that actually started, including any partial collection. A source that is unavailable cannot be accepted based on synthetic test results. At exit, keep the OTLP sent/failed summary; without a receiver yet, export failures are expected until section 7 or 8 is configured.
Restore the service after the run, including when you interrupted it or a check failed:
6. View the live ledger and built-in dashboard¶
sudo /usr/bin/shadowclaw --ledger path
sudo /usr/bin/shadowclaw --ledger --since 1h
sudo /usr/bin/shadowclaw --ledger watch --since 15m
Match time, process, provider or endpoint, and severity to your workload. Ctrl+C stops the viewer without stopping the sensor. watch is a reader and can run beside the service; launching another sensor is not a substitute for it.
sudo /usr/bin/shadowclaw --ledger verify
sudo /usr/bin/shadowclaw --ledger export \
--since 24h --format html --out /tmp/shadowclaw-report.html
sudo /usr/bin/shadowclaw --dashboard
Open the complete token-bearing URL printed by the dashboard in a browser on the endpoint. It binds to loopback and runs until Ctrl+C. Add --dashboard-port 0 if the default port is occupied. This built-in dashboard reads the local ledger and needs neither Grafana nor a Collector.
For a headless host, use an SSH local forward from your workstation, substituting the actual dashboard port shown by the command:
Then open the printed URL with its local port replaced by 8765, preserving the entire token and path. Leave both the dashboard and SSH sessions running. Do not expose the built-in viewer on a public interface. Treat reports as host-activity data. Hash-chain verification establishes the integrity of stored records, not complete visibility into the endpoint.
7. View detections in Grafana¶
Start the optional local stack¶
The Linux archive does not install Grafana or Loki. You can use your managed service or this public evaluation stack. Install Docker Engine and the Compose plugin for your distribution first. Run the following from a Bash shell under an account permitted to use Docker:
docker compose version
mkdir -p ~/shadowclaw-dashboard
cd ~/shadowclaw-dashboard
curl -fLO https://shadowclaw.tech/assets/user-guides/observability.zip
python3 -m zipfile -e observability.zip .
read -r -s -p 'New Grafana admin password: ' GRAFANA_ADMIN_PASSWORD
printf '\n'
export GRAFANA_ADMIN_PASSWORD
docker compose config --quiet
docker compose up -d
docker compose ps
unset GRAFANA_ADMIN_PASSWORD
Use a strong password and re-supply it for later Compose commands that require configuration interpolation. The stack provisions a Loki datasource and the public operator dashboard. It publishes Grafana on 127.0.0.1:3000 and Loki on 127.0.0.1:3100; if those ports are occupied, resolve the conflict before starting.
Merge these keys into the existing sensor configuration, preserving all other settings:
sudo /usr/bin/python3 - <<'PY'
import json
from pathlib import Path
p = Path('/etc/shadowclaw/shadowclaw.json')
settings = json.loads(p.read_text())
settings.update(otlp_endpoint='http://127.0.0.1:3100/otlp',
emit_otlp_logs=True, emit_otlp_metrics=False)
p.write_text(json.dumps(settings, indent=2) + '\n')
PY
sudo systemctl restart shadowclaw.service
Repeat the approved workload, then inspect its fresh records. On a headless host, forward Grafana with ssh -N -L 3000:127.0.0.1:3000 user@linux-host and browse from your workstation. Check docker compose logs --tail=80 loki grafana if either component does not become available.
Read the Grafana dashboard¶
The optional local stack opens at http://127.0.0.1:3000. Sign in as admin with the password you supplied, then open Dashboards > ShadowClaw > ShadowClaw - operator overview. Its direct path is /d/shadowclaw-operator. The macOS app's native setup installs its own, larger ShadowClaw dashboard; use the URL printed by that setup instead.
Set the time range to Last 15 minutes and refresh. The public operator dashboard shows findings, provider observations, agent activity, and coverage records in separate panels. Expand a log row to inspect its host and process. Use Explore, select the ShadowClaw Loki datasource, and query:
For host-plane observations, change the event filter to shadowclaw.agent.activity. For periodic coverage records, use shadowclaw.ktp.risk_factors and inspect degraded_inputs. An empty agent-activity panel is not a health check. A fresh coverage record shows that logs are arriving; it does not establish full host-plane coverage.
The datasource connects to http://loki:3100 inside the Compose network. The endpoint sensor connects to http://127.0.0.1:3100/otlp on the host. Those addresses serve different callers. The sensor appends /v1/logs. Loki does not accept sensor metrics at that path, so direct-to-Loki configuration must set emit_otlp_metrics to false.
The downloadable stack is for local evaluation, with ports published only on loopback and data in named Docker volumes. It is not a production fleet service. docker compose down stops it and preserves those volumes; adding --volumes deletes the stored dashboard and log data. For a central deployment, use your managed Grafana/Loki service with authentication, TLS, retention and backup policies. Loki's OTLP documentation covers ingestion.
8. Deliver detections centrally with OTLP and Splunk¶
The delivery path¶
Use this path for a central destination such as Splunk:
ShadowClaw sends OTLP/HTTP JSON. Splunk HEC uses a different protocol, so a Collector performs the conversion. Do not point otlp_endpoint at a HEC URL. Use one sensor and one Collector to fan out to multiple destinations; do not start a second sensor against the same ledger.
The sensor accepts plaintext HTTP only on loopback. For a remote OTLP receiver, use an HTTPS base URL with a trusted certificate. Put backend tokens, custom authentication and client-certificate configuration in the Collector. The sensor does not expose those Collector authentication settings.
Prepare Splunk and the Collector¶
- Ask your Splunk administrator for an enabled HEC token permitted to write to a dedicated events index, its exact HTTPS HEC URL, and the appropriate CA trust chain. Use the endpoint supplied for your Splunk Enterprise or Cloud deployment; do not assume a Cloud port or hostname.
- Install the OpenTelemetry Collector Contrib distribution for the host, following its official installation instructions. The
splunk_hecexporter is in Contrib. The example below targets the repository's Collector baseline, 0.158.0; validate it with the version you deploy. - Download collector-splunk.yaml, or save the following complete configuration as that filename. This logs-only path carries all ShadowClaw log event types. Configure the sensor with
emit_otlp_logs: trueandemit_otlp_metrics: falsefor this example.
# ShadowClaw public operator example: logs to a Splunk events index.
# OpenTelemetry Collector Contrib 0.158.0. No detection rules or secrets.
receivers:
otlp:
protocols:
http:
endpoint: 127.0.0.1:4318
processors:
batch:
timeout: 5s
exporters:
splunk_hec:
endpoint: ${env:SPLUNK_HEC_ENDPOINT}
token: ${env:SPLUNK_HEC_TOKEN}
index: ${env:SPLUNK_HEC_INDEX}
source: shadowclaw
sourcetype: shadowclaw:otel
tls:
insecure_skip_verify: false
sending_queue:
enabled: true
queue_size: 1000
retry_on_failure:
enabled: true
initial_interval: 5s
max_interval: 30s
max_elapsed_time: 300s
service:
pipelines:
logs:
receivers: [otlp]
processors: [batch]
exporters: [splunk_hec]
Set SPLUNK_HEC_ENDPOINT, SPLUNK_HEC_INDEX and SPLUNK_HEC_TOKEN in the Collector's environment using the platform commands below. The token stays out of the YAML file. A shell environment belongs to that shell: a Collector installed as a service needs the same values supplied through its service configuration or secret manager.
TLS verification remains enabled. If your HEC certificate uses a private CA, add ca_file: /absolute/path/to/ca.pem under the exporter's tls mapping (use the real local path on Windows). A certificate error is a trust or hostname problem to fix, not a reason to set insecure_skip_verify: true.
See the Contrib HEC exporter reference for supported settings.
Keep Grafana while also sending to Splunk¶
If the local Loki stack is running, add this second exporter to the same Collector file:
exporters:
# Keep the existing splunk_hec exporter here too.
otlp_http/loki:
endpoint: http://127.0.0.1:3100/otlp
Then change service.pipelines.logs.exporters to [splunk_hec, otlp_http/loki]. Merge into the existing exporters mapping; do not create two YAML keys named exporters. Run the Collector on the endpoint host for these loopback addresses to work. The sensor now points only to http://127.0.0.1:4318, and the Collector sends the same logs to both backends. If that port is already owned by DefenseClaw or another Collector, extend the existing pipeline or choose another port and update the sensor to match.
For metrics, add a metrics pipeline and an appropriate metrics backend before setting emit_otlp_metrics back to true. In Splunk, use a separately configured metrics index/exporter. Do not send metrics to the events index in this example or to Loki.
Verify receipt in Splunk¶
First search the events index you configured, over the last 15 minutes:
index=YOUR_EVENTS_INDEX sourcetype="shadowclaw:otel"
| spath
| table _time host event_name process_name pid providers severity
The example exporter sends the sensor's flat JSON body as the event. Find a fresh shadowclaw.ktp.risk_factors record to check the log route even on an idle host. Then perform the approved Plane B workload again and narrow the search:
index=YOUR_EVENTS_INDEX sourcetype="shadowclaw:otel"
| spath
| search event_name="shadowclaw.finding.recorded"
| table _time host process_name pid providers endpoints severity
Match the host, process, provider and time against the local ledger. Use event_name="shadowclaw.agent.activity" for tactics and event_name="shadowclaw.provider.reached" for provider observations. Keep those event types separate when counting detections. If an existing organization pipeline wraps the body, inspect a raw event and adapt the spath input to that wrapper before applying these searches.
The local ledger is written before export. Failed delivery does not automatically replay old ledger entries when connectivity returns. The example Collector has a bounded in-memory queue and retries; it is not durable across Collector restarts. For production, configure persistent Collector storage and an independently tested recovery/backfill procedure. A fresh matching event at the destination is the acceptance criterion, not just a reachable port or a running Collector.
Start the local Collector on Linux¶
After installing otelcol-contrib, open a Bash shell in a dedicated configuration folder. Use the HEC endpoint and index your administrator supplied:
curl -fLO https://shadowclaw.tech/assets/user-guides/collector-splunk.yaml
export SPLUNK_HEC_ENDPOINT='https://splunk.example.com:8088/services/collector'
export SPLUNK_HEC_INDEX='YOUR_EVENTS_INDEX'
read -r -s -p 'Splunk HEC token: ' SPLUNK_HEC_TOKEN
printf '\n'
export SPLUNK_HEC_TOKEN
otelcol-contrib validate --config=collector-splunk.yaml
otelcol-contrib --config=collector-splunk.yaml
Start only after validation succeeds. Keep the Collector running for this evaluation. In another terminal, use the JSON merge command in section 7 but change otlp_endpoint to http://127.0.0.1:4318. Keep logs enabled and metrics disabled for this logs-only pipeline. Restart shadowclaw.service and repeat the workload. Verify both a fresh coverage record and the matching finding in Splunk.
When the foreground Collector stops, unset SPLUNK_HEC_TOKEN. For persistent operation, configure its systemd service with the file path and secret environment, then check its status and journal. Do not start a manual Collector on the same port while that service is active.
9. Troubleshoot by following the record¶
| Symptom | Check and recovery |
|---|---|
| Installer refuses Python | Install the distribution's Python 3.9+ at /usr/bin/python3; a shell alias is not the interpreter used by the service. |
| Installation succeeds but no service runs | Check PID 1. A container without systemd needs its own foreground supervisor and coverage review. |
| Service fails at startup | Read systemctl status, the journal, and the sensor files. Check valid JSON and the existence of the root-owned state/log directories. |
| Plane C partial or unavailable | Read the precise kernel/capability reason. Verify on a supported host with the required kernel interfaces and privileges; preserve the partial result until corrected. |
| No machine findings in the viewer | Use sudo and check --ledger path; ensure you are reading /var/lib/shadowclaw. |
| Grafana says No data | Check the time range, datasource and Loki logs; query coverage records before concluding the detector is idle. |
| OTLP 404 or failed sends | Check the base URL and logs-only setting. Only Loki uses the /otlp suffix. |
| Splunk lacks the matching event | Check Collector errors, HEC authorization/index, CA trust and search window. A healthy local ledger is independent of export success. |
10. Routine operation, upgrade and removal¶
Check service state, fresh coverage records, ledger verification and central receipt regularly. Monitor free space and preserve an acceptance record after kernel, privilege or policy changes. A running service and an empty findings panel do not establish full coverage.
Before upgrading, verify the new archive, preserve the policy and take a consistent ledger backup. Use a SQLite-aware backup or stop the sensor before copying its complete state directory. The tarball installer seeds policy only when absent. On an upgrade, explicitly restart the existing service after installation so the running process uses the new payload, then repeat the acceptance checks.
To stop monitoring and prevent automatic startup:
To remove a tarball installation while retaining evidence and policy:
The ledger at /var/lib/shadowclaw and policy are retained by default. Optional Docker storage and the Collector are managed separately. Do not use evidence-purge options for routine removal. For more detail, read the local ledger, event schemas, and OTLP export.