Skip to content

Latest commit

 

History

History
133 lines (95 loc) · 12.6 KB

File metadata and controls

133 lines (95 loc) · 12.6 KB

Threat model — AI agent sandbox on a revenue-critical legacy system

Status: Sanitized. This is the threat model I use to reason about running semi-autonomous AI agents against a production PHP/MySQL platform whose downtime costs real money. Names, hostnames, credentials, and vendor specifics are omitted. Where a section still needs owner sign-off before publishing, it is marked <!-- REVIEW: ... -->.

Format loosely follows OWASP Threat Modeling: scope → assets → actors → threats → controls → residual risk.


1. Scope

In scope

  • A per-tenant containerized agent (LLM host + orchestration harness) executing operator-authored tasks against a snapshot of the reservation DB and a scoped subset of the file system.
  • The seam between the container and the production data plane — the DB user, the network path, the credential-injection mechanism, and the write path back into production (contract PDFs, itinerary markup, correspondence).
  • The human approval gate that sits between an agent-proposed change and any consequential action.

Out of scope

  • LLM provider-side security (their infrastructure, their prompt-injection filters, their training data). Trust boundary sits at the network egress from the container.
  • End-user browser security for the public booking portal — separately modeled.
  • Physical security of the host.

2. Assets

Ordered by consequence.

# Asset Consequence of compromise
A1 Production reservation DB (write access) Existential business risk. A destructive UPDATE or DELETE at scale takes bookings offline until the last known-good snapshot is restored — hours to days depending on how long ago the write went unnoticed.
A2 Signed / generated contracts and invoices Legal exposure. A forged or tampered signed PDF sent to a client is a legally-binding document under someone else's authority.
A3 Traveler / minor PII Regulatory + reputational. School travel programs mean names, DOBs, medical notes, and passport data for minors.
A4 Third-party API credentials (payment, email, SMS, LLM providers) Financial. A leaked payment-provider key writes charges. A leaked LLM key runs up bills and can exfiltrate anything the compromised agent could see.
A5 Operational continuity Any outage that spans the booking-cutoff-to-departure window is unrecoverable — the trip runs or it doesn't.
A6 Audit trail Everything above is downstream of a legible audit trail. Losing per-action attribution turns a recoverable incident into an unbounded one.

3. Actors

Actor Motivation Access surface
Non-technical operator (trusted) Get their work done fast Browser consoles, agent task authoring, approval gates
Autonomous agent (partially trusted) Execute the task it was given Whatever the sandbox exposes; whatever the harness authorizes
Malicious prompt (untrusted, arriving as ingested content) Anything an attacker can encode in text the agent will read Contract templates, inbound email, itinerary source docs, retrieved web pages
Compromised dependency (untrusted) Whatever it was compromised for Any package pulled into the container image or the harness runtime
External attacker (untrusted) Ransom, data exfiltration, reputational Public admin console, exposed DB port, exposed shell, leaked credential
Insider with legitimate credential (partially trusted) Malice or mistake Everything they're already authorized for; blast radius depends on the boundaries

4. Threats & controls

The threats are ordered by likelihood × consequence. Each maps to one or more controls; the primary ADR reference is in parentheses.

T1 — Agent (or attacker via agent) executes destructive writes on the production DB

Consequence: A1, A5. Existential.

  • Control: Agents write to a periodically-refreshed snapshot of the schema, not the primary. Live replication is deliberately not used. (ADR 0003)
  • Control: DB grants are scoped by user and host. A credential taken out of a container has no reach off that container's host. (ADR 0009)
  • Control: Every agent has a dedicated scoped system user, not a shared service account — every action is attributable. (ADR 0005)
  • Control: Container isolation caps filesystem + process reach at the sandbox boundary. (ADR 0001)
  • Residual risk: A write that reaches production still relies on the human approval gate. A social-engineered operator who rubber-stamps the gate is the remaining path.

T2 — Prompt injection via ingested content causes the agent to take an action outside the operator's intent

Consequence: A1, A2, A3, A4.

  • Control: The agent never has direct write access to prod. All consequential outputs (contracts, invoices, template edits, external emails) go through a human approval gate with per-action logging.
  • Control: For legacy generated markup — where a human reviewer cannot eye-diff — the gate is a pixel-equality assertion: an edit ships only if a headless render of before/after moves zero pixels. Semantic changes never reach the approval queue in the first place. (ADR 0010)
  • Control: The signing instrument for legally-binding documents runs self-hosted and is gated by a named human, not any automation. The agent can prepare a document; only a human can sign it. (ADR 0017)
  • Control: Ingested content is treated as untrusted input. Retrieved web pages and inbound email are stripped of instruction-shaped structures before they enter the context; the harness logs the ingestion path per action.
  • Residual risk: A determined injection that convinces the agent to not flag something ambiguous for review is not fully defended by pixel-equality alone. Operator training on "if the agent surprises you, stop" is the human control.

T3 — Credentials or secrets are leaked from the container to the LLM provider, logs, or an attacker

Consequence: A4, cascading to A1–A3.

  • Control: Secrets are runtime-injected, never persisted to plaintext config. Two backends for two audiences (broker for team, SOPS + age for solo fleet); the invariant is the same. (ADR 0018)
  • Control: The observability collector seam explicitly does not carry prompt content — only agent usage metrics (token counts, latency, cost). (ADR 0021)
  • Control: LLM egress is scoped: each agent has its own provider key, so revocation and cost-attribution work at the per-agent level.
  • Residual risk: Anything the agent puts into a prompt is out of scope for the provider's data handling. Operator training says: prompts don't contain plaintext secrets, and the harness scrubs known secret formats on the way out.

T4 — Compromised dependency or malicious LLM tool call reaches a resource it shouldn't

Consequence: A1–A4.

  • Control: Every consequential tool call is enumerated in the harness's allow-list; nothing is proxied by default.
  • Control: Container's outbound network is restricted; the agent cannot reach the internet arbitrarily.
  • Control: OpenSSF Scorecard on the security-sensitive libraries (webhook-verify, webcrypto-envelope, muxboard) surfaces branch-protection, code-review, and signed-release posture as a public badge. Supply-chain surface is minimized further by the "standard-library + zero deps" preference in these libraries.
  • Residual risk: A compromised LLM provider itself would still exercise every tool the agent is authorized to call. See T2 controls for the human gate.

T5 — Public admin console is compromised (credential theft, session hijack, phishing)

Consequence: A5, cascading everywhere.

  • Control: SSH left the public internet entirely. Operator shells reach the fleet via a private WireGuard mesh. (ADR 0020)
  • Control: The browser admin consoles non-technical operators reach by URL stayed public but are MFA-gated per audience. A passkey session gates a second app via forward_auth rather than standing up a second auth stack. (ADR 0019)
  • Control: Passkeys everywhere; no SMS second factor for anyone reachable via the ops surface.
  • Residual risk: A compromised endpoint device (operator's laptop, malware in the browser) still authenticates as the legitimate operator. MDM + browser-side compensating controls sit outside this repo.

T6 — Cluster posture regresses via a well-intentioned but permissive manifest

Consequence: A1, A5. The regression enables T1–T5.

  • Control: Cluster posture is enforced at admission — OPA/Gatekeeper + a ValidatingAdmissionPolicy. A manifest that drops readOnlyRootFilesystem or runAsNonRoot is rejected regardless of who authored it, human or agent. (ADR 0016)
  • Residual risk: The policy set itself must be reviewed with the same rigor as the resources it protects. The policy/ directory lives in code review like any other component.

T7 — Backup/restore path fails when needed

Consequence: A1, A5, A6.

  • Control: Terraform state lives on a different provider than the compute it provisions, so a provider-level outage cannot destroy both simultaneously. (ADR 0015)
  • Control: External managed DB over containerized (durable, backup-friendly). (ADR 0002)
  • Control: Restore procedures are practiced on a schedule against the snapshot pipeline. <!-- REVIEW: name your actual cadence — quarterly? monthly? -->.
  • Residual risk: A restore that hasn't been tested since a schema migration will surprise you. Restore rehearsals must run after every schema-breaking release.

T8 — Loss of per-action attribution (audit trail gap)

Consequence: A6, cascading everywhere.

  • Control: Scoped system user per agent means shell audit logs already attribute per named identity. (ADR 0005)
  • Control: Signing instrument keeps its own signed audit log, gated on a named human per action. (ADR 0017)
  • Control: Observability collector seam preserves structured events for every consequential action.
  • Residual risk: A gap between the harness's per-action log and the DB's row-level history is where questions land during incident review. Cross-referencing must be scripted, not memory-based.

5. Residual risks (what's still yours to own)

The controls above reduce these but do not eliminate them:

  • A human operator who approves a bad change under time pressure. The pixel-equality gate helps for generated markup; for other categories, operator training and pairing on high-consequence approvals is the compensating control.
  • A prompt injection that persuades the agent to stay silent. Defense-in-depth via multiple review paths (log + gate + observability) rather than any single filter.
  • A supply-chain compromise upstream of what OpenSSF Scorecard covers — e.g. a compromised container base image whose vendor was itself compromised. Base-image pinning + SBOM scanning are the compensating controls; both live outside this repo's scope.
  • A schema migration whose restore path is untested. Rehearse restores after schema breaks.

6. When I'd revisit

  • After any incident, blameless post-mortem revisits this doc.
  • Before adding a new category of agent capability (e.g. the first agent that writes to prod DB directly under a specific narrow grant; the first agent that spends money).
  • On every major LLM-provider change (new provider, revoked provider, provider-side policy shift).
  • On a <!-- REVIEW: cadence --> calendar-driven review regardless — threat models rot when nothing forces them to be re-read.

Written by Jacob Stephens. Sanitized for public distribution; the underlying system is not open-source and its private repositories are not linked from here.