Status: Sanitized. This is the threat model I use to reason about running semi-autonomous AI agents against a production PHP/MySQL platform whose downtime costs real money. Names, hostnames, credentials, and vendor specifics are omitted. Where a section still needs owner sign-off before publishing, it is marked <!-- REVIEW: ... -->.
Format loosely follows OWASP Threat Modeling: scope → assets → actors → threats → controls → residual risk.
In scope
- A per-tenant containerized agent (LLM host + orchestration harness) executing operator-authored tasks against a snapshot of the reservation DB and a scoped subset of the file system.
- The seam between the container and the production data plane — the DB user, the network path, the credential-injection mechanism, and the write path back into production (contract PDFs, itinerary markup, correspondence).
- The human approval gate that sits between an agent-proposed change and any consequential action.
Out of scope
- LLM provider-side security (their infrastructure, their prompt-injection filters, their training data). Trust boundary sits at the network egress from the container.
- End-user browser security for the public booking portal — separately modeled.
- Physical security of the host.
Ordered by consequence.
| # | Asset | Consequence of compromise |
|---|---|---|
| A1 | Production reservation DB (write access) | Existential business risk. A destructive UPDATE or DELETE at scale takes bookings offline until the last known-good snapshot is restored — hours to days depending on how long ago the write went unnoticed. |
| A2 | Signed / generated contracts and invoices | Legal exposure. A forged or tampered signed PDF sent to a client is a legally-binding document under someone else's authority. |
| A3 | Traveler / minor PII | Regulatory + reputational. School travel programs mean names, DOBs, medical notes, and passport data for minors. |
| A4 | Third-party API credentials (payment, email, SMS, LLM providers) | Financial. A leaked payment-provider key writes charges. A leaked LLM key runs up bills and can exfiltrate anything the compromised agent could see. |
| A5 | Operational continuity | Any outage that spans the booking-cutoff-to-departure window is unrecoverable — the trip runs or it doesn't. |
| A6 | Audit trail | Everything above is downstream of a legible audit trail. Losing per-action attribution turns a recoverable incident into an unbounded one. |
| Actor | Motivation | Access surface |
|---|---|---|
| Non-technical operator (trusted) | Get their work done fast | Browser consoles, agent task authoring, approval gates |
| Autonomous agent (partially trusted) | Execute the task it was given | Whatever the sandbox exposes; whatever the harness authorizes |
| Malicious prompt (untrusted, arriving as ingested content) | Anything an attacker can encode in text the agent will read | Contract templates, inbound email, itinerary source docs, retrieved web pages |
| Compromised dependency (untrusted) | Whatever it was compromised for | Any package pulled into the container image or the harness runtime |
| External attacker (untrusted) | Ransom, data exfiltration, reputational | Public admin console, exposed DB port, exposed shell, leaked credential |
| Insider with legitimate credential (partially trusted) | Malice or mistake | Everything they're already authorized for; blast radius depends on the boundaries |
The threats are ordered by likelihood × consequence. Each maps to one or more controls; the primary ADR reference is in parentheses.
Consequence: A1, A5. Existential.
- Control: Agents write to a periodically-refreshed snapshot of the schema, not the primary. Live replication is deliberately not used. (ADR 0003)
- Control: DB grants are scoped by user and host. A credential taken out of a container has no reach off that container's host. (ADR 0009)
- Control: Every agent has a dedicated scoped system user, not a shared service account — every action is attributable. (ADR 0005)
- Control: Container isolation caps filesystem + process reach at the sandbox boundary. (ADR 0001)
- Residual risk: A write that reaches production still relies on the human approval gate. A social-engineered operator who rubber-stamps the gate is the remaining path.
T2 — Prompt injection via ingested content causes the agent to take an action outside the operator's intent
Consequence: A1, A2, A3, A4.
- Control: The agent never has direct write access to prod. All consequential outputs (contracts, invoices, template edits, external emails) go through a human approval gate with per-action logging.
- Control: For legacy generated markup — where a human reviewer cannot eye-diff — the gate is a pixel-equality assertion: an edit ships only if a headless render of before/after moves zero pixels. Semantic changes never reach the approval queue in the first place. (ADR 0010)
- Control: The signing instrument for legally-binding documents runs self-hosted and is gated by a named human, not any automation. The agent can prepare a document; only a human can sign it. (ADR 0017)
- Control: Ingested content is treated as untrusted input. Retrieved web pages and inbound email are stripped of instruction-shaped structures before they enter the context; the harness logs the ingestion path per action.
- Residual risk: A determined injection that convinces the agent to not flag something ambiguous for review is not fully defended by pixel-equality alone. Operator training on "if the agent surprises you, stop" is the human control.
Consequence: A4, cascading to A1–A3.
- Control: Secrets are runtime-injected, never persisted to plaintext config. Two backends for two audiences (broker for team, SOPS + age for solo fleet); the invariant is the same. (ADR 0018)
- Control: The observability collector seam explicitly does not carry prompt content — only agent usage metrics (token counts, latency, cost). (ADR 0021)
- Control: LLM egress is scoped: each agent has its own provider key, so revocation and cost-attribution work at the per-agent level.
- Residual risk: Anything the agent puts into a prompt is out of scope for the provider's data handling. Operator training says: prompts don't contain plaintext secrets, and the harness scrubs known secret formats on the way out.
Consequence: A1–A4.
- Control: Every consequential tool call is enumerated in the harness's allow-list; nothing is proxied by default.
- Control: Container's outbound network is restricted; the agent cannot reach the internet arbitrarily.
- Control: OpenSSF Scorecard on the security-sensitive libraries (
webhook-verify,webcrypto-envelope,muxboard) surfaces branch-protection, code-review, and signed-release posture as a public badge. Supply-chain surface is minimized further by the "standard-library + zero deps" preference in these libraries. - Residual risk: A compromised LLM provider itself would still exercise every tool the agent is authorized to call. See T2 controls for the human gate.
Consequence: A5, cascading everywhere.
- Control: SSH left the public internet entirely. Operator shells reach the fleet via a private WireGuard mesh. (ADR 0020)
- Control: The browser admin consoles non-technical operators reach by URL stayed public but are MFA-gated per audience. A passkey session gates a second app via forward_auth rather than standing up a second auth stack. (ADR 0019)
- Control: Passkeys everywhere; no SMS second factor for anyone reachable via the ops surface.
- Residual risk: A compromised endpoint device (operator's laptop, malware in the browser) still authenticates as the legitimate operator. MDM + browser-side compensating controls sit outside this repo.
Consequence: A1, A5. The regression enables T1–T5.
- Control: Cluster posture is enforced at admission — OPA/Gatekeeper + a ValidatingAdmissionPolicy. A manifest that drops
readOnlyRootFilesystemorrunAsNonRootis rejected regardless of who authored it, human or agent. (ADR 0016) - Residual risk: The policy set itself must be reviewed with the same rigor as the resources it protects. The
policy/directory lives in code review like any other component.
Consequence: A1, A5, A6.
- Control: Terraform state lives on a different provider than the compute it provisions, so a provider-level outage cannot destroy both simultaneously. (ADR 0015)
- Control: External managed DB over containerized (durable, backup-friendly). (ADR 0002)
- Control: Restore procedures are practiced on a schedule against the snapshot pipeline.
<!-- REVIEW: name your actual cadence — quarterly? monthly? -->. - Residual risk: A restore that hasn't been tested since a schema migration will surprise you. Restore rehearsals must run after every schema-breaking release.
Consequence: A6, cascading everywhere.
- Control: Scoped system user per agent means shell audit logs already attribute per named identity. (ADR 0005)
- Control: Signing instrument keeps its own signed audit log, gated on a named human per action. (ADR 0017)
- Control: Observability collector seam preserves structured events for every consequential action.
- Residual risk: A gap between the harness's per-action log and the DB's row-level history is where questions land during incident review. Cross-referencing must be scripted, not memory-based.
The controls above reduce these but do not eliminate them:
- A human operator who approves a bad change under time pressure. The pixel-equality gate helps for generated markup; for other categories, operator training and pairing on high-consequence approvals is the compensating control.
- A prompt injection that persuades the agent to stay silent. Defense-in-depth via multiple review paths (log + gate + observability) rather than any single filter.
- A supply-chain compromise upstream of what OpenSSF Scorecard covers — e.g. a compromised container base image whose vendor was itself compromised. Base-image pinning + SBOM scanning are the compensating controls; both live outside this repo's scope.
- A schema migration whose restore path is untested. Rehearse restores after schema breaks.
- After any incident, blameless post-mortem revisits this doc.
- Before adding a new category of agent capability (e.g. the first agent that writes to prod DB directly under a specific narrow grant; the first agent that spends money).
- On every major LLM-provider change (new provider, revoked provider, provider-side policy shift).
- On a
<!-- REVIEW: cadence -->calendar-driven review regardless — threat models rot when nothing forces them to be re-read.
Written by Jacob Stephens. Sanitized for public distribution; the underlying system is not open-source and its private repositories are not linked from here.