Add LLM app prompt injection skill and prompt

promptbook/prompts.json

@@ -719,6 +719,26 @@
       ],
       "formatNotes": "All three. Point the tool at the build file, Dockerfile or base image (runtime JDK), `application.yml`/`application.properties` (`spring.threads.virtual.enabled`, `server.tomcat.threads.*`, `spring.datasource.hikari.*`, `spring.task.*`), every executor and async configuration class, and the HTTP client config. Copilot: java-virtual-threads path instructions plus those files in agent mode. Claude Code: the java-virtual-threads-spring-guardrails skill plus the config files. Codex: AGENTS.md concurrency section, then the diff.",
       "body": "Task: Decide whether the Spring Boot service I specify should run on Java virtual threads, and if so, enable them safely. Replace the old thread-pool ceiling with explicit limits on every finite resource, keep request context working across async hops, fix only the pinning that matters on our JDK, and finish with a minimal diff plus a load-test plan that compares off and on.\n\nContext placeholders (fill from the repo or ask only if missing):\n- Service root / module: {{APP_PATH}}\n- JDK version (build and runtime image): {{JDK_VERSION}}  (21 | 24 | 25 | other)\n- Spring Boot version: {{BOOT_VERSION}}  (e.g. 3.4, 3.5, 4.0, 4.1)\n- Web stack: {{STACK}}  (Spring MVC servlet | WebFlux | both)\n- Finite resources: {{POOLS}}  (e.g. Hikari 10, Apache HttpClient 20 per route, Redis Lettuce, Kafka producer)\n- Downstream limits: {{DOWNSTREAM_LIMITS}}  (name → max concurrency or rate limit, or unknown)\n- Custom executors: {{EXECUTORS}}  (bean names and purpose, or \"find them\")\n- Context that must propagate: {{CONTEXT}}  (SecurityContext | MDC correlation ID | trace | tenant)\n- Current symptom: {{SYMPTOM}}  (thread exhaustion | high latency under load | pool pending spikes after enabling | none, evaluating)\n- Load-test tool and target: {{LOAD_TEST}}  (Gatling | k6 | JMeter | none yet; target RPS and p99)\n\nRequirements:\n\n1. Inventory\n   - Read the build file, Dockerfile or base image (actual runtime JDK), and `application*.yml|properties` (`spring.threads.virtual.enabled`, `server.tomcat.threads.*`, `spring.datasource.hikari.*`, HTTP client pool settings, `spring.task.*`)\n   - List every executor: auto-configured (`applicationTaskExecutor`, task scheduler, listener containers) and custom `TaskExecutor` / `ExecutorService` beans, with what runs on each and whether it is I/O or CPU bound\n   - List every finite pool with size and acquire timeout, every `ThreadLocal` use, and every `synchronized` block or method that wraps blocking I/O\n   - Note whether {{STACK}} is servlet or reactive and whether any reactive code blocks\n\n2. Decision\n   - State whether the service is I/O bound and thread-limited, with evidence (thread dumps, Tomcat busy threads, latency vs load). If it is CPU bound or a non-blocking WebFlux app, recommend not enabling and stop after the findings\n   - State the JDK verdict: 24+ (no `synchronized` pinning) or 21 to 23 (pinning audit required); recommend JDK 25 if the base image allows\n\n3. Findings (rank by risk: pools with no acquire timeout or sized below expected concurrency > unbounded fan-out or missing downstream limits > missing timeouts on blocking calls > pooled virtual threads or custom executors assumed to move > pinning on hot I/O paths (JDK 21 to 23 only) > ThreadLocal caches and lost context > CPU-bound work on request threads > observability gaps)\n\n4. Fix\n   - Enable `spring.threads.virtual.enabled=true` per environment (profile or ConfigMap), keeping the rollback a single property\n   - Size each pool in {{POOLS}} from the database or downstream capacity in {{DOWNSTREAM_LIMITS}}; set Hikari `connection-timeout` and HTTP pending-acquire timeouts so waits fail fast\n   - Add a `Semaphore` or Resilience4j bulkhead per fragile downstream; no fixed pools of virtual threads\n   - Move blocking custom executors to virtual-thread-per-task (`SimpleAsyncTaskExecutor` with virtual threads and an optional concurrency limit); keep CPU-bound executors on a bounded platform pool\n   - Propagate {{CONTEXT}} with a `TaskDecorator` or Micrometer context propagation; replace `ThreadLocal` caches with shared immutable objects or explicit parameters\n   - On JDK 21 to 23 only: change `synchronized` to `ReentrantLock` (with `unlock()` in `finally`) where the lock is held across blocking I/O on a hot path; on 24+ leave `synchronized` alone\n   - Bound any per-request fan-out and pass the remaining deadline to each child; cancel children when the parent fails\n   - Keep or add timeouts on every JDBC query, HTTP call, lock wait, and queue poll\n\n5. Tests\n   - `@Async` and fan-out test proving SecurityContext, MDC correlation ID, and trace ID survive the hop\n   - Concurrency test that fires more requests than the pool size and asserts acquire timeouts fail fast with a clear error instead of hanging\n   - Bulkhead or semaphore test showing excess calls are rejected or queued as designed\n   - A startup check or test that fails if the runtime JDK is below the stated floor\n\n6. Load-test plan and verification\n   - Off-versus-on comparison with {{LOAD_TEST}} at the same pool sizes: throughput, p95, p99, error rate, `hikaricp.connections.pending` and acquire time, HTTP pool pending, heap and GC, and JFR `jdk.VirtualThreadPinned` events (threshold about 20 ms)\n   - Resource table: resource | size | acquire timeout | downstream capacity | limiter in front\n   - Executor table: executor | before | after | I/O or CPU | concurrency limit\n   - Alerts to add: pool pending and acquire time, bulkhead rejections, heap pressure, p99 latency\n\nConstraints:\n- Minimal diff. Don't rewrite working concurrency code that isn't on a hot blocking path.\n- Don't invent pool sizes, downstream limits, or load numbers. Use placeholders and state assumptions.\n- Don't add a reactive stack or a new concurrency library to \"help\" virtual threads.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete diffs and a checklist.\n- Leave JDK base-image upgrades and alert rule changes as follow-up notes unless the repo already manages them.\n\nOutput format:\n1. Decision (enable or not, and why) and JDK verdict\n2. Findings table: severity | location | issue | fix\n3. Patch (properties + pool limits + limiters + executor changes + context propagation + pinning fixes)\n4. Tests added and their results\n5. Resource and executor tables, load-test plan, and follow-ups\n\nStart from {{APP_PATH}}. Bound the pools and downstreams before turning the property on.\n"
+    },
+    {
+      "id": "harden-llm-app-against-prompt-injection",
+      "title": "Harden an LLM Feature Against Prompt Injection",
+      "whenToUse": "Harden a chat, RAG, or tool-calling LLM feature against direct and indirect prompt injection, excessive tool agency, and unsafe output handling: inventory and trust map, ranked findings, retrieval filtered by caller access, tools in the caller's identity with validated arguments, code-enforced approval, loop and budget caps, output encoding per sink, guards and redacted logging, plus a red-team eval set wired into CI with before/after attack success rates.",
+      "tools": [
+        "copilot",
+        "claude-code",
+        "codex"
+      ],
+      "tags": [
+        "llm",
+        "security",
+        "prompt-injection",
+        "rag",
+        "agents",
+        "owasp-llm"
+      ],
+      "formatNotes": "All three. Point the tool at the prompt templates, the classes that build chat requests (Spring AI `ChatClient` and advisors, LangChain4j AI services, or SDK calls), every tool or function definition, the retrieval and vector-store query code, the code that renders or stores model output, and any eval folder. Copilot: llm-app-security path instructions plus those files in agent mode. Claude Code: the llm-app-prompt-injection-guardrails skill plus the files. Codex: AGENTS.md LLM section, then the diff.",
+      "body": "Task: Harden the LLM feature I specify against prompt injection (direct and indirect), excessive tool agency, and unsafe output handling. Map what the model reads and what it can do, close the paths from untrusted content to tools and output sinks in code, and finish with a minimal diff plus a red-team eval set that runs in CI.\n\nContext placeholders (fill from the repo or ask only if missing):\n- Feature / module root: {{APP_PATH}}\n- Feature type: {{FEATURE}}  (chat | RAG Q&A | summarizer | tool-calling agent | LLM gateway | other)\n- LLM stack: {{STACK}}  (Spring AI | LangChain4j | Azure OpenAI / OpenAI SDK | OpenAI-compatible router | other)\n- Untrusted sources: {{SOURCES}}  (e.g. uploaded files, wiki pages, support tickets, emails, web pages, tool results)\n- Tools exposed to the model: {{TOOLS}}  (name → what it reads or writes, or \"find them\")\n- Tool identity today: {{TOOL_IDENTITY}}  (end-user on-behalf-of token | service principal / managed identity | API key | mixed)\n- Output sinks: {{SINKS}}  (web UI Markdown/HTML | DB write | email or chat message | another service | shell or SQL)\n- Tenancy and access model: {{TENANCY}}  (single tenant | multi-tenant with tenant ID from token | document ACLs)\n- Guard services available: {{GUARDS}}  (e.g. Azure AI Content Safety Prompt Shields, PII detector, none)\n- Eval runner: {{EVAL_RUNNER}}  (JUnit tests | promptfoo | custom harness | none yet)\n\nRequirements:\n\n1. Inventory\n   - List every LLM call site in {{APP_PATH}}: prompt template and version, model and provider, the messages it builds (system, user, tool) and what goes into each\n   - List every tool or function with its schema, what it reads or writes, the identity it runs as, and whether it can send data outside the tenant\n   - List where each model output goes and how it is rendered, parsed, or executed\n   - Note what is logged per call (raw prompts, completions, tool arguments) and where\n\n2. Trust map\n   - Mark every prompt input as trusted (code-built), user (signed-in caller), or untrusted (third-party content, tool output, other agents)\n   - Draw each path from an untrusted source to a tool or an output sink; these paths are the attack surface\n\n3. Findings (rank by risk: untrusted content reaching a write or outbound tool without approval > tools running with an identity broader than the caller > retrieval not filtered by the caller's access > model output reaching HTML, SQL, shell, file path, or URL sinks unencoded > generic tools such as fetch-any-URL, raw SQL, or exec > secrets, internal endpoints, or authorization logic in the system prompt > unbounded agent loops or budgets > missing guards > raw prompts and customer data in logs > no red-team evals)\n\n4. Fix (code first, prompt wording last)\n   - Move untrusted content out of the system message into its own delimited user or tool message, strip delimiter look-alikes from the content, and add one short rule that delimited content is data, not instructions\n   - Remove secrets, internal URLs, and authorization logic from system prompts; enforce those rules in code\n   - Filter retrieval by the caller's tenant and document ACLs before prompting\n   - Switch tools to the caller's identity (on-behalf-of token or row-level filter); take tenant and user IDs from the authenticated principal, never from model arguments\n   - Narrow tool schemas (enums, IDs, bounded strings), validate every argument server-side with authorization checks, host and path allow-lists, size limits, and idempotency keys for writes; replace generic fetch, SQL, or exec tools with specific ones\n   - Add a code-enforced approval step (pending action plus explicit user confirmation showing exact arguments) for sends, deletes, bulk edits, permission changes, and payments\n   - Cap tool rounds, total tokens or cost, and wall-clock time per request; abort on repeated identical tool calls\n   - Encode output for each sink in {{SINKS}}: sanitized Markdown with raw HTML off, remote images disabled or proxied through an allow-list, strict CSP; use structured output with strict parsing where output drives code; never build SQL, shell commands, paths, or URLs from output by concatenation\n   - Wire {{GUARDS}} on user input and retrieved documents, and PII or secret detection on output; log verdicts\n   - Replace raw prompt logging with request ID, user, tenant, template version, document IDs, redacted tool arguments, guard verdicts, and token counts\n\n5. Red-team evals\n   - Direct injection in user input (override instructions, role-play, encoding tricks)\n   - Indirect injection planted in a retrieved document and in a tool result, asking the model to call a tool or leak data\n   - System-prompt extraction attempts\n   - Cross-tenant or other-user data requests\n   - Tool abuse: out-of-scope arguments, other tenant IDs, disallowed hosts, oversized inputs, repeated calls\n   - Output-sink payloads: script tags, Markdown image exfiltration URLs, SQL and shell metacharacters\n   - Each case asserts behaviour (no tool call, tool call rejected, output encoded, refusal, no foreign data), not exact wording; wire them into {{EVAL_RUNNER}} so CI fails on any new successful attack\n\n6. Verification\n   - Run the eval set before and after and report attack success rate per category\n   - Run the normal-task eval or test suite to confirm answer quality and latency didn't regress\n   - Tool table: tool | reads or writes | identity before | identity after | validation added | approval needed\n   - Sink table: sink | before | encoding or parsing after\n\nConstraints:\n- Minimal diff. Don't rewrite prompts or features that aren't on an untrusted-to-tool or untrusted-to-sink path.\n- Don't treat a keyword blocklist or a stronger system prompt as the fix for any finding.\n- Don't invent guard services, tenant models, or eval scores. Use placeholders and state assumptions.\n- Don't log real prompts, completions, or customer data in tests or examples.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete diffs and a checklist.\n- Leave model or provider changes and new guard service rollouts as follow-up notes unless the repo already uses them.\n\nOutput format:\n1. Trust map (sources → prompt messages → tools and sinks) with the attack-surface paths\n2. Findings table: severity | location | issue | fix\n3. Patch (message separation + retrieval filtering + tool identity and validation + approval step + loop limits + output encoding + guards + logging)\n4. Red-team eval cases added and before/after attack success rate per category\n5. Tool and sink tables, plus follow-ups\n\nStart from {{APP_PATH}}. Close the paths from untrusted content to tools and sinks before touching prompt wording.\n"
     }
   ]
 }