Add RAG retrieval and grounding evaluation skill and prompt

promptbook/prompts.json

@@ -739,6 +739,26 @@
       ],
       "formatNotes": "All three. Point the tool at the prompt templates, the classes that build chat requests (Spring AI `ChatClient` and advisors, LangChain4j AI services, or SDK calls), every tool or function definition, the retrieval and vector-store query code, the code that renders or stores model output, and any eval folder. Copilot: llm-app-security path instructions plus those files in agent mode. Claude Code: the llm-app-prompt-injection-guardrails skill plus the files. Codex: AGENTS.md LLM section, then the diff.",
       "body": "Task: Harden the LLM feature I specify against prompt injection (direct and indirect), excessive tool agency, and unsafe output handling. Map what the model reads and what it can do, close the paths from untrusted content to tools and output sinks in code, and finish with a minimal diff plus a red-team eval set that runs in CI.\n\nContext placeholders (fill from the repo or ask only if missing):\n- Feature / module root: {{APP_PATH}}\n- Feature type: {{FEATURE}}  (chat | RAG Q&A | summarizer | tool-calling agent | LLM gateway | other)\n- LLM stack: {{STACK}}  (Spring AI | LangChain4j | Azure OpenAI / OpenAI SDK | OpenAI-compatible router | other)\n- Untrusted sources: {{SOURCES}}  (e.g. uploaded files, wiki pages, support tickets, emails, web pages, tool results)\n- Tools exposed to the model: {{TOOLS}}  (name → what it reads or writes, or \"find them\")\n- Tool identity today: {{TOOL_IDENTITY}}  (end-user on-behalf-of token | service principal / managed identity | API key | mixed)\n- Output sinks: {{SINKS}}  (web UI Markdown/HTML | DB write | email or chat message | another service | shell or SQL)\n- Tenancy and access model: {{TENANCY}}  (single tenant | multi-tenant with tenant ID from token | document ACLs)\n- Guard services available: {{GUARDS}}  (e.g. Azure AI Content Safety Prompt Shields, PII detector, none)\n- Eval runner: {{EVAL_RUNNER}}  (JUnit tests | promptfoo | custom harness | none yet)\n\nRequirements:\n\n1. Inventory\n   - List every LLM call site in {{APP_PATH}}: prompt template and version, model and provider, the messages it builds (system, user, tool) and what goes into each\n   - List every tool or function with its schema, what it reads or writes, the identity it runs as, and whether it can send data outside the tenant\n   - List where each model output goes and how it is rendered, parsed, or executed\n   - Note what is logged per call (raw prompts, completions, tool arguments) and where\n\n2. Trust map\n   - Mark every prompt input as trusted (code-built), user (signed-in caller), or untrusted (third-party content, tool output, other agents)\n   - Draw each path from an untrusted source to a tool or an output sink; these paths are the attack surface\n\n3. Findings (rank by risk: untrusted content reaching a write or outbound tool without approval > tools running with an identity broader than the caller > retrieval not filtered by the caller's access > model output reaching HTML, SQL, shell, file path, or URL sinks unencoded > generic tools such as fetch-any-URL, raw SQL, or exec > secrets, internal endpoints, or authorization logic in the system prompt > unbounded agent loops or budgets > missing guards > raw prompts and customer data in logs > no red-team evals)\n\n4. Fix (code first, prompt wording last)\n   - Move untrusted content out of the system message into its own delimited user or tool message, strip delimiter look-alikes from the content, and add one short rule that delimited content is data, not instructions\n   - Remove secrets, internal URLs, and authorization logic from system prompts; enforce those rules in code\n   - Filter retrieval by the caller's tenant and document ACLs before prompting\n   - Switch tools to the caller's identity (on-behalf-of token or row-level filter); take tenant and user IDs from the authenticated principal, never from model arguments\n   - Narrow tool schemas (enums, IDs, bounded strings), validate every argument server-side with authorization checks, host and path allow-lists, size limits, and idempotency keys for writes; replace generic fetch, SQL, or exec tools with specific ones\n   - Add a code-enforced approval step (pending action plus explicit user confirmation showing exact arguments) for sends, deletes, bulk edits, permission changes, and payments\n   - Cap tool rounds, total tokens or cost, and wall-clock time per request; abort on repeated identical tool calls\n   - Encode output for each sink in {{SINKS}}: sanitized Markdown with raw HTML off, remote images disabled or proxied through an allow-list, strict CSP; use structured output with strict parsing where output drives code; never build SQL, shell commands, paths, or URLs from output by concatenation\n   - Wire {{GUARDS}} on user input and retrieved documents, and PII or secret detection on output; log verdicts\n   - Replace raw prompt logging with request ID, user, tenant, template version, document IDs, redacted tool arguments, guard verdicts, and token counts\n\n5. Red-team evals\n   - Direct injection in user input (override instructions, role-play, encoding tricks)\n   - Indirect injection planted in a retrieved document and in a tool result, asking the model to call a tool or leak data\n   - System-prompt extraction attempts\n   - Cross-tenant or other-user data requests\n   - Tool abuse: out-of-scope arguments, other tenant IDs, disallowed hosts, oversized inputs, repeated calls\n   - Output-sink payloads: script tags, Markdown image exfiltration URLs, SQL and shell metacharacters\n   - Each case asserts behaviour (no tool call, tool call rejected, output encoded, refusal, no foreign data), not exact wording; wire them into {{EVAL_RUNNER}} so CI fails on any new successful attack\n\n6. Verification\n   - Run the eval set before and after and report attack success rate per category\n   - Run the normal-task eval or test suite to confirm answer quality and latency didn't regress\n   - Tool table: tool | reads or writes | identity before | identity after | validation added | approval needed\n   - Sink table: sink | before | encoding or parsing after\n\nConstraints:\n- Minimal diff. Don't rewrite prompts or features that aren't on an untrusted-to-tool or untrusted-to-sink path.\n- Don't treat a keyword blocklist or a stronger system prompt as the fix for any finding.\n- Don't invent guard services, tenant models, or eval scores. Use placeholders and state assumptions.\n- Don't log real prompts, completions, or customer data in tests or examples.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete diffs and a checklist.\n- Leave model or provider changes and new guard service rollouts as follow-up notes unless the repo already uses them.\n\nOutput format:\n1. Trust map (sources → prompt messages → tools and sinks) with the attack-surface paths\n2. Findings table: severity | location | issue | fix\n3. Patch (message separation + retrieval filtering + tool identity and validation + approval step + loop limits + output encoding + guards + logging)\n4. Red-team eval cases added and before/after attack success rate per category\n5. Tool and sink tables, plus follow-ups\n\nStart from {{APP_PATH}}. Close the paths from untrusted content to tools and sinks before touching prompt wording.\n"
+    },
+    {
+      "id": "evaluate-rag-retrieval-quality",
+      "title": "Evaluate RAG Retrieval & Grounding Quality",
+      "whenToUse": "Measure whether a RAG feature retrieves the right evidence and answers only from it: golden questions with document-level relevance labels, retrieval-only recall@k, MRR, and nDCG, claim-level faithfulness, citation validity, correct- and false-refusal rates, judge calibration against human labels, per-query latency and cost, a paired comparison for a chunking, embedding, re-ranker, prompt, or model change, stage-tagged failures, and CI regression thresholds with a leakage check.",
+      "tools": [
+        "copilot",
+        "claude-code",
+        "codex"
+      ],
+      "tags": [
+        "llm",
+        "rag",
+        "evals",
+        "retrieval",
+        "grounding",
+        "ci"
+      ],
+      "formatNotes": "All three. Point the tool at the ingest and chunking code, the embedding and vector-store or search client, the retriever and re-ranker, the prompt template and the class that builds the chat request with citations, the eval folder and golden set, and the CI workflow. Copilot: rag-eval path instructions plus those files in agent mode (or save it as `.github/prompts/evaluate-rag-retrieval-quality.prompt.md`). Claude Code: the rag-retrieval-eval-guardrails skill plus the files. Codex: AGENTS.md RAG evals section, then the diff.",
+      "body": "Task: Evaluate the retrieval and grounding quality of the RAG feature I specify. Build or audit a golden question set with labelled evidence, score the retriever on its own, score answers for faithfulness, citation correctness, and refusal when evidence is missing, calibrate any LLM judge against human labels, and finish with a minimal diff that runs these evals in CI with regression thresholds.\n\nContext placeholders (fill from the repo or ask only if missing):\n- RAG module root: {{APP_PATH}}\n- Stack: {{STACK}}  (Spring AI | LangChain4j | LlamaIndex | LangChain | custom)\n- Retriever: {{RETRIEVER}}  (vector store and distance metric | keyword search | hybrid; plus re-ranker or none)\n- Chunking and embedding today: {{CHUNKING}}  (splitter, chunk size, overlap; embedding model and dimensions)\n- Top-k used in the prompt: {{TOP_K}}\n- Corpus snapshot: {{CORPUS}}  (where it lives and how it is versioned)\n- Golden set: {{GOLDEN}}  (e.g. evals/rag/golden.jsonl, or \"none yet\")\n- Change under test: {{CHANGE}}  (e.g. chunk 512→1024, new embedding model, add re-ranker, new prompt, new model, or \"baseline only\")\n- Judge model: {{JUDGE}}  (pinned version, or none)\n- Eval runner and CI: {{EVAL_RUNNER}}  (pytest | JUnit | Ragas or similar library | custom harness; GitHub Actions | Jenkins)\n- Budgets: {{BUDGETS}}  (e.g. p95 latency ≤ 2.5 s, cost per query ≤ $0.004)\n\nRequirements:\n\n1. Inventory\n   - Document the pipeline in {{APP_PATH}}: loaders, splitter and chunk settings, embedding model, index and filters, hybrid weights, {{TOP_K}}, re-ranker, prompt template and version, model, and where citations and refusals are produced\n   - List existing eval code, data, and metrics, and note anything unpinned (model aliases, corpus not versioned, judge unpinned)\n   - Flag any golden or fixture data that contains real customer data or PHI\n\n2. Golden set\n   - Each case: stable `id`, `question`, `relevant_ids` as stable document IDs plus span or required facts (graded 0–3 if possible), short reference answer or required facts, and tags (lookup, multi-hop, comparison, numeric, recent-change, no-answer, ambiguous, past-incident)\n   - At least 10% `no-answer` cases whose answer is not in {{CORPUS}}\n   - A frozen held-out slice that is not used for tuning\n   - Map document-level labels onto current chunks at eval time so a chunking change doesn't break labels\n   - If {{GOLDEN}} is missing, propose 30 starter cases from the corpus and mark them for human review; drop synthetic questions that copy chunk text nearly verbatim\n   - Don't rewrite or delete existing cases; flag questionable ones in the report\n\n3. Retrieval metrics (retriever only, no LLM)\n   - recall@k, MRR, and nDCG@k at {{TOP_K}} (and at 2×{{TOP_K}} to show headroom), context precision at k, and retrieved-context tokens, overall and per tag\n\n4. Generation metrics (end to end)\n   - Claim-level faithfulness against retrieved context only: supported-claim share and the count of answers with any unsupported claim\n   - Citation validity (every cited ID was in the prompt for that request, checked in code), citation precision, and citation recall\n   - Correct-refusal rate on no-answer cases and on cases with forced irrelevant context; false-refusal rate on answerable cases\n   - Answer correctness against the reference or required facts\n   - Per-query retrieval, re-rank, and generation latency (p50/p95), prompt, completion, and context tokens, and cost\n\n5. Judge calibration (only if a judge is used)\n   - Hand-label 100–200 answers with a written binary rubric per claim, split dev and held-out\n   - Report judge precision, recall, and Cohen's kappa on held-out labels; pin judge model, prompt, temperature, and parser\n   - Keep judge metrics warn-only until agreement is acceptable; the model under test never judges itself\n\n6. Compare {{CHANGE}}\n   - Same golden set, same corpus snapshot, same {{TOP_K}} unless top-k is the change; full re-embed for embedding changes\n   - Paired per-case comparison against the current configuration, per tag, with token, latency, and cost deltas\n   - Stage-tag every failure: not retrieved, ranked below k, in context but ignored, unsupported claim, wrong citation, should have refused\n\n7. CI gate\n   - Commit the accepted run as a baseline (separate commit, with the pins it came from)\n   - Per-metric floor and tolerance: recall@k, MRR, faithfulness, citation validity, correct-refusal rate; budgets from {{BUDGETS}} for p95 latency and cost per query\n   - Deterministic retrieval and citation-validity metrics block now; calibrated judge metrics block once step 5 passes\n   - Path-filter the job on retrieval, ingest, prompt, model config, and eval code; write a results file with pins and a short Markdown summary; exit non-zero on regression\n\n8. Leakage checks\n   - Confirm no golden question or reference answer appears in prompts, few-shot examples, fine-tuning data, or re-ranker or embedder training data; add a test that fails if one does\n\nConstraints:\n- Minimal diff. Don't change retrieval or prompt behaviour unless {{CHANGE}} asks for it; report findings instead.\n- Don't edit the golden set, labels, or baseline in the same commit as a retrieval, prompt, or model change.\n- Don't invent scores, labels, or corpus contents. Use placeholders and state assumptions.\n- Don't put real customer data, PHI, or secrets in fixtures, logs, or reports.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete diffs and a checklist.\n- Leave index migrations and model swaps as follow-up notes unless they are {{CHANGE}}.\n\nOutput format:\n1. Pipeline summary with pins and unpinned items\n2. Golden set report: case counts per tag, no-answer share, held-out size, flagged cases\n3. Metrics table: metric | baseline | candidate | delta | per-tag notes (retrieval, generation, refusal, cost, latency)\n4. Judge calibration table (if used): metric | precision | recall | kappa | blocking yes/no\n5. Stage-tagged failure table: case id | stage | evidence | suggested fix\n6. Patch (eval harness, metrics, gate config, CI wiring, leakage test) plus follow-ups\n\nStart from {{APP_PATH}}. Score the retriever on its own before judging any answer.\n"
     }
   ]
 }