Skip to content

feat(anthropic): Gate prompt collection on data_collection option - #7054

Merged
ericapisani merged 5 commits into
masterfrom
py-2588-anthropic
Aug 7, 2026
Merged

feat(anthropic): Gate prompt collection on data_collection option#7054
ericapisani merged 5 commits into
masterfrom
py-2588-anthropic

Conversation

@ericapisani

Copy link
Copy Markdown
Member

Replace include_prompts and send_default_pii checks with the new data_collection configuration for controlling whether messages and system instructions are captured. Maintain backwards compatibility: when data_collection is not configured, fall back to the legacy pii/include_prompts behavior. Tools are always collected regardless of the message collection setting.

Refs PY-2588

Replace include_prompts and send_default_pii checks with the new data_collection
configuration for controlling whether messages and system instructions are
captured. Maintain backwards compatibility: when data_collection is not
configured, fall back to the legacy pii/include_prompts behavior. Tools are
always collected regardless of the message collection setting.

Refs PY-2588
@ericapisani
ericapisani requested a review from a team as a code owner August 5, 2026 19:49
@linear-code

linear-code Bot commented Aug 5, 2026

Copy link
Copy Markdown

PY-2588

Comment thread sentry_sdk/integrations/anthropic.py
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Codecov Results 📊

99957 passed | ⏭️ 6515 skipped | Total: 106472 | Pass Rate: 93.88% | Execution Time: 358m 35s

📊 Comparison with Base Branch

Metric Change
Total Tests 📈 +728
Passed Tests 📈 +692
Failed Tests
Skipped Tests 📈 +36

All tests are passing successfully.

✅ Patch coverage is 88.46%. Project has 2509 uncovered lines.
❌ Project coverage is 89.97%. Comparing base (base) to head (head).

Files with missing lines (1)
File Patch % Lines
sentry_sdk/integrations/anthropic.py 88.46% ⚠️ 3 Missing and 5 partials
Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
- Coverage    90.00%    89.97%    -0.03%
==========================================
  Files          193       193         —
  Lines        24991     25003       +12
  Branches      9012      9026       +14
==========================================
+ Hits         22490     22494        +4
- Misses        2501      2509        +8
- Partials      1437      1438        +1

Generated by Codecov Action

@alexander-alderman-webb

Copy link
Copy Markdown
Contributor

This is currently failing tests so I'll review when they're green

Comment thread sentry_sdk/integrations/anthropic.py Outdated
@ericapisani

Copy link
Copy Markdown
Member Author

@alexander-alderman-webb Got it sorted - the original tests I had written didn't work on older versions of the Anthropic SDK because the tool property wasn't supported at that point in messages.create.

Should be ready for 👀 now

Comment thread sentry_sdk/integrations/anthropic.py Outdated
@@ -456,23 +487,12 @@ def _set_common_input_data(
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anthropic response text/tool calls ignore data_collection gen_ai.outputs opt-out

This change gates prompt (input) collection on data_collection["gen_ai"]["inputs"], but _set_output_data (line 609) still records GEN_AI_RESPONSE_TEXT and GEN_AI_RESPONSE_TOOL_CALLS based solely on the legacy should_send_default_pii() and integration.include_prompts check, so a user who sets _experiments={"data_collection": {"gen_ai": {"outputs": False}}} while still having send_default_pii=True will keep sending LLM response content — which frequently echoes user PII — to Sentry despite the explicit opt-out and despite the SDK warning that send_default_pii "is ignored when data_collection is set".

Evidence
  • _set_common_input_data (this hunk) now resolves record_inputs from client.options["data_collection"]["gen_ai"]["inputs"], falling back to should_send_default_pii() and integration.include_prompts only when has_data_collection_enabled() is false.
  • _set_output_data (line 609) gates GEN_AI_RESPONSE_TEXT/GEN_AI_RESPONSE_TOOL_CALLS only on should_send_default_pii() and integration.include_prompts; it never reads options["data_collection"]["gen_ai"]["outputs"] (confirmed via grep — no data_collection reference in the output path).
  • data_collection._gen_ai_from_value supports an explicit outputs: False, and _resolve_data_collection documents that data_collection wins over send_default_pii, emitting a DeprecationWarning that send_default_pii "is ignored when data_collection is set."
  • With send_default_pii=True plus data_collection={"gen_ai": {"outputs": False}}, the resolved config says outputs are off, yet the legacy check at line 609 still evaluates true and response text is attached to the span and sent to Sentry.

Identified by Warden · security-review · JXZ-LMZ

@ericapisani
ericapisani merged commit d03110e into master Aug 7, 2026
143 of 145 checks passed
@ericapisani
ericapisani deleted the py-2588-anthropic branch August 7, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants