Skip to content

test(reporting): run the spec's canonical coverage vectors and align the vocabularies - #1738

Merged
bokelley merged 1 commit into
nextfrom
claude/issue-1695-coverage-vectors
Oct 31, 2026
Merged

bokelley merged 1 commit into
nextfrom
claude/issue-1695-coverage-vectors

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

Folds the spec's canonical reporting-subscription vectors (adcontextprotocol/adcp#8616, 023bbd26e) into the Python SDK. Stacked on #1737.

Parity test on the spec vectors

  • tests/fixtures/reporting_subscriptions/spec/ vendors vectors.json (113 cases), case.schema.json, error-codes.json and reason-codes.json. Each is pinned by SHA-256 in the test. The vendored TypeScript table is removed.
  • tzdata pin. Rows are derived with IANA tzdata 2026f, so the dev extra pins tzdata==2026.6 (2026f). The test resets zoneinfo to that distribution and refuses to run on any other release, so the host's zoneinfo can't drift the results.
  • Result: all 113 cases pass, including the 9 expected_error cases, each asserting code and detail.

Disagreements found and fixed

Vector Python before Spec
max-accounts-capacity-per-offering ignored excluded[].offering_id, so an offering-scoped capacity exclusion removed every offering's periods core/reporting-subscription-coverage.json excluded[].offering_id: each entry removes only its own offering's periods
p1w-is-not-a-schedule-duration never validated the selector's period_duration; P1W simply matched no offering core/reporting-schedule.json period_duration pattern has no W (a week is P7D): COVERAGE_INPUT_INVALID / invalid_period_duration

No other Python result disagrees with a vector.

Vocabularies (error-codes.json, reason-codes.json)

  • Typed coverage errors. CoverageInputError (an ADCPError and a ValueError) carries:

    • code, COVERAGE_INPUT_INCOMPLETE or COVERAGE_INPUT_INVALID;
    • detail, one of the 11 spec details;
    • recovery="correctable", with dispatched unset.

    Every coverage raise site maps to its detail.

  • Pins. Tests pin the constants to the vendored files:

    • REPORTING_SUBSCRIPTION_ERROR_CODES (P3) equals error-codes.json's buyer codes, recovery and dispatch;
    • COVERAGE_INPUT_DETAILS equals its detail list;
    • each code's Python class exists;
    • new COVERAGE_RULE_ORDER, COVERAGE_REASON_CODES (20 reasons) and COVERAGE_VOCABULARY_VERSION equal reason-codes.json.
  • Facade (P4) changes to match the spec:

    • SUBSCRIPTION_OPERATOR_SCOPE_REQUIRED now applies exactly when an all-accounts subscription names a destination without operator_domains;
    • errors carry the spec context: would_revoke, delivery_config_id and account_id, and nearest (offerings failing on the fewest attributes, with every failed attribute).

Gaps to note

  • Python has no coverage period budget yet, so it never raises COVERAGE_BUDGET_EXCEEDED.
  • The seller-side codes (REPORTING_STORAGE_NOT_DURABLE, REPORTING_SCHEMA_BEHIND, SUBSCRIPTIONS_REQUIRE_STANDARD_DIMENSIONS, SUBSCRIPTION_ROSTER_UNAVAILABLE) land with P5 and P7.

Tests

  • test_reporting_subscription_parity.py: 113 vectors, the hash pin, both vocabulary pins, and schema validity, skipping the one wire_valid: false case.
  • The facade, consumer, engine, principal, production and public-API suites pass, as do ruff and mypy.

Refs #1695

🤖 Generated with Claude Code

…the vocabularies

The parity test now runs the spec repository's coverage vectors
(adcontextprotocol/adcp#8616, 023bbd26e: 113 cases), vendored under
tests/fixtures/reporting_subscriptions/spec/ and pinned by SHA-256, in
place of the vendored TypeScript table. Rows are derived with IANA tzdata
2026f, so the dev extra pins tzdata==2026.6 and the test resolves zones
from it, never the host's zoneinfo. All 113 cases pass.

Two disagreements with the vectors are fixed:
- excluded[].offering_id was ignored, so an offering-scoped capacity
  exclusion under match "all" removed every offering's periods
  (max-accounts-capacity-per-offering; core/reporting-subscription-coverage.json
  excluded[].offering_id).
- A selector period_duration was never validated, so P1W resolved to no
  offering instead of failing (p1w-is-not-a-schedule-duration;
  core/reporting-schedule.json period_duration has no W designator).

Coverage input errors are typed: CoverageInputError (an ADCPError and a
ValueError) carries code COVERAGE_INPUT_INCOMPLETE or COVERAGE_INPUT_INVALID,
the detail from error-codes.json (missing_account_timezone,
invalid_period_duration, ...) and recovery "correctable". The vocabularies
are pinned against the vendored files: REPORTING_SUBSCRIPTION_ERROR_CODES
and COVERAGE_INPUT_DETAILS against error-codes.json, and the new
COVERAGE_RULE_ORDER and COVERAGE_REASON_CODES against reason-codes.json.

The buyer facade follows error-codes.json: SUBSCRIPTION_OPERATOR_SCOPE_REQUIRED
applies when an all-accounts subscription names a destination without
operator_domains, and the errors carry their context (would_revoke,
delivery_config_id and account_id, nearest with every failed attribute).

Refs #1695

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean diff, no blocking or medium findings.

Checked:

  • No findings at any severity; reviewer verified the change end-to-end.
  • CoverageInputError subclasses both ADCPError and ValueError, preserving existing except ValueError catch sites (_clock.py:115).
  • excluded[].offering_id fix flows correctly through pre-existing SubscriptionException.removes(..., offering_id) scoping (coverage.py:331,341,825).
  • _mismatches + SUBSCRIPTION_OPERATOR_SCOPE_REQUIRED gating on destination is not None per spec, both covered by updated facade tests (facade.py:519-542).
  • Source changes backed by SHA-256-pinned canonical coverage vectors; tzdata==2026.6 zoneinfo reset fails closed on any other release.
  • No stale references to renamed symbols.

Gate checks: high_risk=false; gated_paths=false; no no-auto-approve team match; no prior decision. Decision table rows 1–8 do not fire → row 9 approve.

Non-blocking note (not a finding): new public exports (CoverageInputError, COVERAGE_REASON_CODES/RULE_ORDER/VOCABULARY_VERSION/INPUT_DETAILS) and the ValueError→CoverageInputError change ship under a test(reporting): prefix, which release-please will not release. If these are meant to reach PyPI, a feat in the stack must carry them — worth confirming before merge, but does not block.

Base automatically changed from claude/issue-1695-p4-buyer-facade to next October 31, 2026 14:24
@bokelley
bokelley merged commit 513eda1 into next Oct 31, 2026
4 checks passed
@bokelley
bokelley deleted the claude/issue-1695-coverage-vectors branch October 31, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant