Skip to content

fix(retention): keep support reports until they are deleted - #10

Merged
veryCrunchy merged 2 commits into
mainfrom
fix/retain-support-reports
Oct 29, 2026
Merged

veryCrunchy merged 2 commits into
mainfrom
fix/retain-support-reports

Conversation

@veryCrunchy

@veryCrunchy veryCrunchy commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Outcome

Support reports, their private conversations, and their encrypted diagnostic ZIPs are now kept until they are deleted. Nothing is deleted because of its age.

  • The per-product RetentionDays setting is removed from product registration and from GET /api/v1/products.
  • The hourly task (renamed PurgeDeleted, also run at startup) only hard-deletes rows whose deleted_at is set by a reporter deletion, a submission cancellation, or a maintainer deletion. If removing the diagnostic object fails, the row is kept so the deletion can be retried.
  • New admin action DELETE /api/v1/admin/reports/{id}: requires the admin session and the X-CSRF-Token header, is written to support_admin_audit as report_deletion_requested before it runs (fails closed if the audit cannot be written), soft-deletes the report (revoking the reporter's link immediately), and removes the diagnostic object through the same path as reporter deletion. Returns 204, or 404 for an unknown or already deleted report. The console has a confirmed Delete report button.
  • Startup refuses to run when the private object root is not writable (create, sync, and remove a probe file). After migrations it logs the object root and the number of live reports whose diagnostic object is missing. It never deletes or changes a report because its object is missing.
  • Portal copy: "Kept until deleted" replaces the expiry dates, and the intake form says: "Your report and diagnostic attachment are kept until you delete them or Obiente Support deletes them after your request is resolved."
  • README, docs/operations.md, docs/threat-model.md, and openapi.yaml describe the new retention, manual deletion, the persistent /data requirement, and a runbook for restoring missing diagnostic objects. The expired-backup deletion schedule and the "retention purge" recovery step are removed.

Compatibility constraint: retentionUntil stays in contract v1

Installed nati.ve apps (support contract version 1) parse retentionUntil in receipts and private status responses as a required, non-null string, and reject a receipt unless it is a timestamp no earlier than createdAt and at most 30 days after it (for a new submission: also after now and at most now + 30 days). Returning null, omitting the field, or returning a later date would make every installed app treat new submissions as failed and break status checks.

So every contract v1 response (receipt, reconcile, private status, reporter reply, and the admin list, detail, update, and message responses) still contains retentionUntil as an RFC 3339 timestamp equal to createdAt + 30 days, computed in the response mapping. This is the same value existing reports already had. openapi.yaml documents it as a deprecated, informational legacy field that is not a deletion date. The contract version is unchanged. A new HTTP contract test pins the field as present, non-null, and equal to createdAt + 30 days on all those endpoints, including for a report created in 2016 whose value passed long ago, and checks that the purge leaves that report in place.

The portal no longer shows retentionUntil as an expiry.

Database migration

005_retain_until_deleted.sql (idempotent, since migrations run on every startup):

  • makes support_reports.retention_until nullable and sets it to NULL for every row, so no existing report is queued for deletion again;
  • adds a partial index on deleted_at for the purge.

The column is kept rather than dropped: if an older image is started against the migrated database, its purge cannot select NULL rows by age, while a dropped column would add a second, irreversible schema change for no benefit. New reports store NULL. Rows that were already purged are gone and do not come back.

Rolling back to an earlier image after this migration is not supported. The old image cannot read NULL retention_until values, so its report reads fail, but it deletes nothing because of age.

Deployment note

The production /data volume must be a persistent volume that the platform reattaches on every redeploy, together with the same database and SUPPORT_DATA_KEY. The Dockerfile VOLUME line alone does not guarantee this. After deploying, check the startup log for private diagnostic objects verified with missing 0. A warning that names the object root with missing above zero means objects are absent; follow "Restoring missing diagnostic objects" in docs/operations.md.

Validation

Run locally on Windows with Go 1.25.5 and Node 22.13.0:

  • go vet ./...: passed.
  • go test -count=1 ./...: passed. Skipped locally: the two PostgreSQL tests in internal/store/postgres_test.go (no database or Docker daemon available) and the read-only directory probe test (not applicable on Windows). CI now runs the PostgreSQL tests against a postgres:17-alpine service through SUPPORT_TEST_DATABASE_URL, and runs the read-only probe test on Linux.
  • frontend: npm ci, npm run lint (vue-tsc), npm test (16 tests, 3 files), npm run build: passed.
  • The container build was not run locally; CI builds it.
  • CI on 62672e0: service (including both PostgreSQL tests against postgres:17-alpine), portal, and container passed. The first CI run caught a fixture bug in the new PostgreSQL tests (hashes shorter than the 32-byte check constraint), fixed in the second commit.

New tests cover the migration clearing existing deadlines (and running twice); long-expired live reports never being purged (memory and PostgreSQL); reporter deletions being purged; an object-delete failure keeping the row and succeeding on retry; admin deletion (session, CSRF, audit entry, object removal, revoked link, repeated, unknown, and malformed IDs); the startup probe refusing an unwritable root; missing-object count logging without changing reports; the retentionUntil v1 contract; and the portal showing "Kept until deleted", the intake sentence, and the confirmed and cancelled admin delete flow.

Limitations

  • nati.ve apps still forget a report's private link on the device 30 days after submission, as before, based on retentionUntil. Keeping it longer needs a client change and a new contract version.
  • Reporters were told their data would expire after 30 days. Existing reports are now kept until deleted; if any user-facing privacy notice states a 30-day expiry, it needs a matching update.

Reports, their conversations, and their encrypted diagnostic objects are
no longer deleted because of their age.

- Remove the per-product retention period. Migration 005 makes
  retention_until nullable and clears it for every existing row; new
  reports store NULL.
- The hourly purge, renamed PurgeDeleted, only hard-deletes rows whose
  deleted_at is set by a reporter, a cancellation, or a maintainer. A
  failed object deletion still keeps the row for a later retry.
- Add DELETE /api/v1/admin/reports/{id}, which requires the admin
  session and CSRF token, is audited as report_deletion_requested, and
  reuses the reporter deletion path. The console gains a confirmed
  "Delete report" action.
- Refuse to start when the private object root is not writable, and log
  the object root and the number of live reports whose diagnostic object
  is missing. Missing objects never change or delete a report.
- Keep retentionUntil in every contract version 1 response as
  createdAt + 30 days, because released clients require it. It is
  documented as a legacy informational field, not a deletion date.
- Show "Kept until deleted" in the portal and document indefinite
  retention, manual deletion, the persistent /data requirement, and a
  runbook for restoring missing diagnostic objects.
- Run the PostgreSQL migration and store tests in CI.
@veryCrunchy
veryCrunchy merged commit 35dac0b into main Oct 29, 2026
4 checks passed
@veryCrunchy
veryCrunchy deleted the fix/retain-support-reports branch October 29, 2026 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant