Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions .github/workflows/testsprite.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@

# TestSprite regression gate — runs the portal suite on every pull request.
# Backend tests target the live app defined in docs/openapi.json,
# regardless of the preview deployment URL.

name: TestSprite

on:
pull_request:
types:
- opened
- synchronize
- reopened
- ready_for_review
workflow_dispatch:

permissions:
contents: read
pull-requests: read

concurrency:
group: testsprite-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
testsprite:
if: >-
github.event_name != 'pull_request' ||
(
github.event.pull_request.head.repo.full_name == github.repository &&
!github.event.pull_request.draft
)

runs-on: ubuntu-latest
timeout-minutes: 30

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "22"

- name: Install TestSprite CLI
run: npm install --global @testsprite/testsprite-cli@latest

- name: Run TestSprite suite
env:
TESTSPRITE_API_KEY: ${{ secrets.TESTSPRITE_API_KEY }}
run: |
testsprite ci run \
--project "aa7e92b6-93ed-43a2-8cdf-9d4889269c86" \
--env "default" \
--timeout 600 \
--fail-on-skipped \
--summary-file testsprite-summary.json

- name: Upload TestSprite summary
if: always()
uses: actions/upload-artifact@v4
with:
name: testsprite-results
path: testsprite-summary.json
if-no-files-found: ignore
40 changes: 25 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,9 @@ Routes:
- `/playground` — the dashboard (timeline, diff, metrics, 3D city)
- `/api/invoices` — the live target API that the test suite exercises

Deploy on Vercel with `framework: Next.js` and no extra configuration. Set
`SECURE_MODE` in the project environment to control the demo vulnerability.
Deploy on Vercel with `framework: Next.js` and no extra configuration. No
environment variables are required: ownership is enforced in code, not behind a
flag.

---

Expand All @@ -107,7 +108,7 @@ ghostinthecodebase/
│ ├─ components/ # Dashboard, AIPanel, CodeDiff, MetricsChart, GhostCity
│ ├─ components/site/ # SiteNav, SiteFooter
│ ├─ data/fixture.ts # recorded demo session
│ ├─ store/invoices.ts # in-memory store + SECURE_MODE gate (BOLA)
│ ├─ store/invoices.ts # in-memory store, ownership-scoped reads
│ ├─ collector/opencode.ts # OpenCode SSE adapter (stub)
│ ├─ collector/shadow-git.ts # commits, diffs and rollback per turn
│ ├─ runner/metrics.ts # polls /api/health into MetricSample
Expand Down Expand Up @@ -157,19 +158,28 @@ Seed data is deterministic: users `alice` and `bob`, invoice `inv-1001` owned by

## What the suite covers

The suite is authored in the TestSprite 3.0 portal against the **deployed URL**
(never localhost) and runs on every pull request through the GitHub App. The
cases below are what it is designed to assert.
13 backend tests in the TestSprite 3.0 project, built from
[`docs/openapi.json`](docs/openapi.json) and executed against the **deployed URL**
(never localhost) by the `TestSprite` workflow on every pull request. The job is a
required check: a red suite blocks the merge.

| Case | Request | Expected when secure |
| --- | --- | --- |
| Owner reads own invoice | `GET /api/invoices/inv-1001`, `x-user-id: alice` | `200` with `inv-1001` |
| Cross-owner read (BOLA) | `GET /api/invoices/inv-1001`, `x-user-id: bob` | `403` |
| List is owner-scoped | `GET /api/invoices`, `x-user-id: bob` | `200`, no `inv-1001` |
| Unknown invoice | `GET /api/invoices/nope`, `x-user-id: alice` | `404` |
| Create invoice | `POST /api/invoices`, valid body | `201` with the new id |

With `SECURE_MODE=off`, the cross-owner case returns `200` and the suite goes red.
| # | Case | Request | Asserts |
| --- | --- | --- | --- |
| 1 | Health probe | `GET /api/health` | `200`, `ok=true`, metrics in range |
| 2 | Leak detected | `GET /api/autopsy` | one `leak` anomaly on `mem`, severity `high` |
| 3 | Turn attribution | `GET /api/autopsy` | anomaly on `t5` blamed on `t4`, names `invoices.ts` |
| 4 | Alice lists | `GET /api/invoices`, alice | only alice's, never `inv-1002` |
| 5 | Bob lists | `GET /api/invoices`, bob | only bob's, never `inv-1001` |
| 6 | List unauthenticated | `GET /api/invoices` | `401`, no data |
| 7 | Owner reads own | `GET /api/invoices/inv-1001`, alice | `200`, seeded fields |
| 8 | Cross-owner read (BOLA) | `GET /api/invoices/inv-1001`, bob | `403`, no payload |
| 9 | Unknown id | `GET /api/invoices/inv-9999` | `404 not_found` |
| 10 | Read unauthenticated | `GET /api/invoices/inv-1001` | `401` |
| 11 | Create | `POST /api/invoices` | `201`, `USD` default, `draft` |
| 12 | Create without customer | `POST /api/invoices` | `422` |
| 13 | Create unauthenticated | `POST /api/invoices` | `401`, nothing created |

Run the same suite locally with `npx @testsprite/testsprite-cli ci run --project aa7e92b6-93ed-43a2-8cdf-9d4889269c86 --env default`.
That is the **catch** half of the demo; flipping the gate to `on` is the **green**
half. No verdict is ever fabricated in this repository — the suite is authored in
the portal against the deployed URL and executed by the GitHub App.
Expand Down
5 changes: 2 additions & 3 deletions app/api/invoices/[id]/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,8 @@ export async function GET(
return NextResponse.json({ error: "not_found" }, { status: 404 });
}

// In SECURE_MODE=on the store returns null for cross-owner reads; in
// SECURE_MODE=off it returns the invoice (the BOLA leak). When the invoice
// exists but is not owned, deny with 403 rather than leaking existence.
// The store reports "exists but not owned" separately from "missing", so a
// cross-owner read is denied with 403 instead of leaking existence via 404.
if (!allowed) {
return NextResponse.json({ error: "forbidden" }, { status: 403 });
}
Expand Down
Loading
Loading