Skip to content

refactor(test): restructure SEO test harness for clarity and value (#66) - #68

Closed
martyy-code wants to merge 2 commits into
stagingfrom
refactor/seo-tests-cleanup
Closed

refactor(test): restructure SEO test harness for clarity and value (#66)#68
martyy-code wants to merge 2 commits into
stagingfrom
refactor/seo-tests-cleanup

Conversation

@martyy-code

Copy link
Copy Markdown
Contributor

Summary

Replaces the SEO test harness from #67 with a senior-grade restructure. The previous iteration shipped 19 tests that passed locally but did not prove their value, mixed layers (fixtures vs. helpers vs. re-exports), and used a hack (require() + dotted-path alias) to dodge TS5097 instead of addressing it properly. This PR is the cleanup.

What changed

Architecture

  • Fixtures now live under tests/seo/fixtures/stubs/ — a single directory for every external module that does not work under Vitest (Fumadocs, next/font/google, the collections/server virtual module).
  • The dotted-path alias hack (~llms-txt-route) is gone. We now use tsconfig.test.json which extends the production tsconfig and only enables allowImportingTsExtensions for tests. The production tsconfig.json is no longer polluted with test-only path entries.
  • The shim fixtures/llms-routes.ts is gone. Tests import the route modules directly with the .ts extension; tsconfig.test.json accepts it.
  • fixtures/layout.ts (a one-line re-export) is gone. Tests import JsonLd and HeadLinks directly from the production layout.

Layout addition

  • apps/web/src/app/layout.tsx now exports a small HeadLinks component so the SEO suite can assert on <link rel="sitemap">, RSS, and <meta name="google-site-verification"> tags without booting the full layout (which depends on next/font, Vercel Analytics, and the Fumadocs UI provider). The layout's default export still renders <HeadLinks /> as before; the export is additive.

Test discipline

  • console.warn('pending #NN') soft assertions are replaced with it.todo('... (pending #NN)'). The Vitest report now shows pending invariants as named gaps instead of silently passing tests.
  • The snapshot test file is renamed from snapshots.test.ts to routes.test.ts so the snapshot lands in __snapshots__/routes.test.ts.snap (Vitest's default naming) instead of the awkward snapshots.snap.snap.

Coverage

Package hygiene

  • Bumped vitest to ^4.1.10, @vitejs/plugin-react to ^6.0.5, happy-dom to ^20.11.2 (all latest stable, all compatible with Vite 8 / Node 22).
  • Removed the schema-org-validator package mention from the issue body: it is not on npm. The current harness uses structural string assertions on JSON-LD blocks; a real Schema.org validator is left for a future PR.

CI

  • The Type Check workflow now runs pnpm --filter web type-check:test (the new tsconfig) instead of pnpm --filter web type-check (production tsconfig) for the apps/web workspace, since the test tsconfig is the one that knows about .ts extensions.

Mutation testing

Three mutations applied manually, all detected by the harness:

  1. Renaming /llms.txt in robots.tsdisallows every LLM surface fails.
  2. Changing baseUrl to a wrong host → contains the canonical home URL and points crawlers to the sitemap fail.

(Plus the inverse: I confirmed the existing tests still catch a removed Disallow: /api/.)

Local checks

$ pnpm --filter web type-check:test  # passes (clean)
$ pnpm --filter web test:run        # 21 passed | 11 todo

Notes

  • .changeset/seo-test-harness.md is gitignored (per the repo .gitignore). The Require changeset CI check needs the file to be tracked; we either ship a follow-up commit that removes .changeset/ from .gitignore for this branch, or we drop the requirement for apps/web-only PRs. The first option is cleaner; happy to add it here if asked.
  • 11 it.todo() entries map directly to the open issues in the v1.4.1 SEO batch. Each #60, #61, #63, #64, #65 PR should convert the matching it.todo() into a hard it(...) assertion.

claude added 2 commits August 10, 2026 14:42
Adds a Vitest harness in apps/web/ that locks in the textual and
structured-data outputs the SEO audit (v1.4.1 batch) protects:

- snapshots.test.ts: snapshots robots.txt, sitemap.xml (lastmod
  normalised), llms.txt (head), llms-full.txt (head).
- jsonld.test.ts: structural assertions on the Organization,
  SoftwareApplication, APIReference, and TechArticle JSON-LD blocks.
- invariants.test.ts: cheap string assertions (Disallow entries,
  llms.txt parent-entity prologue, engines.node, etc.).

The suite runs against a stubbed Fumadocs + next/font/google so it
completes in a few seconds without booting a real Next.js build. It is
wired into .github/workflows/tests.yml so PRs to staging fail when
the audit invariants regress.

Each invariant is encoded as a soft check until the corresponding
fix lands (#60-#65) - the harness does not block CI on a known-pending
issue, but emits a console warning so the missing entity is visible
in the test output.
This is a complete restructure of the SEO regression harness that
address the audit feedback. The previous iteration shipped tests that
passed but did not prove their value (mutation testing caught gaps),
mixed layers (fixtures vs. helpers vs. re-exports), and used a hack
(`require()` + dotted-path alias) to dodge TS5097 instead of
addressing it properly.

## What changed

**Architecture**
- Fixtures now live under `tests/seo/fixtures/stubs/` (a single
  directory for every external module that does not work under
  Vitest: Fumadocs, next/font/google, the `collections/server`
  virtual module).
- The dotted-path alias hack (`~llms-txt-route`) is gone. We now
  use `tsconfig.test.json` which extends the production tsconfig
  and only enables `allowImportingTsExtensions` for tests. The
  production `tsconfig.json` is no longer polluted.
- The shim `fixtures/llms-routes.ts` is gone. Tests import the
  route modules directly with the `.ts` extension; `tsconfig.test.json`
  accepts it.
- `fixtures/layout.ts` (a one-line re-export) is gone. Tests
  import `JsonLd` and `HeadLinks` from the production layout
  directly.

**Layout addition**
- `apps/web/src/app/layout.tsx` now exports a small `HeadLinks`
  component so the SEO suite can assert on the `<link rel="sitemap">`,
  RSS, and `<meta name="google-site-verification">` tags without
  booting the full layout (which depends on `next/font`, Vercel
  Analytics, and the Fumadocs UI provider).

**Test discipline**
- `console.warn('pending #NN')` soft assertions are replaced with
  `it.todo('... (pending #NN)')`. The Vitest report now shows
  pending invariants as named gaps instead of silently passing.
- The snapshot test file is renamed from `snapshots.test.ts` to
  `routes.test.ts` so the snapshot lands in
  `__snapshots__/routes.test.ts.snap` (Vitest's default naming).

**Coverage**
- New `head.test.ts` asserts on the `<link>` / `<meta>` /
  JSON-LD blocks emitted by the layout. Pending invariants for
  #63, #64, and #65 are explicit `it.todo()` entries.
- The `robots.txt disallows /llms-full.txt` test is a
  hard assertion (after #60 ships it becomes a real fail).
- The `sitemap lastModified is stable` test asserts two
  consecutive sitemap bodies are byte-identical (catches
  `new Date()` mutations).

**Package hygiene**
- Bumped `vitest` to ^4.1.10, `@vitejs/plugin-react` to ^6.0.5,
  `happy-dom` to ^20.11.2 (latest stable, all compatible with
  Vite 8 / Node 22).
- Dropped the `schema-org-validator` package mention from the
  issue: it is not on npm. The current harness uses string
  assertions on JSON-LD blocks; a real validator is left for a
  future PR.

## Mutation testing

Three mutations applied manually, all detected by the harness:

1. Renaming `/llms.txt` in robots.txt -> breaks
   `disallows every LLM surface`.
2. Changing `baseUrl` to a wrong host -> breaks
   `contains the canonical home URL` and
   `points crawlers to the sitemap`.
3. Removing `Disallow: /api/` -> breaks robots.txt invariants.

## Local checks

```
pnpm --filter web type-check:test  # passes
pnpm --filter web test:run        # 21 passed | 11 todo
```
@martyy-code
martyy-code deleted the refactor/seo-tests-cleanup branch August 10, 2026 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants