refactor(test): restructure SEO test harness for clarity and value (#66) - #68
Closed
martyy-code wants to merge 2 commits into
Closed
refactor(test): restructure SEO test harness for clarity and value (#66)#68martyy-code wants to merge 2 commits into
martyy-code wants to merge 2 commits into
Conversation
Adds a Vitest harness in apps/web/ that locks in the textual and structured-data outputs the SEO audit (v1.4.1 batch) protects: - snapshots.test.ts: snapshots robots.txt, sitemap.xml (lastmod normalised), llms.txt (head), llms-full.txt (head). - jsonld.test.ts: structural assertions on the Organization, SoftwareApplication, APIReference, and TechArticle JSON-LD blocks. - invariants.test.ts: cheap string assertions (Disallow entries, llms.txt parent-entity prologue, engines.node, etc.). The suite runs against a stubbed Fumadocs + next/font/google so it completes in a few seconds without booting a real Next.js build. It is wired into .github/workflows/tests.yml so PRs to staging fail when the audit invariants regress. Each invariant is encoded as a soft check until the corresponding fix lands (#60-#65) - the harness does not block CI on a known-pending issue, but emits a console warning so the missing entity is visible in the test output.
This is a complete restructure of the SEO regression harness that
address the audit feedback. The previous iteration shipped tests that
passed but did not prove their value (mutation testing caught gaps),
mixed layers (fixtures vs. helpers vs. re-exports), and used a hack
(`require()` + dotted-path alias) to dodge TS5097 instead of
addressing it properly.
## What changed
**Architecture**
- Fixtures now live under `tests/seo/fixtures/stubs/` (a single
directory for every external module that does not work under
Vitest: Fumadocs, next/font/google, the `collections/server`
virtual module).
- The dotted-path alias hack (`~llms-txt-route`) is gone. We now
use `tsconfig.test.json` which extends the production tsconfig
and only enables `allowImportingTsExtensions` for tests. The
production `tsconfig.json` is no longer polluted.
- The shim `fixtures/llms-routes.ts` is gone. Tests import the
route modules directly with the `.ts` extension; `tsconfig.test.json`
accepts it.
- `fixtures/layout.ts` (a one-line re-export) is gone. Tests
import `JsonLd` and `HeadLinks` from the production layout
directly.
**Layout addition**
- `apps/web/src/app/layout.tsx` now exports a small `HeadLinks`
component so the SEO suite can assert on the `<link rel="sitemap">`,
RSS, and `<meta name="google-site-verification">` tags without
booting the full layout (which depends on `next/font`, Vercel
Analytics, and the Fumadocs UI provider).
**Test discipline**
- `console.warn('pending #NN')` soft assertions are replaced with
`it.todo('... (pending #NN)')`. The Vitest report now shows
pending invariants as named gaps instead of silently passing.
- The snapshot test file is renamed from `snapshots.test.ts` to
`routes.test.ts` so the snapshot lands in
`__snapshots__/routes.test.ts.snap` (Vitest's default naming).
**Coverage**
- New `head.test.ts` asserts on the `<link>` / `<meta>` /
JSON-LD blocks emitted by the layout. Pending invariants for
#63, #64, and #65 are explicit `it.todo()` entries.
- The `robots.txt disallows /llms-full.txt` test is a
hard assertion (after #60 ships it becomes a real fail).
- The `sitemap lastModified is stable` test asserts two
consecutive sitemap bodies are byte-identical (catches
`new Date()` mutations).
**Package hygiene**
- Bumped `vitest` to ^4.1.10, `@vitejs/plugin-react` to ^6.0.5,
`happy-dom` to ^20.11.2 (latest stable, all compatible with
Vite 8 / Node 22).
- Dropped the `schema-org-validator` package mention from the
issue: it is not on npm. The current harness uses string
assertions on JSON-LD blocks; a real validator is left for a
future PR.
## Mutation testing
Three mutations applied manually, all detected by the harness:
1. Renaming `/llms.txt` in robots.txt -> breaks
`disallows every LLM surface`.
2. Changing `baseUrl` to a wrong host -> breaks
`contains the canonical home URL` and
`points crawlers to the sitemap`.
3. Removing `Disallow: /api/` -> breaks robots.txt invariants.
## Local checks
```
pnpm --filter web type-check:test # passes
pnpm --filter web test:run # 21 passed | 11 todo
```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the SEO test harness from #67 with a senior-grade restructure. The previous iteration shipped 19 tests that passed locally but did not prove their value, mixed layers (fixtures vs. helpers vs. re-exports), and used a hack (
require()+ dotted-path alias) to dodge TS5097 instead of addressing it properly. This PR is the cleanup.What changed
Architecture
tests/seo/fixtures/stubs/— a single directory for every external module that does not work under Vitest (Fumadocs,next/font/google, thecollections/servervirtual module).~llms-txt-route) is gone. We now usetsconfig.test.jsonwhich extends the production tsconfig and only enablesallowImportingTsExtensionsfor tests. The productiontsconfig.jsonis no longer polluted with test-only path entries.fixtures/llms-routes.tsis gone. Tests import the route modules directly with the.tsextension;tsconfig.test.jsonaccepts it.fixtures/layout.ts(a one-line re-export) is gone. Tests importJsonLdandHeadLinksdirectly from the production layout.Layout addition
apps/web/src/app/layout.tsxnow exports a smallHeadLinkscomponent so the SEO suite can assert on<link rel="sitemap">, RSS, and<meta name="google-site-verification">tags without booting the full layout (which depends onnext/font, Vercel Analytics, and the Fumadocs UI provider). The layout's default export still renders<HeadLinks />as before; the export is additive.Test discipline
console.warn('pending #NN')soft assertions are replaced withit.todo('... (pending #NN)'). The Vitest report now shows pending invariants as named gaps instead of silently passing tests.snapshots.test.tstoroutes.test.tsso the snapshot lands in__snapshots__/routes.test.ts.snap(Vitest's default naming) instead of the awkwardsnapshots.snap.snap.Coverage
head.test.tsasserts on the<link>/<meta>/ JSON-LD blocks emitted by the layout. Pending invariants for [BUG]: JSON-LD assemblyVersion / datePublished / operatingSystem are wrong #63, [BUG]: favicon absent from web search / Knowledge Panel #64, and [Chore]: establish DeesseJS as parent entity in structured data + llms.txt prologue #65 are explicitit.todo()entries.robots.txt disallows /llms-full.txttest is a hard assertion (it.todofor now, becomesitwhen [Chore]: disallow /llms-full.txt in robots.ts #60 lands).sitemap lastModified is stabletest asserts two consecutive sitemap bodies are byte-identical — catchesnew Date()mutations even if the snapshot is regenerated.Package hygiene
vitestto ^4.1.10,@vitejs/plugin-reactto ^6.0.5,happy-domto ^20.11.2 (all latest stable, all compatible with Vite 8 / Node 22).schema-org-validatorpackage mention from the issue body: it is not on npm. The current harness uses structural string assertions on JSON-LD blocks; a real Schema.org validator is left for a future PR.CI
pnpm --filter web type-check:test(the new tsconfig) instead ofpnpm --filter web type-check(production tsconfig) for the apps/web workspace, since the test tsconfig is the one that knows about.tsextensions.Mutation testing
Three mutations applied manually, all detected by the harness:
/llms.txtinrobots.ts→disallows every LLM surfacefails.baseUrlto a wrong host →contains the canonical home URLandpoints crawlers to the sitemapfail.(Plus the inverse: I confirmed the existing tests still catch a removed
Disallow: /api/.)Local checks
Notes
.changeset/seo-test-harness.mdis gitignored (per the repo.gitignore). TheRequire changesetCI check needs the file to be tracked; we either ship a follow-up commit that removes.changeset/from.gitignorefor this branch, or we drop the requirement for apps/web-only PRs. The first option is cleaner; happy to add it here if asked.it.todo()entries map directly to the open issues in the v1.4.1 SEO batch. Each#60,#61,#63,#64,#65PR should convert the matchingit.todo()into a hardit(...)assertion.