test(web): add SEO output snapshot + JSON-LD regression suite (#66) - #67
Merged
Conversation
Adds a Vitest harness in apps/web/ that locks in the textual and structured-data outputs the SEO audit (v1.4.1 batch) protects: - snapshots.test.ts: snapshots robots.txt, sitemap.xml (lastmod normalised), llms.txt (head), llms-full.txt (head). - jsonld.test.ts: structural assertions on the Organization, SoftwareApplication, APIReference, and TechArticle JSON-LD blocks. - invariants.test.ts: cheap string assertions (Disallow entries, llms.txt parent-entity prologue, engines.node, etc.). The suite runs against a stubbed Fumadocs + next/font/google so it completes in a few seconds without booting a real Next.js build. It is wired into .github/workflows/tests.yml so PRs to staging fail when the audit invariants regress. Each invariant is encoded as a soft check until the corresponding fix lands (#60-#65) - the harness does not block CI on a known-pending issue, but emits a console warning so the missing entity is visible in the test output.
martyy-code
force-pushed
the
feat/seo-test-harness
branch
from
August 10, 2026 12:42
fbed5f2 to
191ca85
Compare
Contributor
Author
|
Force-pushed an amended commit ( Changeset — added Type Check (TS5097) — replaced the dotted-path imports with a typed alias shim. The
All test imports now go through this shim, so the dotted directory never appears in a relative path. Lint / Prettier — the harness's own files are formatted (verified locally with Files added or modified in this amended commit
Local checks (all green):
|
This is a complete restructure of the SEO regression harness that
address the audit feedback. The previous iteration shipped tests that
passed but did not prove their value (mutation testing caught gaps),
mixed layers (fixtures vs. helpers vs. re-exports), and used a hack
(`require()` + dotted-path alias) to dodge TS5097 instead of
addressing it properly.
## What changed
**Architecture**
- Fixtures now live under `tests/seo/fixtures/stubs/` (a single
directory for every external module that does not work under
Vitest: Fumadocs, next/font/google, the `collections/server`
virtual module).
- The dotted-path alias hack (`~llms-txt-route`) is gone. We now
use `tsconfig.test.json` which extends the production tsconfig
and only enables `allowImportingTsExtensions` for tests. The
production `tsconfig.json` is no longer polluted.
- The shim `fixtures/llms-routes.ts` is gone. Tests import the
route modules directly with the `.ts` extension; `tsconfig.test.json`
accepts it.
- `fixtures/layout.ts` (a one-line re-export) is gone. Tests
import `JsonLd` and `HeadLinks` from the production layout
directly.
**Layout addition**
- `apps/web/src/app/layout.tsx` now exports a small `HeadLinks`
component so the SEO suite can assert on the `<link rel="sitemap">`,
RSS, and `<meta name="google-site-verification">` tags without
booting the full layout (which depends on `next/font`, Vercel
Analytics, and the Fumadocs UI provider).
**Test discipline**
- `console.warn('pending #NN')` soft assertions are replaced with
`it.todo('... (pending #NN)')`. The Vitest report now shows
pending invariants as named gaps instead of silently passing.
- The snapshot test file is renamed from `snapshots.test.ts` to
`routes.test.ts` so the snapshot lands in
`__snapshots__/routes.test.ts.snap` (Vitest's default naming).
**Coverage**
- New `head.test.ts` asserts on the `<link>` / `<meta>` /
JSON-LD blocks emitted by the layout. Pending invariants for
#63, #64, and #65 are explicit `it.todo()` entries.
- The `robots.txt disallows /llms-full.txt` test is a
hard assertion (after #60 ships it becomes a real fail).
- The `sitemap lastModified is stable` test asserts two
consecutive sitemap bodies are byte-identical (catches
`new Date()` mutations).
**Package hygiene**
- Bumped `vitest` to ^4.1.10, `@vitejs/plugin-react` to ^6.0.5,
`happy-dom` to ^20.11.2 (latest stable, all compatible with
Vite 8 / Node 22).
- Dropped the `schema-org-validator` package mention from the
issue: it is not on npm. The current harness uses string
assertions on JSON-LD blocks; a real validator is left for a
future PR.
## Mutation testing
Three mutations applied manually, all detected by the harness:
1. Renaming `/llms.txt` in robots.txt -> breaks
`disallows every LLM surface`.
2. Changing `baseUrl` to a wrong host -> breaks
`contains the canonical home URL` and
`points crawlers to the sitemap`.
3. Removing `Disallow: /api/` -> breaks robots.txt invariants.
## Local checks
```
pnpm --filter web type-check:test # passes
pnpm --filter web test:run # 21 passed | 11 todo
```
Contributor
Author
|
Force-pushed a refactor commit ( What changed in this push:
Mutation testing (proof of value): Three mutations applied manually — all detected by the harness:
Local checks: Notes:
|
The production `tsc --noEmit` (run by `pnpm turbo type-check` and the Type Check CI workflow) was failing with TS5097 on the test files, because they import route modules with the `.ts` extension and that flag is only enabled by `tsconfig.test.json`. The fix is structural: the prod tsconfig now explicitly includes only `src/**` and excludes `tests/`. The test tsconfig (`tsconfig.test.json`) already covers the test files via `allowImportingTsExtensions`. This keeps the two compilation contexts cleanly separated and means `pnpm turbo type-check` no longer needs to know about the test-only flag.
Contributor
Author
|
Fix Type Check CI: pushed The production
After this, |
This was referenced Aug 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Vitest harness in
apps/web/that locks in the textual and structured-data outputs the v1.4.1 SEO audit protects. The suite runs in a few seconds without booting a real Next.js build, and is wired into.github/workflows/tests.ymlso PRs targetingstagingfail if the audit invariants regress.Resolves #66.
What lands in this PR
apps/web/vitest.config.ts— Vitest config with@vitejs/plugin-reactandhappy-dom. Aliases@/tosrc/, stubscollections/server,fumadocs-core/source{,/lucide-icons},fumadocs-mdx/runtime/server, andnext/font/googleso the SEO tests can import the production routes and layout without a Next.js dev server.apps/web/tests/seo/helpers.ts— Shared helpers:callRoute()for Next route handlers (auto-rendersMetadataRoute.Sitemapto XML andMetadataRoute.Robotsto plain text),renderServerComponent()viareact-dom/server,extractJsonLdBlocks(), andnormaliseSitemap()for snapshot stability.apps/web/tests/seo/snapshots.test.ts— Snapshots forrobots.txt,sitemap.xml(with<lastmod>placeholdered),llms.txt(first 4 KB),llms-full.txt(first 2 KB).apps/web/tests/seo/jsonld.test.ts— Structural assertions on theOrganization,SoftwareApplication,APIReference, andTechArticleJSON-LD blocks.apps/web/tests/seo/invariants.test.ts— Cheap string assertions (Disallow entries, llms.txt parent-entity prologue,engines.node, etc.).apps/web/tests/seo/fixtures/— Three small stubs (loader.ts,collections.ts,next-font-google.ts) and a re-exportlayout.tsso the test fixtures render exactly what the production layout renders.apps/web/package.json— Addedvitest,@vitejs/plugin-react,happy-domdevDeps +test/test:runscripts.apps/web/src/app/layout.tsx— Re-exportsJsonLdso the SEO suite can render the JSON-LD blocks without booting the full layout (which depends onnext/font, Vercel Analytics, and the Fumadocs UI provider — none of which work in happy-dom). One-line additive change..github/workflows/tests.yml— Adds aRun apps/web SEO testsstep to the existing test job.Why each fix (#60–#65) gets stricter assertions after it lands
The invariants are encoded as soft checks (no hard failure, just a
console.warn) until the corresponding fix lands. Each #60–#65 PR that ships a fix should tighten its matching soft check into a hard assertion. Concretely:/llms-full.txtdisallow) — tightenrobots.txt > disallows every LLM surfaceto requireDisallow: /llms-full.txt.assemblyVersion/datePublished/operatingSystem) — tightenSoftwareApplication.operatingSystem matches engines.nodeand add anAPIReference.assemblyVersion === pkg.versionassertion.Organization > names the correct parent(currently soft) andllms.txt > mentions parent entity within first 800 bytes.This way the harness does not block CI on known-pending issues but does fail loudly the moment any of those checks regress.
Local verification
Out of scope
test/test:run(the rootpnpm testalready coversturbo test).packages/errorssource.apps/webroute — only an additiveexport { JsonLd }in the layout.Checklist
pnpm --filter web test:runis green locally (19/19 passing)..github/workflows/tests.ymlupdated to run the suite on every PR.main-branch behaviour, so future diffs are visible in code review..changeset/*.md(apps/web does not publish).stagingperCONTRIBUTING.md.