Skip to content

fix: webjs check refuses to run outside an app - #1356

Merged
vivek7405 merged 3 commits into
mainfrom
fix/check-target-not-an-app
Aug 9, 2026
Merged

fix: webjs check refuses to run outside an app#1356
vivek7405 merged 3 commits into
mainfrom
fix/check-target-not-an-app

Conversation

@vivek7405

@vivek7405 vivek7405 commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Closes #1301

Summary

webjs check now refuses to run in a directory that has no app/, exits 1, and names the workspace member apps to run it in. Running it at the monorepo root reported 67 violations, effectively all of them false, which mattered because AGENTS.md tells every agent to run the check and fix what it finds. An agent starting from the repo root was handed 67 findings with no signal that the scope was wrong.

Every rule assumes ONE application: one module graph, one custom-element registry, one runtime. The repo root is none of those, it is a workspace holding two apps plus every package's test suite plus editor fixtures plus the scaffold templates. The tag my-counter was reported as duplicated across a blog component, an editor-plugin fixture, two unit tests, and a type fixture, five files that never load together, so the hazard the rule states cannot occur. The rule is right; the scope it was pointed at was not.

What changed

  • packages/cli/lib/check-target.js (new). Pure guard plus message renderers, the same shape as its siblings lib/node-preflight.js and lib/app-name.js. findCheckTarget(cwd) returns { isApp, workspaceApps }; the predicate is the presence of an app/ DIRECTORY and nothing else. A workspaces key is not part of the predicate (a directory with no app/ is not an app either way), it only enriches the message with the members that ARE apps.
  • packages/cli/bin/webjs.js. The guard sits after the --rules branch (pure documentation, must keep working anywhere) and before checkConventions runs. Exit 1, because an agent gates on the exit status and 0 would read as "clean", the exact false signal this fixes. Under --json the refusal is emitted as JSON carrying no violations key on purpose, so a consumer that ignores the exit code and reads report.violations.length throws rather than being told the workspace is clean.
  • packages/server/src/check.js is untouched. The rule set inside an app is byte-identical before and after; this scopes only where the command may be invoked, so no rule-engine test changes.

Next.js refuses on the identical predicate in packages/next/src/lib/find-pages-dir.ts.

Verification

  • npx webjs check at the root: exit 1, refusal naming ( cd examples/blog && npx webjs check ) and ( cd website && npx webjs check ), zero violation lines.
  • --json at the root: exit 1, error.code === 'NOT_AN_APP', no violations key.
  • --rules at the root: exit 0, rules listed.
  • Both apps still check clean via the CI loop, each exit 0 with webjs check: all checks pass ✓.
  • webjs doctor at the root: unchanged, still exit 0.

Test plan

test/cli/check-target.test.mjs, 13 cases: unit coverage of the predicate (app dir, workspace filtering, yarn's { packages: [...] } form, a file named app, a missing or malformed manifest) and spawned-CLI coverage of the real bin.

Two that carry the weight:

  • The counterfactual. A fixture app with the same tag registered in two components still exits 1 with the no-duplicate-tag violation, proving the guard does not swallow real findings. Reverting the bin wiring at 5227e14e reds three cases including the repo-root one; verified by toggling the file and re-running.
  • The CI drift guard. Parses the app list out of the for app in ...; do loop in the webjs check step of .github/workflows/ci.yml and asserts set equality with the list the refusal derives. Both sides are ['examples/blog', 'website']. This is what a --workspaces flag was rejected in favour of.

Layers that do not apply: browser, e2e, and smoke (this is a CLI argv branch, no DOM, no request path; the spawned-CLI cases already exercise the real binary end to end). Bun parity is not required, judged by running both changed paths through the two greps in .claude/hooks/require-bun-parity-with-runtime-src.sh: lib/check-target.js matches the prefix but none of the runtime-sensitive filenames, and bin/webjs.js is not under a covered prefix at all. The surface is node:fs, node:path, and argv, so there is nothing for a cross-runtime assertion to prove.

Docs

  • AGENTS.md "Code workflow" item 4: run the check from inside an app, with the two commands.
  • packages/cli/AGENTS.md: the webjs check row of the commands table, plus check-target.js in the lib/ module map.
  • framework-dev.md: a section recording the per-app invocation, the two apps, and the drift test.
  • website/app/docs/conventions/page.ts: one paragraph after the invocation block, for the monorepo user outside this repo.

Not changed, deliberately: the HELP.check entry and the AGENTS.md CLI reference block (no flag added, so webjs help check is still accurate), packages/server/AGENTS.md (check.js untouched), and the scaffold templates (every scaffolded app has app/ at its root and its CI runs the check from there, so no scaffolded app can hit the refusal).

Results

  • Full Node suite: 4154 pass, 7 fail. All 7 are pre-existing and none are from this branch: 5 are the known linked-worktree failures (2 listener, 3 elision) that pass in the primary checkout and in CI, and 2 are test/scaffolds/gallery-coverage.test.js reding on a stale loadFrame gallery-manifest entry left when loadFrame stopped being a @webjsdev/core export in fix: loadFrame is undefined from @webjsdev/core/client-router in prod #1346. I reproduced both gallery failures in a clean checkout at 79fc28fc with none of this branch applied. They are in a file this PR does not touch, so I have left them alone; main is currently red on that job independently of this change.
  • Blog e2e: 94/94.
  • Website dogfood boot, prod mode: /, /docs/conventions, /ui, /ui/button all 200, 81 modulepreload hints probed, none broken.
  • Both apps still check clean via the CI loop, each exit 0.
  • webjs doctor at the root still exits 0.

One existing fixture needed shaping. packages/mcp/test/check-report.test.mjs spawns the real bin against a temp directory that wrote only components/broken.ts, so the run refused and the assertion blew up on undefined.length. Its clean-app sibling in the same file already writes app/page.ts; giving the violation fixture one makes the pair consistent, and the assertion is unchanged so it still fails if the projector stops reporting the violation. That file is the only place in the repo that spawns webjs check through the bin, and nothing in .github/workflows, scripts/, .hooks/, or the root package.json invokes the check at a workspace root.

@vivek7405 vivek7405 self-assigned this Aug 9, 2026
@vivek7405

Copy link
Copy Markdown
Collaborator Author

Design rationale: why not exclude test directories, and why no --workspaces flag

Two alternatives look more obvious than the refusal, and both were measured before being dropped.

Excluding test/ and tests/ from the walk. This is the reflex fix, since a lot of the noise came from fixtures. It does not work. Filtering every violation whose path has a test/ or tests/ segment, and re-deriving each no-duplicate-tag group (a pair collapses when one side leaves), still leaves roughly 15 findings: theme-toggle across the blog and the website, the five ui-dialog* tags across the blog and the registry, slow-fact and token-stream across the blog and the scaffold templates, like-button across the templates and the website, plus check.js's own rule prose and mcp-docs.js. None of those are test code. The cause is cross-app scope, so an exclusion removes about three quarters of the noise and leaves an agent in the same position.

It would also break a documented lockstep. The rule's comment says it scans every source file to stay in step with the editor's 9004 diagnostic, which runs over the whole TypeScript program and is not gated on the import graph. A normal tsconfig includes test files, so the editor would underline a tag the CLI called clean. And a duplicate tag inside a test file is a real hazard rather than a false positive: browser tests load in one realm alongside the components under test, and a second customElements.define for the same tag throws there.

A --workspaces loop. It would have to discover which members are apps, and the only workable predicate for that is "the member has an app/ directory", which is the predicate the refusal already needs. So the flag buys no capability, only a second way to spell a loop, and it creates the drift surface: CI would either keep its own loop (two definitions of the app set) or depend on a fresh CLI feature in a required job. The refusal carries the whole payload instead, since it derives the app list from the workspaces globs and prints the per-app commands.

Drift is handled by a test rather than a flag. test/cli/check-target.test.mjs parses the app list out of the for app in ...; do line in the webjs check step of ci.yml and asserts set equality with the list the refusal derives. A third app added to CI is picked up automatically; one dropped from CI reds the test.

One deviation from the plan, deliberate. The predicate uses statSync(...).isDirectory() rather than a bare existsSync. A plain FILE named app is not an application, and existsSync would call it one. It is strictly narrower, it costs nothing, and there is a test for it.

@vivek7405

Copy link
Copy Markdown
Collaborator Author

Fallout: one existing fixture was not app-shaped

The refusal turned up a fixture that had been relying on the old behaviour. packages/mcp/test/check-report.test.mjs:78 spawns the real bin against a temp directory that wrote only components/broken.ts, with no app/, so the run now refuses and report.violations comes back undefined.

Worth being precise about which side was wrong, because the easy read is that the guard is too strict. It is not. That fixture's own clean-app sibling twenty lines above it already writes app/page.ts, so the pair was inconsistent, and the directory it built was not an app by any definition the checker uses. Giving it an app/page.ts makes it one. The assertion is untouched, so it still fails if the projector stops reporting the violation.

That is the only fixture affected. packages/mcp/test/check-report.test.mjs is the only place in the repo that spawns webjs check through the bin, and its third case passes --rules, which the guard is exempt from by construction. Nothing in .github/workflows, scripts/, .hooks/, or the root package.json invokes the check at a workspace root.

Two unrelated failures in the same run, pre-existing on main. test/scaffolds/gallery-coverage.test.js reds on a stale loadFrame manifest entry, left behind when loadFrame stopped being a @webjsdev/core export in #1346. Both cases reproduce in a clean checkout at 79fc28fc with none of this branch applied, so they are not from this change and I have left them alone.

@vivek7405 vivek7405 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the whole diff. The guard itself is right and I found nothing functionally wrong with it: the predicate matches what check.js already applies per-rule, it sits after the --rules branch so documentation still works anywhere, and the rule engine is genuinely untouched, which is what keeps this on the "where the command runs" side of the correctness-only line.

Two problems, both in the docs, and both the same underlying mistake: the change is more visible to an AGENT than the doc edits admit. One is a misplaced insertion that silently reassigned another module's tests. The other is that --json now has a second output shape and no doc surface says so, which matters more than it looks, because --json exists specifically so an agent can consume the result without scraping stdout. The thing I would keep an eye on generally is that pattern: a refusal added to a machine-readable command is a contract change, not just a message.

Comment thread packages/cli/AGENTS.md
Comment thread packages/cli/bin/webjs.js
@vivek7405
vivek7405 marked this pull request as ready for review August 9, 2026 08:27
Running `webjs check` at the monorepo root reported 67 violations, and
effectively all of them were false. That mattered because AGENTS.md tells
every agent to run the check and fix what it finds, so an agent starting
from the repo root was handed 67 findings with no signal that the scope
was wrong, and the likely outcomes were a wasted investigation or renames
in test fixtures to satisfy a checker that should never have looked at
them.

Every rule assumes ONE application: one module graph, one custom-element
registry, one runtime. The repo root is none of those. The tag
`my-counter` was reported as duplicated across a blog component, an
editor-plugin fixture, two unit tests, and a type fixture, five files
that never load together, so the hazard the rule states cannot occur. The
rule is right; the scope it was pointed at was not.

So the command now refuses in any directory with no `app/` directory,
exits 1, and names the workspace member apps to run it in. The rule
engine is untouched, so the rule set inside an app is byte-identical
before and after; this scopes only where the command may be invoked.
Next.js refuses on the same predicate in find-pages-dir.ts.

Closes #1301
`webjs check` now refuses outside an app (#1301), and this fixture wrote
only `components/broken.ts`, so the run refused instead of reporting the
violation it exists to assert. Its clean-app sibling in the same file
already writes `app/page.ts`; this makes the pair consistent.

The assertion is unchanged, so it still fails if the projector stops
reporting the violation.
Two review findings, both on doc surfaces.

The check-target.js entry landed in the MIDDLE of the app-name.js entry,
between its prose and its trailing test listing, so app-name.js lost its
test references and check-target.js claimed three test files that have
nothing to do with it. The module map exists to tell an agent where a
module's tests live, so both halves were wrong. Moved it below.

The refusal also gives `--json` a second output shape, and no doc surface
said so. The one place describing that contract still promised only the
violations array plus a summary count, which is what an agent reads
before writing a consumer.
@vivek7405
vivek7405 force-pushed the fix/check-target-not-an-app branch from 0aa2db6 to 63c63f0 Compare August 9, 2026 09:47
@vivek7405
vivek7405 merged commit aeda5c8 into main Aug 9, 2026
10 checks passed
@vivek7405
vivek7405 deleted the fix/check-target-not-an-app branch August 9, 2026 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

webjs check at the monorepo root reports 61 false violations

1 participant