feat(hygiene): detect folders no README documents, the inverse of a dead ref

agents/conductors/hygiene/AGENTS.md

@@ -78,7 +78,7 @@ names — and they keep reporting even when the repo-array modes are `unscanned`
 | `docs` | `docs/api/*.rst` + `currentmodule` counts across every managed repo shipping a `docs/api/` tree (**surface**) | `/audit_docs` (Heart, imports) |
 | `crlf` | executable scripts (`.sh` + shebang-`755` `.py`) with CRLF — the shebang breaks on Linux/HPC (**debris**, the ranked count); plain `.py` CRLF is reported separately as *cosmetic* (Python reads it fine — don't mass-normalise) | `/refactor` + `.gitattributes eol=lf` |
 | `docstrings` | consecutive module-level triple-quoted expressions separated only by whitespace in user-facing `*_workspace` and `HowTo*` root `*.py` entry scripts and `scripts/**/*.py` files (**finding**) | `/refactor` (mechanically merge each confirmed boundary) |
-| `refs` | file/folder references in user-facing `*_workspace` and `HowTo*` prose (`scripts/**/*.py` docstrings + comments, every `scripts/**/README.md` and `config/**/README.md`, and the top-level README) whose target no longer exists — restructure debt no health sweep can see, since the scripts still run (**finding**). Covers the README idioms a `scripts/`-anchored matcher cannot see: structure-list bullets (``- `slam_pipeline`: ``), slash-less relative folder paths (`data_preparation/imaging`), and config YAML names | `/refactor` (re-point each reference; judge the intended target) |
+| `refs` | file/folder references in user-facing `*_workspace` and `HowTo*` prose (`scripts/**/*.py` docstrings + comments, every `scripts/**/README.md` and `config/**/README.md`, and the top-level README) whose target no longer exists — restructure debt no health sweep can see, since the scripts still run (**finding**). Covers the README idioms a `scripts/`-anchored matcher cannot see: structure-list bullets (``- `slam_pipeline`: ``), slash-less relative folder paths (`data_preparation/imaging`), and config YAML names. Scans the **inverse direction** too: a folder that exists but whose own parent README never names it — a package can ship fully working, with every reference resolving, and still be invisible to a reader browsing the folder list (`interferometer/features/datacube` sat unlisted for three months) | `/refactor` (re-point each dead reference; add an entry for each undocumented folder, sourced from that folder's own README or a script docstring) |
 | `optdeps` | smoke-listed workspace scripts that construct an optional-dependency-gated API (`TransformerNUFFT` → `nufftax`) without the house `find_spec` skip guard, so they hard-fail the CI matrices that omit the extras (**finding**). AST-confirmed — prose mentions don't count; scripts outside `smoke_tests.txt` are never flagged | `/refactor` (add the skip guard) |
 | `extras` | the complement of `optdeps`: an optional dependency a library **declares** (in the `[optional]` extra `mode=release` installs) that the `workspace-validation.yml` **`mode=smoke`** leg never installs (**finding**). The extras chain only reaches each library's own `[jax]`, never a sibling's `[optional]`, so those need hand-adding and silently drift — the symptom is a script red in smoke and **green in release** | `/bug` (add the install; fix the install set, **never** the script) |
 | `config` | library `config/*.yaml` keys missing from the matching workspace config — recursive diff (**surface**) | `/refactor` (mirror keys) |
@@ -97,7 +97,7 @@ pyauto-brain hygiene deps         # dependency-cap surface → /dep_audit
 pyauto-brain hygiene docs         # API-docs surface → /audit_docs
 pyauto-brain hygiene crlf         # CRLF .py files → /refactor
 pyauto-brain hygiene docstrings   # adjacent top-level documentation → /refactor
-pyauto-brain hygiene refs         # dead internal references in workspace prose → /refactor
+pyauto-brain hygiene refs         # folder-list drift in workspace prose → /refactor
 pyauto-brain hygiene optdeps      # smoke-listed scripts missing an optional-dep skip guard → /refactor
 pyauto-brain hygiene extras       # optional deps the smoke CI leg never installs → /bug
 pyauto-brain hygiene config       # library→workspace config drift → /refactor

agents/conductors/hygiene/_hygiene_refs.py

@@ -1,5 +1,14 @@
 #!/usr/bin/env python3
-"""Read-only scanner for DEAD INTERNAL REFERENCES in workspace prose.
+"""Read-only scanner for FOLDER-LIST DRIFT in workspace prose.
+
+Two directions of the same defect, scanned together:
+
+* **dead references** — prose names a file or folder that no longer exists;
+* **undocumented folders** — a folder exists that its own parent README never
+  names (see ``documented_directories``).
+
+The rest of this docstring describes the dead-reference direction, which is the
+older and more intricate of the two.
 
 Workspace/HowTo scripts document themselves by pointing at their siblings —
 "see the ``modeling/start_here.ipynb`` example", "checkout
@@ -173,6 +182,7 @@ class Finding:
     file: str
     line: int
     reference: str
+    kind: str = "dead"
 
 
 def canonical(path: str) -> str:
@@ -527,6 +537,65 @@ def findings_in_file(
     return sorted(findings, key=lambda finding: (finding.line, finding.reference))
 
 
+def documented_directories(repository: Path) -> list[Finding]:
+    """Report example folders that their own parent README never mentions.
+
+    The inverse of the dead-reference scan above: that one starts from prose and
+    asks whether the target exists, this one starts from what exists and asks
+    whether any prose names it. Both directions are the same defect seen from
+    two ends — a folder list that has drifted from the tree — but they fail
+    apart. A package added without touching its parent README leaves every
+    reference resolving perfectly while the folder is invisible to a reader
+    browsing the list. ``interferometer/features/datacube`` sat unlisted that
+    way for three months (autolens_workspace#482), and the audit that found it
+    turned up thirteen more in the same repo.
+
+    Precision comes from the direction of travel: the candidates are real
+    directories read off the filesystem, so the "is this token a path or a
+    parameter name?" ambiguity that constrains the structure-list rule above
+    cannot arise here, and no quorum heuristic is needed. The mention test is
+    deliberately permissive in the other axis — a bare word-boundary search of
+    the whole README — so bold bullets (``- **`name`**``), trailing slashes
+    (``- `name/```) and plain prose all count as documenting the folder. Only a
+    folder named *nowhere at all* is reported.
+
+    A directory is a candidate only if it carries example content (a script,
+    notebook, or its own README); asset and output directories are skipped.
+    """
+    findings: list[Finding] = []
+    scripts = repository / "scripts"
+    if not scripts.is_dir():
+        return findings
+    for readme in sorted(scripts.rglob("README.md")):
+        try:
+            text = readme.read_text(encoding="utf-8", errors="replace")
+        except OSError:
+            continue
+        for child in sorted(readme.parent.iterdir()):
+            if not child.is_dir() or child.name in PRUNED_DIRECTORIES:
+                continue
+            if child.name.startswith((".", "__")):
+                continue
+            has_content = any(
+                any(child.rglob(pattern))
+                for pattern in ("*.py", "*.ipynb", "README.md")
+            )
+            if not has_content:
+                continue
+            if re.search(rf"\b{re.escape(child.name)}\b", text):
+                continue
+            findings.append(
+                Finding(
+                    repo=repository.name,
+                    file=readme.relative_to(repository).as_posix(),
+                    line=1,
+                    reference=f"{child.name}/ (folder exists, README never names it)",
+                    kind="orphan",
+                )
+            )
+    return findings
+
+
 def scan(root: Path) -> tuple[list[Finding], int, int]:
     resolver = Resolver(root)
     repositories = repository_paths(root)
@@ -535,15 +604,19 @@ def scan(root: Path) -> tuple[list[Finding], int, int]:
         resolver.indexes.setdefault(repository.name, RepositoryIndex(repository))
         for path in scanned_files(repository):
             findings.extend(findings_in_file(resolver, repository, path))
+        findings.extend(documented_directories(repository))
     return findings, len(repositories), resolver.suppressed
 
 
 def summary_for(findings: list[Finding], repository_count: int, skipped: int) -> str:
     file_count = len({(finding.repo, finding.file) for finding in findings})
     affected = len({finding.repo for finding in findings})
     file_label = "file" if file_count == 1 else "files"
+    dead = sum(1 for finding in findings if finding.kind == "dead")
+    orphans = sum(1 for finding in findings if finding.kind == "orphan")
     return (
-        f"{len(findings)} dead internal references in {file_count} {file_label} "
+        f"{len(findings)} folder-list defects ({dead} dead references, "
+        f"{orphans} undocumented folders) in {file_count} {file_label} "
         f"across {affected}/{repository_count} repos; "
         f"{skipped} unresolvable refs skipped"
     )
@@ -569,7 +642,8 @@ def render_human(row: dict) -> None:
         if finding["repo"] != repo:
             repo = finding["repo"]
             print(f"  {repo}:")
-        print(f"    {finding['file']}:{finding['line']} -> {finding['reference']}")
+        arrow = "!!" if finding.get("kind") == "orphan" else "->"
+        print(f"    {finding['file']}:{finding['line']} {arrow} {finding['reference']}")
 
 
 def main() -> int:

agents/conductors/hygiene/hygiene.sh

@@ -16,7 +16,7 @@
 #   noise -> /cli_noise_clean (Heart)      deps  -> /dep_audit (Heart)
 #   docs  -> /audit_docs (Heart)           packaging -> clean_slate.sh (Brain)
 #   docstrings -> /refactor (exact findings; Hygiene remains read-only)
-#   refs  -> /refactor (dead internal references in workspace prose)
+#   refs  -> /refactor (folder-list drift in workspace prose: dead refs + undocumented folders)
 #   optdeps -> /refactor (smoke-listed scripts missing an optional-dep skip guard)
 #   extras  -> /bug (optional deps a library declares that the smoke CI leg never installs)
 # The three Heart skills are read-only observation skills — measurement lives in
@@ -34,7 +34,7 @@
 #   hygiene.sh docs            # API-docs pre-scan -> /audit_docs
 #   hygiene.sh crlf            # executable scripts w/ CRLF break on HPC (+ cosmetic .py) -> /refactor
 #   hygiene.sh docstrings      # adjacent top-level script documentation -> /refactor
-#   hygiene.sh refs            # dead internal references in workspace prose -> /refactor
+#   hygiene.sh refs            # folder-list drift in workspace prose -> /refactor
 #   hygiene.sh optdeps         # smoke-listed scripts w/ a gated API but no skip guard -> /refactor
 #   hygiene.sh extras          # optional deps declared by a library but missing from the smoke CI install -> /bug
 #   hygiene.sh config          # library config keys missing downstream + orphan config files -> /refactor
@@ -294,7 +294,9 @@ prescan_extras() {
 # scripts still run, so no health sweep can see it. The stdlib helper resolves
 # each reference against the checked-out repos (scripts/ and notebooks/ are one
 # namespace) and holds precision with documented suppressions; this compact form
-# feeds the default ranked worklist.
+# feeds the default ranked worklist. The same helper scans the inverse direction
+# — a folder that exists but whose own parent README never names it, which is how
+# a package can ship invisible to every reader browsing the folder list.
 prescan_refs() {
   python3 "$HERE/_hygiene_refs.py" --root "$ROOT" --summary
 }
@@ -713,12 +715,15 @@ if [[ "$mode" == "docstrings" ]]; then
   echo
   echo "→ route the mechanical merges to /refactor; Hygiene never edits source."
 elif [[ "$mode" == "refs" ]]; then
-  echo "Dead internal references in workspace prose (read-only scan):"
+  echo "Folder-list drift in workspace prose (read-only scan):"
   python3 "$HERE/_hygiene_refs.py" --root "$ROOT"
   echo
-  echo "→ route the re-points to /refactor; Hygiene never edits source. Each finding is"
+  echo "→ route the re-points to /refactor; Hygiene never edits source. A '->' finding is"
   echo "  the reference AS WRITTEN — judge the intended target before repointing (a moved"
   echo "  file, a file that became a directory, or a reference meant for a sibling repo)."
+  echo "  A '!!' finding is the inverse: the folder exists and the README never names it,"
+  echo "  so the fix is a new entry describing it, sourced from that folder's own README"
+  echo "  or a script docstring — never inferred from the folder name."
 elif [[ "$mode" == "optdeps" ]]; then
   echo "Smoke-listed scripts missing an optional-dependency skip guard (read-only scan):"
   python3 "$HERE/_hygiene_optdeps.py" --root "$ROOT"