feat(cli): generate the CLI reference tables in cli/README.md from the option definitions by mbiernacik · Pull Request #2419 · dataform-co/dataform

mbiernacik · 2026-11-03T17:42:57Z

What

The command list, usage lines and argument/option tables in cli/README.md are now generated from the same command/option definitions that the CLI registers with yargs. They live between invisible markers:

<!-- BEGIN GENERATED: options run -->
| Flag | Type | Default | Description |
...
<!-- END GENERATED: options run -->

Everything outside the markers (command descriptions, Notes, exit codes, credentials file, examples) stays hand-written and is never touched. The markers don't render on GitHub or npm.

bazel run //cli:update_readme regenerates the blocks.

//cli:readme_generated_test fails if the README is out of date. It shows the differing lines with their README line numbers and explains how to fix them:

Block "options run" (cli/README.md, lines 248-266):
  README (line 250):  | `--dry-run` | string  | — | Validates the run SQL ... |
  expected from cli/: | `--dry-run` | boolean | — | Validates the run SQL ... |

A new command without a README section fails with a section skeleton to paste, so a command can't be added without documentation.

Changes

Generator (cli/readme_generator.ts, separate library, not bundled into @dataform/cli)

Blocks: commands, usage <cmd>, positionals <cmd>, options <cmd>, project-config-options.
Data comes from name, option.type, option.default and option.describe, plus a new optional INamedOption.docs field (default, required) for values computed or validated at runtime (e.g. --timeout's default comes from DEFAULT_COMPILATION_TIMEOUT_MILLIS).
CLI_COMMANDS in cli/commands/index.ts is the single list of commands used by runCli() and the generator.

Option definitions (user-visible in dataform help)

Fixed inaccurate descriptions: --actions claimed * wildcard support, but matchPatterns() matches exactly; --vars referenced the wrong key; format --actions described action names, but it takes file globs (now its own option object, same flag name, no behaviour change).
Clarified unclear descriptions and documented flag dependencies in words ("Mutually exclusive with --json.", "Requires --actions or --tags."). The existing checks are unchanged.
Reordered options to match the README.
Added type: "string" to --credentials and the project-config options that had no type. Previously yargs parsed numeric-looking values as numbers (e.g. --credentials 1e3 → 1000, --schema-suffix 123 → 123); unset options are unaffected.

README

Generated blocks replace the hand-written tables; per-row details moved to "Notes" under each table.
Removed "Other shared flags" (a summary of the per-command tables that would drift).

CI

The README tests (tag cli_docs) run as the last step of the GitHub Actions build-linux job, which already builds //cli for every PR, including forks. Cloud Build only runs for external contributors after /gcbrun and its logs aren't public.
They're excluded from Cloud Build (scripts/run_tests_on_cloudbuild) to avoid running them twice; a local ./scripts/run_tests still runs them.
contributing.md documents the workflow.

Testing

bazel test //cli/... --test_tag_filters=-integration: all pass, including the new readme_generator_test (generator unit tests on fixtures: each block kind, project-config collapsing, escaping, idempotency, error messages) and readme_generated_test.
bazel run //cli:update_readme is idempotent.
Hand-editing a generated row fails the test with the diff above; update_readme restores the file byte for byte.
The Cloud Build tag filter skips exactly the two README tests.
./scripts/run help <command> shows the updated descriptions.

For reviewers

The user-visible dataform help text changes are in cli/common_options.ts, cli/commands/*_options.ts, cli/commands/format_command.ts and cli/project_config_options.ts.
Repo admins: consider marking the build-linux job as a required check on main, so an out-of-date README blocks merging.

…finitions

mbiernacik · 2026-11-03T17:47:18Z

Since #2418 modifies both the CLI and README.md, one of these PRs must be adjusted to align with the other.

rafal-hawrylak · 2026-11-04T06:45:40Z

          name: windows-smoke-test
          path: bazel-bin/cli/tests/windows/smoke_test_bundle.js

+      # Reuses the //cli build above. Kept last so that a failure doesn't block the Windows 


nit: remove traliing spaces

The step is gone now (the README tests run in Cloud Build again, see the run_tests_on_cloudbuild thread), so test.yaml is back to its state on main.

rafal-hawrylak · 2026-11-04T06:51:47Z

-./scripts/run_tests
+# Run unit tests. The cli/README.md tests (tag cli_docs) are skipped here: they run on GitHub
+# Actions (build-linux job in .github/workflows/test.yaml) for every PR, including forks.
+./scripts/run_tests --test_tag_filters=-integration,-cli_docs


Excluding -cli_docs here removes readme_tests from test-pull-request-trigger, which is the required branch protection check on main (whereas build-linux in .github/workflows/test.yaml is not a required check). Since readme_tests takes around 100ms, please keep it enabled in scripts/run_tests_on_cloudbuild so stale READMEs block merging out of the box without changing repo branch protection settings.

Good point, I missed that test-pull-request-trigger is the required check. Reverted:

the README tests run in Cloud Build again (scripts/run_tests_on_cloudbuild is back to its state on main),

the GitHub Actions step and the cli_docs tag are removed,

contributing.md now points to the unit tests instead of the build-linux job.

So there's a single place for this check, and it blocks merging without any branch protection changes.

rafal-hawrylak · 2026-11-04T06:52:43Z

          name: windows-smoke-test
          path: bazel-bin/cli/tests/windows/smoke_test_bundle.js

+      # Reuses the //cli build above. Kept last so that a failure doesn't block the Windows 


The comment states Kept last so that a failure doesn't block the Windows smoke tests, which is inaccurate: test-windows declares needs: build-linux, so if the last step of build-linux fails, GitHub Actions skips test-windows altogether. Either move the README check into its own job/step after test-windows, or update the comment.

You're right, with needs: build-linux a failure in that step would skip test-windows. The step is removed now, so this no longer applies.

rafal-hawrylak · 2026-11-04T06:55:29Z

+// first (so that the escapes added below can't be altered), then characters that markdown or GFM
+// tables would interpret (`|` ends the cell, `*` and `` ` `` format text), then HTML (e.g. the
+// "<job-prefix>" in "dataform-<job-prefix>-" would otherwise be hidden as an unknown tag).
+function textCell(text: string): string {


textCell escapes |, * and ` so plain-text option descriptions are not interpreted as Markdown, but it does not escape _ or [ and ] (which Markdown interprets as italics text or links text). Consider including _, [ and ] in the escaped character set in textCell.

I also wonder if there is out of the shelf lib that could help us here. Before adding one - please conversate.

Done. textCell now also escapes [, ] and ~ (GFM strikethrough), plus _ at word boundaries. An intraword _ (e.g. workflow_settings.yaml) can't start or end emphasis in CommonMark/GFM, so it's left as is to keep the README source readable. URLs are left as is too, since a backslash would break GitHub's autolinking. Tests cover all of these.

On a library: I'd rather not add a runtime dependency for this:

escaping-only packages (e.g. markdown-escape) are a few lines and wouldn't add much,

mdast-util-to-markdown pulls in the unified ecosystem for a single function,

markdown-it is only in the lockfile as a transitive dependency of jsdoc.

What might be worth it is markdown-it as a devDependency for a round-trip test: render the generated tables and check that each cell's text equals the option's describe. That would guarantee the escaping instead of relying on a character list. WDYT? Happy to do it in this PR or as a follow-up.

rafal-hawrylak · 2026-11-04T06:58:26Z


 export const emitLineageOption: INamedOption<yargs.Options, IRunArgs> = {
  name: "emit-lineage",
  option: {


In emitLineageOption, the parenthetical (requires datalineage.googleapis.com enabled and roles/datalineage.producer, see docs/lineage.md) prints a repo-relative path in dataform help run and duplicates the Notes bullet in cli/README.md (consider moving the prerequisites to Notes like in compile and format). In jitTimeoutOption, consider replacing the internal proto field name jitCode with user-facing wording like Each JiT action.

Done:

--emit-lineage: the prerequisites (Data Lineage API and roles/datalineage.producer) moved to the run Notes in the README. The help text now ends with an absolute link to the lineage guide instead of the repo-relative path, since index_help_test expects the help to point to the guide.

--jit-timeout: now says "Each JiT action gets its own fresh deadline" instead of referring to jitCode.

rafal-hawrylak · 2026-11-04T06:58:37Z

 export const jitTimeoutOption: INamedOption<yargs.Options, IRunArgs> = {
  name: "jit-timeout",
  option: {
    describe:


In emitLineageOption, the parenthetical (requires datalineage.googleapis.com enabled and roles/datalineage.producer, see docs/lineage.md) prints a repo-relative path in dataform help run and duplicates the Notes bullet in cli/README.md (consider moving the prerequisites to Notes like in compile and format). In jitTimeoutOption, consider replacing the internal proto field name jitCode with user-facing wording like Each JiT action.

Same as above, addressed together with the --emit-lineage comment.

mbiernacik added 4 commits November 2, 2026 19:06

Add generator for the CLI reference tables in cli/README.md

d45b7be

Generate the CLI reference tables in cli/README.md from the option de…

24b482e

…finitions

Check cli/README.md on GitHub Actions instead of Cloud Build

682acc1

Show the differing lines when cli/README.md is out of date

170055b

mbiernacik requested review from Ford544, apilaskowski and rafal-hawrylak November 3, 2026 17:42

mbiernacik requested a review from a team as a code owner November 3, 2026 17:43

github-advanced-security Bot found potential problems Nov 3, 2026

View reviewed changes

Comment thread cli/readme_generator.ts Fixed

Escape README table cells by content type

789ee88

rafal-hawrylak requested changes Nov 4, 2026

View reviewed changes

Address review comments on the generated cli/README.md

1c46ddd

mbiernacik requested a review from rafal-hawrylak November 4, 2026 10:50

Conversation

mbiernacik commented Nov 3, 2026

What

Changes

Testing

For reviewers

Uh oh!

mbiernacik commented Nov 3, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants