Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 13 additions & 102 deletions .orly/audits/spec-template.sh
Original file line number Diff line number Diff line change
@@ -1,35 +1,17 @@
#!/usr/bin/env bash
# spec-template.sh — enforce docs/TEMPLATE.md across every spec under
# docs/v*/{pending,active,done}/. Two check families:
#
# 1. PROHIBITED patterns (negative space) — TEMPLATE.md "Prohibited" section:
# time/effort estimates, effort/complexity columns, %-complete, owners, dates.
# Always BLOCK.
# 2. REQUIRED-PRESENT + NO-PLACEHOLDER (positive space) — the determinism
# sections the agent-facing template mandates (PR Intent, Applicable Gates,
# Prior-Art, Metrics, Decomposition, tiered Test Spec, Product Clarity,
# Discovery, …) must EXIST and carry content: no known unfilled slots and
# no surviving "tpl:" guidance comments (template fill grammar). A spec
# that omits them forces the executing agent to guess intent. This is the
# half that makes a spec "built for the agent".
# 2b. DECLARED-COMMAND PARITY — a pending/active spec's Acceptance Rubric
# must quote the repository's declared conform + applicable verify.* commands
# (.orly/orly.json) verbatim: the rubric and `orly gate` grade one
# boundary. Staged/file scope only; done/ specs stay historical.
#
# Dispatch façade: dispatch/write_spec.md (SPEC TEMPLATE GATE)
# Fires in: make lint (after audit-logging, before audit-error-codes).
# spec-template.sh — select specs and snapshot staged content for the shared
# validator in spec-template.ts. The native orly spec.gate criterion calls the
# same validator directly; this entry point serves script callers.
# Dispatch façade: dispatch/write_spec.md (SPEC TEMPLATE GATE).
#
# Modes:
# --staged diff-scope: only specs in `git diff --cached`
# --all (default) pending+active specs only — current/in-flight work
# --include-done adds done/ specs to the scan (one-time sweep tool)
#
# Family 2 runs in --staged and --file scope — the spec being authored/edited right
# now, which is exactly the agent's own output. The bulk scans (--all /
# --include-done) run Family 1 only, so they behave identically over the whole
# corpus and never break an existing spec. No legacy carve-out, no heuristics:
# author a spec → full template required; scan the tree → prohibited-only.
# Staged/file checks require complete sections, references, test mappings and
# declared commands. Bulk scans check prohibited patterns only. Bun is required
# for both scopes; bulk checks do not require repository command configuration.
#
# Done specs are excluded by default: historical artifacts, not work product.
# Exits 0 clean, 1 on any blocking finding.
Expand All @@ -44,7 +26,6 @@ cd "$ROOT"
FAIL=0
fail() { printf "FAIL: %s\n" "$*" >&2; FAIL=1; }
ok() { printf "OK: %s\n" "$*"; }
note() { printf "NOTE: %s\n" "$*"; }

# Discover spec files in scope.
case "$MODE" in
Expand All @@ -62,8 +43,7 @@ case "$MODE" in
while IFS= read -r s; do SPECS+=("$s"); done < <(find docs/v[0-9]* -type f -name '*.md' 2>/dev/null | grep -E '/(pending|active|done)/' || true)
;;
--file|file)
# Single named spec, both families — the lifecycle engine's spec.gate
# criterion calls this so the engine and pre-commit share one enforcer.
# Single named spec, using the same checks as the native criterion.
[[ -n "${2:-}" ]] || { printf "usage: %s --file <spec path>\n" "$0" >&2; exit 64; }
[[ -f "$2" ]] || { printf "FAIL: spec not found: %s\n" "$2" >&2; exit 1; }
SPECS=("$2")
Expand All @@ -79,70 +59,6 @@ if [[ ${#SPECS[@]} -eq 0 ]]; then
exit 0
fi

# ---------------------------------------------------------------------------
# Family 1 — Prohibited patterns from docs/TEMPLATE.md "Prohibited" section.
# Each pattern: <severity>:<one-line description>:<regex> — the regex comes
# LAST because it may contain colons (e.g. the [:|] label-row class); read(1)
# assigns the remainder of the line to the final field, so the regex survives
# intact. Descriptions must stay colon-free.
# Severity: BLOCK = exit 1; INFO = stdout note only. Patterns use ERE (egrep -E)
# only — no PCRE constructs like (?:...), which grep -E rejects.
#
# DESIGN NOTE — patterns are deliberately structural, not prose-level.
# Bare regex like `\b\d+\s*days?\b` false-positives on legitimate prose
# ("Day 50", "≥7 days uptime", "within 3 days of install"). Instead we
# match where time/effort estimates LIVE structurally:
# - Section headings: `## Estimated effort`, `## Effort`, `## Sizing`
# - Label rows: `**Effort:** medium`, `**Owner:** alice`
# - Tilde-bulleted: `- ~2 h survey`, `* ~5 days OAuth setup`
# (the `~` prefix is Indy's idiom for effort estimates;
# legitimate prose doesn't use it)
# This catches the actual drift class without churning real content.
# ---------------------------------------------------------------------------
PATTERNS=(
'BLOCK:Estimated effort section heading:^#+ .*Estimated [Ee]ffort'
'BLOCK:Effort/Complexity/Sizing section heading:^#+ .*\b(Effort|Complexity|Sizing|Cost)\b\s*$'
'BLOCK:Scale estimate section heading:^#+ .*\bScale\b.*estimate'
'BLOCK:Tilde-bulleted hour estimate (e.g. "- ~2 h"):^[-*]\s*~\s*[0-9]+\s*h\b'
'BLOCK:Tilde-bulleted time estimate:^[-*]\s*~\s*[0-9]+\s*(hours?|days?|min(utes?)?)\b'
'BLOCK:Effort/Estimate label row:^\s*\*\*\s*(Effort|Estimate|Sizing|Complexity|Cost)\s*(:\s*\*\*|\*\*\s*[:|])'
'BLOCK:Owner label row:^\s*\*\*\s*Owner\s*(:\s*\*\*|\*\*\s*[:|])'
'BLOCK:Assigned-to label row:^\s*\*\*\s*Assigned to\s*(:\s*\*\*|\*\*\s*[:|])'
'BLOCK:Date-bound deadline label:^\s*\*\*\s*(Due|Deadline)\s*(:\s*\*\*|\*\*\s*[:|])\s*[0-9]'
'BLOCK:Percentage-complete metric:^[-*\|]?\s*\*?\*?\s*[0-9]+\s*%\s*complete'
'BLOCK:Hour-range estimate in effort context:\b(effort|estimate|takes?|spend)\b[^.]{0,40}\b[0-9]+\s*[-–]\s*[0-9]+\s*h\b'
'BLOCK:Effort/complexity rating:\b(low|medium|high|small|large)\s*[/|-]\s*(effort|complexity)\b'
)

# Carve-outs — lines that look prohibited but are legitimate. Skip any match.
SAFE_LINES_RE='^\*\*Date:\*\*|^\s*```|<!--|^\s*//|"[0-9]+\s*(h|hour|day|min)"'

scan_spec() {
local spec="$1" content_file="$2"
local hits=0
local pattern desc severity line content
while IFS=: read -r severity desc pattern; do
while IFS= read -r match; do
[[ -z "$match" ]] && continue
if grep -qE "$SAFE_LINES_RE" <<<"$match"; then
continue
fi
line="${match%%:*}"
content="${match#*:}"
content="${content%$'\r'}"
if [[ "$severity" = "BLOCK" ]]; then
fail "$spec:$line — $desc"
printf " > %s\n" "$content" >&2
else
note "$spec:$line — $desc"
fi
hits=$((hits + 1))
done < <(grep -nE "$pattern" "$content_file" 2>/dev/null || true)
done < <(printf '%s\n' "${PATTERNS[@]}")
if [[ $hits -eq 0 ]]; then ok "$spec — no prohibited patterns"; fi
return 0
}

# Staged checks grade the index, including files removed only from the worktree.
CONTENT_DIR="$(mktemp -d)"
trap 'rm -rf "$CONTENT_DIR"' EXIT
Expand All @@ -158,16 +74,11 @@ for spec in "${SPECS[@]}"; do
;;
esac
[[ -f "$content_file" ]] || { fail "$spec — cannot read spec"; continue; }
scan_spec "$spec" "$content_file"
case "$MODE" in
--staged|staged|--file|file)
if ! command -v bun >/dev/null 2>&1; then
fail "$spec — Bun is required for spec readiness checks"
elif ! bun "$SCRIPT_DIR/spec-template.ts" "$spec" "$content_file" "$MODE"; then
FAIL=1
fi
;;
esac
if ! command -v bun >/dev/null 2>&1; then
fail "$spec — Bun is required for spec checks"
elif ! bun "$SCRIPT_DIR/spec-template.ts" "$spec" "$content_file" "$MODE"; then
FAIL=1
fi
done

if [[ $FAIL -ne 0 ]]; then
Expand Down
45 changes: 39 additions & 6 deletions .orly/audits/spec-template.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { existsSync } from "node:fs";
import { existsSync, readFileSync } from "node:fs";
import { isAbsolute, join, normalize, resolve } from "node:path";

type MarkdownElement = { type?: unknown; props: { children?: MarkdownNode[] } };
Expand All @@ -23,7 +23,24 @@ const REQUIRED = [
];
const RESIDUE = /<!--\s*tpl:|\{\{fill:|\{unresolved_[^}]+\}|path\/to\/file\.ext|path\/to\/spec_or_doc\.md|test_<short_name>|\{one-line reason\}|\{Slice title\}|\{smallest verifiable behaviour\}|\{one-line behavioural claim\}/;
const STAGED_MODES = new Set(["--staged", "staged"]);
const BULK_MODES = new Set(["--all", "all", "--include-done", "include-done"]);
const EXPECTED_COLUMN = "Expected";
const TEXT_ENCODING = "utf8";
const SAFE_PROHIBITION_LINE = /<!--|"[0-9]+\s*(h|hour|day|min)"/;
const PROHIBITED: Array<[RegExp, string]> = [
[/^#+ .*Estimated [Ee]ffort/, "Estimated effort section heading"],
[/^#+ .*\b(Effort|Complexity|Sizing|Cost)\b\s*$/, "Effort/Complexity/Sizing section heading"],
[/^#+ .*\bScale\b.*estimate/, "Scale estimate section heading"],
[/^[-*]\s*~\s*[0-9]+\s*h\b/, 'Tilde-bulleted hour estimate (e.g. "- ~2 h")'],
[/^[-*]\s*~\s*[0-9]+\s*(hours?|days?|min(utes?)?)\b/, "Tilde-bulleted time estimate"],
[/^\s*\*\*\s*(Effort|Estimate|Sizing|Complexity|Cost)\s*(:\s*\*\*|\*\*\s*[:|])/, "Effort/Estimate label row"],
[/^\s*\*\*\s*Owner\s*(:\s*\*\*|\*\*\s*[:|])/, "Owner label row"],
[/^\s*\*\*\s*Assigned to\s*(:\s*\*\*|\*\*\s*[:|])/, "Assigned-to label row"],
[/^\s*\*\*\s*(Due|Deadline)\s*(:\s*\*\*|\*\*\s*[:|])\s*[0-9]/, "Date-bound deadline label"],
[/^[-*|]?\s*\*?\*?\s*[0-9]+\s*%\s*complete/, "Percentage-complete metric"],
[/\b(effort|estimate|takes?|spend)\b[^.]{0,40}\b[0-9]+\s*[-–]\s*[0-9]+\s*h\b/, "Hour-range estimate in effort context"],
[/\b(low|medium|high|small|large)\s*[/|-]\s*(effort|complexity)\b/, "Effort/complexity rating"],
];

function sectionsOf(text: string): Section[] {
const sections: Section[] = [];
Expand Down Expand Up @@ -111,8 +128,9 @@ function git(argv: string[], root: string): { ok: boolean; text: string } {
return { ok: result.exitCode === 0, text: result.stdout.toString() };
}

async function commands(staged: boolean, root: string): Promise<Commands> {
const text = staged ? git(["show", `:${CONFIG_PATH}`], root) : { ok: true, text: await Bun.file(join(root, CONFIG_PATH)).text() };
// Gate criteria are synchronous; standalone audits share the same command reader.
export function readSpecCommands(staged: boolean, root: string): Commands {
const text = staged ? git(["show", `:${CONFIG_PATH}`], root) : { ok: true, text: readFileSync(join(root, CONFIG_PATH), TEXT_ENCODING) };
if (!text.ok) throw new Error(`cannot read ${CONFIG_PATH} from the index`);
const config: unknown = JSON.parse(text.text);
if (!object(config) || !object(config.commands)) throw new Error(`${CONFIG_PATH} needs declared commands`);
Expand Down Expand Up @@ -154,9 +172,22 @@ function rubricCommands(sections: Section[], declared: Commands): string[] {
return errors;
}

export function validateSpecProhibitions(text: string): string[] {
const errors: string[] = [];
const lines = text.split("\n");
for (const [pattern, description] of PROHIBITED) {
for (const [index, line] of lines.entries()) {
if (pattern.test(line) && !SAFE_PROHIBITION_LINE.test(line)) {
errors.push(`line ${index + 1} — ${description}\n > ${line.replace(/\r$/, EMPTY)}`);
}
}
}
return errors;
}

export function validateSpec(text: string, declared: Commands, exists: (path: string) => boolean): string[] {
const sections = sectionsOf(text);
const errors: string[] = [];
const errors = validateSpecProhibitions(text);
const count = text.trimEnd().split(/\r?\n/).length;
if (count > SPEC_LINE_LIMIT) errors.push(`spec has ${count} lines; cap ${SPEC_LINE_LIMIT}`);
for (const heading of REQUIRED) {
Expand All @@ -170,9 +201,11 @@ export function validateSpec(text: string, declared: Commands, exists: (path: st
export async function auditSpec(args: string[], root: string): Promise<string[]> {
const [spec, path, mode] = args;
if (!spec || !path || !mode) throw new Error("expected spec path, content path, and audit mode");
const text = await Bun.file(resolve(root, path)).text();
if (BULK_MODES.has(mode)) return validateSpecProhibitions(text);
const staged = STAGED_MODES.has(mode);
const exists = (reference: string) => staged ? git(["cat-file", "-e", `:${reference}`], root).ok : existsSync(join(root, reference));
return validateSpec(await Bun.file(resolve(root, path)).text(), await commands(staged, root), exists);
return validateSpec(text, readSpecCommands(staged, root), exists);
}

if (import.meta.main) {
Expand All @@ -184,7 +217,7 @@ if (import.meta.main) {
}
if (errors.length) process.exitCode = 1;
// logging: report structural checks without claiming semantic readiness.
else console.log(`OK: ${Bun.argv[2]} — sections, references, test mappings, and declared commands checked`);
else console.log(`OK: ${Bun.argv[2]} — ${BULK_MODES.has(Bun.argv[4] ?? EMPTY) ? "no prohibited patterns" : "sections, references, test mappings, and declared commands checked"}`);
} catch (error) {
// logging: unavailable audit inputs must fail, never silently skip checks.
console.error(`FAIL: spec readiness check — ${error instanceof Error ? error.message : String(error)}`);
Expand Down
20 changes: 13 additions & 7 deletions .orly/dispatch/write_spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,12 @@ and who may decide it. A required human sign-off remains a human dependency.

## Mechanical checks

`.orly/audits/spec-template.sh` is the entry point; its Bun helper
`.orly/audits/spec-template.ts` reads headings, lists, and tables from the runtime's
parsed Markdown tree. Code examples and quoted headings cannot supply sections.
`orly gate pr` calls the native validator in `.orly/audits/spec-template.ts` through
`spec.gate`. The validator reads headings, lists, and tables from Bun's parsed
Markdown tree. Code examples and quoted headings cannot supply sections.

`.orly/audits/spec-template.sh` selects files and snapshots staged content before
calling the same validator.

- Prohibited authoring estimates, ownership labels, and completion percentages.
- Required sections with content, known placeholder residue, and the spec length cap.
Expand All @@ -38,9 +41,11 @@ parsed Markdown tree. Code examples and quoted headings cannot supply sections.
## Scope

`--staged` reads spec contents, configuration, and local references from the Git
index. `--file <path>` checks the working copy. Missing runtime or configuration
fails these readiness checks. Bulk `--all` and `--include-done` scans retain
prohibited-pattern checks only; they do not retrofit historical documents.
index. `--file <path>` checks the working copy. All script modes require Bun;
staged and file checks also require repository configuration.

Bulk `--all` and `--include-done` scans retain prohibited-pattern checks only;
they do not retrofit historical documents.

## Author review

Expand Down Expand Up @@ -90,4 +95,5 @@ Incident-derived rules the deterministic half can't check — apply while writin

- `.orly/docs/TEMPLATE.md` — canonical Prohibited section + required sections.
- `orly-spec-new` skill — creates specs from the template; inserts the banner.
- `.orly/audits/spec-template.sh` — structural and prohibited-section checks; `orly gate pr` invokes it through `spec.gate`.
- `.orly/audits/spec-template.ts` — native structural and prohibited-section validator, called directly by `spec.gate`.
- `.orly/audits/spec-template.sh` — file selection and staged snapshots for script callers of the same validator.
8 changes: 4 additions & 4 deletions .orly/orly.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"schema_version": 1,
"orly_version": "0.15.0",
"orly_version": "0.17.0",
"packs": [],
"commands": {
"conform": [
Expand Down Expand Up @@ -112,8 +112,8 @@
".orly/audits/msid-ui.sh": "sha256:50fa3a6dd3edb5e1e75722509259c753858687213e8aba6fc4159ba84185401f",
".orly/audits/rule-ledger-lib.sh": "sha256:168182658aa0d53bf08bbd571f81e04d538fc5b9b0c6d0006e57c6974c083164",
".orly/audits/scope.sh": "sha256:83725913f561729377f4b913495823e749ed64de5aeb9eda8fa1b35e01ef707b",
".orly/audits/spec-template.sh": "sha256:ea2aa79fd39c632e88be4d5b9f755beabfdd844c2c3fa882a0d13f9822daae29",
".orly/audits/spec-template.ts": "sha256:fcfc1534a8d33586835b456cf0d27284ffb3441fa00abe75848c2db6a05af247",
".orly/audits/spec-template.sh": "sha256:403046b83d8be1fa8279940f39a6aae2b23336e9a33d53d2af8b227406fffd2f",
".orly/audits/spec-template.ts": "sha256:ef10e170d86731d40cab6a9466b9ce26b9b20f6974b122e6defd870af40749e1",
".orly/audits/ufs-strings.awk": "sha256:a6ef0f62d627f3c09f64c9d8b6e33ea6b25d85ccf2585cfddb2f3500be1a80c0",
".orly/audits/ufs.sh": "sha256:18b93033a91ea5ef2d7f453fc1f893201245e17a6932f67a8227c4ebb707e19e",
".orly/dispatch/lib.sh": "sha256:63b5e2b8e962808ce41523f122e34a75d316e58bf4026c3aafab54c28e73ea97",
Expand All @@ -128,7 +128,7 @@
".orly/dispatch/write_mdx.md": "sha256:70ce22667e2ca60e1bf623d5ba0ca1a9bcc0586faf018d44328ddfeaa2cd4306",
".orly/dispatch/write_pr_description.md": "sha256:2e1a91b09e98faa0971968092b53be8d087c617fc1b9afb267b2deeccc5372e0",
".orly/dispatch/write_python.md": "sha256:2b6fd567a12ccb92f19f29669cc9b1a2254127b49501a057b155991adde54988",
".orly/dispatch/write_spec.md": "sha256:ffc8f991965faed799f8ce83ff4052a3de18defafdfe0d112ab2ebf176804ee4",
".orly/dispatch/write_spec.md": "sha256:a1b89dcf110985b949ec8277f7a79241fedab4da2744ac727019f53c1a152bfb",
".orly/dispatch/write_ts_adhere_bun.md": "sha256:466784d3161014c6dc735a6e55831a83dad66883db15027cc0543f62a9dd7f11",
".orly/dispatch/write_ts_adhere_bun.sh": "sha256:3244892292500a18f5a053c9f1e954ef0ff51cb79b7ae83883aa3350289d1fa2",
".orly/docs/CHANGELOG_VOICE.md": "sha256:c523f2a1002aff6583ad6570fb93acc322229e7421f6c889685084a47fe00195",
Expand Down
10 changes: 7 additions & 3 deletions billing/budgets.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ title: Fleet budgets
description: Daily and monthly spending limits for one fleet.
type: explanation
audience: user
verified: 2026-10-19
product_version: 0.62.1
verified: 2026-10-23
product_version: 0.65.0
executable: false
---

Expand All @@ -27,7 +27,7 @@ Each limit is a decimal string of whole nanodollars. One dollar is `"1000000000"

## Why it exists

Every workspace under one tenant shares the same credit balance. A fleet budget stops one fleet from draining that shared balance.
Every workspace in one account shares the same credit balance. A fleet budget stops one fleet from draining that shared balance.

The dashboard header shows what is left of that balance on every page, as a **Credits** chip beside the workspace switcher. It appears on screens at least 640px wide. Select the chip to open **Settings → Billing**, which holds the live figure and the charge history.

Expand All @@ -53,6 +53,10 @@ If billing cannot answer, work continues and can exceed that bound. The full cha

### How charges are counted

In **Settings → Billing**, **Model** shows its published display name when available. **Usage** separates input and output counts.

**Occurred** shows relative time; hover to see the exact timestamp. Unavailable usage stays marked as unavailable.

`agentsfleet` charges each provider request on its own line. A fleet run or setup chat also adds one run-time line for each stretch of time it ran. The charge history at `GET /v1/tenants/me/billing/charges` returns one row per line.

Each request is priced at the rate in effect when it was made, and at the tier its whole prompt falls in. Cached reads, cache writes, and output each use their own rate.
Expand Down
Loading
Loading