Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .claude/templates/coding_prompt.template.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,11 @@ After thorough verification, mark the feature as passing:
Use the feature_mark_passing tool with feature_id=42
```

If the project has quality gates enabled, `feature_mark_passing` first runs the project's
lint and type checks. If it responds that quality gates failed, the feature was NOT marked
passing: fix the reported errors and call it again. You can run the same checks earlier with
`feature_verify_quality` (it returns an error when quality gates are not enabled; that is fine, just continue).

**NEVER:**

- Delete features
Expand Down Expand Up @@ -240,6 +245,12 @@ feature_skip with feature_id={id}

# 7. Clear in-progress status (when abandoning a feature)
feature_clear_in_progress with feature_id={id}

# 8. Run lint/type-check quality gates (before marking passing)
feature_verify_quality with feature_id={id}

# 9. Record why an attempt failed (before skipping/abandoning a feature)
feature_report_failure with feature_id={id} and reason="..."
```

### RULES:
Expand Down
3 changes: 2 additions & 1 deletion .claude/templates/testing_prompt.template.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,8 @@ A regression has been introduced. You MUST fix it:
- `feature_get_stats` - Get progress overview (passing/in_progress/total counts)
- `feature_get_by_id` - Get your assigned feature details
- `feature_mark_failing` - Mark a feature as failing (when you find a regression)
- `feature_mark_passing` - Mark a feature as passing (after fixing a regression)
- `feature_mark_passing` - Mark a feature as passing (after fixing a regression; may be rejected if quality gates fail)
- `feature_verify_quality` - Run the project's lint/type-check quality gates

### Browser Automation (Playwright CLI)
Use `playwright-cli` commands for browser interaction. Key commands:
Expand Down
10 changes: 10 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,13 @@
# ANTHROPIC_DEFAULT_OPUS_MODEL=kimi-k2.5
# ANTHROPIC_DEFAULT_SONNET_MODEL=kimi-k2.5
# ANTHROPIC_DEFAULT_HAIKU_MODEL=kimi-k2.5

# Quality Gates (optional)
# Run the project's linters/type checkers (ESLint, Biome, ruff, flake8, tsc, mypy)
# before a feature can be marked passing. off (default) | strict | warn
# WARNING: checks run project-local tooling and config (node_modules/.bin, .venv,
# eslint.config.js, ...) OUTSIDE the agent's Bash sandbox. Only enable this for
# projects you trust to run unsandboxed on your machine.
# AUTOFORGE_QUALITY_GATES=strict
# AUTOFORGE_QUALITY_GATES_TIMEOUT=120
# AUTOFORGE_QUALITY_GATES_CHECKS=lint,type_check
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ jobs:
run: ruff check .
- name: Run security tests
run: python test_security.py
- name: Run quality gate tests
run: python -m pytest test_quality_gates.py

ui:
runs-on: ubuntu-latest
Expand Down
20 changes: 20 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ python test_security_integration.py # Integration tests (9 tests)
python -m pytest test_client.py # Client tests (20 tests)
python -m pytest test_dependency_resolver.py # Dependency resolver tests (12 tests)
python -m pytest test_rate_limit_utils.py # Rate limit tests (22 tests)
python -m pytest test_quality_gates.py # Quality gate tests
```

### React UI
Expand Down Expand Up @@ -172,6 +173,7 @@ Publishing: `npm publish` (triggers `prepublishOnly` which builds UI, then publi
- `auth.py` - Authentication error detection for Claude CLI
- `env_constants.py` - Shared environment variable constants (API_ENV_VARS) used by client.py and chat sessions
- `rate_limit_utils.py` - Rate limit detection, retry parsing, exponential backoff with jitter
- `quality_gates.py` - Lint/type-check auto-detection and enforcement before `feature_mark_passing`
- `api/database.py` - SQLAlchemy models (Feature, Schedule, ScheduleOverride)
- `api/dependency_resolver.py` - Cycle detection (Kahn's algorithm + DFS) and dependency validation
- `api/migration.py` - JSON-to-SQLite migration utility
Expand Down Expand Up @@ -244,6 +246,24 @@ MCP tools available to the agent:
- `feature_add_dependency` - Add dependency between features (with cycle detection)
- `feature_remove_dependency` - Remove a dependency
- `feature_set_dependencies` - Set all dependencies for a feature at once
- `feature_verify_quality` - Run lint/type-check quality gates and store the result on the feature
- `feature_report_failure` - Record why a feature attempt failed (increments `failure_count`)

#### Quality Gates

`quality_gates.py` auto-detects a project's linters (ESLint, Biome, ruff, flake8) and type checkers (tsc, mypy) from its config files and runs them. Node tools only run from the project's `node_modules/.bin`; Python tools from the project's `.venv`/`venv` or PATH.

Disabled by default. Configured **only** via environment variables of the AutoForge process (e.g. `~/.autoforge/.env`, see `.env.example`), which `client.py` forwards to the feature MCP server. Nothing inside the project directory (which the agent can write) can enable or weaken the gates:

```bash
AUTOFORGE_QUALITY_GATES=strict # off (default) | strict | warn
AUTOFORGE_QUALITY_GATES_TIMEOUT=120 # per-check timeout in seconds (max 600)
AUTOFORGE_QUALITY_GATES_CHECKS=lint,type_check
```

When enabled, `feature_mark_passing` runs the checks first. In `strict` mode a failing check blocks marking the feature passing, records `failure_reason`, and increments `failure_count`; in `warn` mode it only warns. The latest report is stored in the feature's `quality_result` column as test evidence. While disabled, `feature_verify_quality` returns an error and runs nothing.

**Security / trust:** the checks run inside the feature MCP server, outside the agent's Bash sandbox and the `security.py` allowlist, and execute project-local tooling and config the agent can modify (`node_modules/.bin/*`, `.venv`, `eslint.config.js`, ...). Enabling quality gates is a trust decision; only enable them for projects you trust to run unsandboxed.

### React UI (ui/)

Expand Down
25 changes: 25 additions & 0 deletions api/database.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,11 @@ class Feature(Base):
human_input_request = Column(JSON, nullable=True, default=None) # Agent's structured request
human_input_response = Column(JSON, nullable=True, default=None) # Human's response

# Quality gates: evidence from the last lint/type-check run and failure tracking
quality_result = Column(JSON, nullable=True, default=None) # Last quality report (see quality_gates.py)
failure_reason = Column(Text, nullable=True, default=None) # Why the last attempt failed
failure_count = Column(Integer, nullable=False, default=0) # Number of failed attempts

def to_dict(self) -> dict:
"""Convert feature to dictionary for JSON serialization."""
return {
Expand All @@ -84,6 +89,10 @@ def to_dict(self) -> dict:
"needs_human_input": self.needs_human_input if self.needs_human_input is not None else False,
"human_input_request": self.human_input_request,
"human_input_response": self.human_input_response,
# Quality gate fields
"quality_result": self.quality_result,
"failure_reason": self.failure_reason,
"failure_count": self.failure_count if self.failure_count is not None else 0,
}

def get_dependencies_safe(self) -> list[int]:
Expand Down Expand Up @@ -326,6 +335,21 @@ def _migrate_add_human_input_columns(engine) -> None:
conn.commit()


def _migrate_add_quality_columns(engine) -> None:
"""Add quality gate columns to existing databases that don't have them."""
with engine.connect() as conn:
result = conn.execute(text("PRAGMA table_info(features)"))
columns = [row[1] for row in result.fetchall()]

if "quality_result" not in columns:
conn.execute(text("ALTER TABLE features ADD COLUMN quality_result TEXT DEFAULT NULL"))
if "failure_reason" not in columns:
conn.execute(text("ALTER TABLE features ADD COLUMN failure_reason TEXT DEFAULT NULL"))
if "failure_count" not in columns:
conn.execute(text("ALTER TABLE features ADD COLUMN failure_count INTEGER NOT NULL DEFAULT 0"))
conn.commit()


def _migrate_add_schedules_tables(engine) -> None:
"""Create schedules and schedule_overrides tables if they don't exist."""
from sqlalchemy import inspect
Expand Down Expand Up @@ -450,6 +474,7 @@ def create_database(project_dir: Path) -> tuple:
_migrate_add_dependencies_column(engine)
_migrate_add_testing_columns(engine)
_migrate_add_human_input_columns(engine)
_migrate_add_quality_columns(engine)

# Migrate to add schedules tables
_migrate_add_schedules_tables(engine)
Expand Down
7 changes: 7 additions & 0 deletions client.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
from claude_agent_sdk.types import HookContext, HookInput, HookMatcher, SyncHookJSONOutput
from dotenv import load_dotenv

from quality_gates import QUALITY_GATE_ENV_VARS
from security import SENSITIVE_DIRECTORIES, bash_security_hook

# Load environment variables from .env file if present
Expand Down Expand Up @@ -159,6 +160,8 @@ def get_extra_read_paths() -> list[Path]:
"mcp__features__feature_mark_failing",
"mcp__features__feature_skip",
"mcp__features__feature_clear_in_progress",
"mcp__features__feature_verify_quality",
"mcp__features__feature_report_failure",
]

TESTING_AGENT_TOOLS = [
Expand All @@ -170,6 +173,7 @@ def get_extra_read_paths() -> list[Path]:
"mcp__features__feature_get_graph",
"mcp__features__feature_mark_passing",
"mcp__features__feature_mark_failing",
"mcp__features__feature_verify_quality",
]

INITIALIZER_AGENT_TOOLS = [
Expand Down Expand Up @@ -335,6 +339,9 @@ def create_client(
# (subprocess inherits parent environment automatically)
"PROJECT_DIR": str(project_dir.resolve()),
"PYTHONPATH": str(Path(__file__).parent.resolve()),
# Trusted quality-gate switch: comes from the host environment only,
# never from files the agent can write inside the project
**{k: os.environ[k] for k in QUALITY_GATE_ENV_VARS if k in os.environ},
},
},
}
Expand Down
Loading