// release engineering [Validation harness]
ProjectForge
Supervised AI passes with release gates
AI-assisted release engineering
AI-assisted development still needs hard stops. ProjectForge (v0.11+) extracts an input archive into an isolated workspace, optionally runs a scoped LLM improvement pass, starts services, validates with deterministic checks, captures authenticated UI screenshots, compares against approved baselines, and packages a clean output ZIP — with autonomy guardrails so the model cannot weaken the harness that caught a failure.
- Python
- Playwright
- patchright
- OpenAI
- YAML profiles
- Visual regression
// architecture
Project ZIP ──► isolated workspace ──► LLM pass (optional) ──► Playwright + visual diff ──► PASS / REVIEW / FIX
// how it's built
Single CLI entry coordinates diagnose-only and supervised passes: extract ZIP → workspace, run profile steps in order, emit structured JSON reports with PASS, HUMAN_REVIEW, or FIX_REQUIRED verdicts.
Each project ships a profile declaring start commands, health checks, Playwright routes, screenshot contracts, and packaging rules. Generic web, self-hosted ProjectForge, and other targets share the same runner — only the profile changes.
Cloudflare-protected pages return 403 to plain fetch and headless Playwright. patchright in headed mode clears the block — validation captures real authenticated UI, not empty shells.
Authenticated route capture → content-signal checks → pixel diff against approved baselines → operator review HTML when diffs need eyes. Baseline promotion is explicit; no silent auto-accept.
Protected validation files sit outside the LLM edit allowlist. If a pass tries to weaken a gate, the run ends FIX_REQUIRED with the attempt logged — the harness cannot negotiate itself down.
Roadmap items, focus briefs, and pass limits bound what the model may change. Dependency cache and isolated workspaces keep runs reproducible; output ZIPs exclude secrets, local state, and runtime seeds by rule.
CLI for CI, local console for quick runs, and a GUI for candidate review, baseline acceptance, and promotion workflows — same backend, different entry points.
// highlights
Per-project YAML profiles and prompts — generic web, self-hosted ProjectForge, and other targets without rewriting the core runner.
Authenticated route capture, content-signal checks, baseline comparison, and operator review HTML when diffs need eyes.
Protected validation rules stay outside what the model may edit; attempts to weaken gates become explicit FIX_REQUIRED outcomes.
PASS, HUMAN_REVIEW, and FIX_REQUIRED so operators know whether to promote, inspect visuals, or fix deterministic failures first.
Roadmap items, focus briefs, and pass limits keep runs targeted instead of open-ended refactors.
CLI, console, and a local GUI for candidates, promotion, and baseline acceptance workflows.
// components
Coordinates diagnose-only and supervised passes, dependency cache, and structured reporting.
Service health, Playwright flows, screenshot contracts, and packaging checks before promotion.
Sanitized output artifacts with explicit exclusion rules for secrets, local state, and runtime seeds.
// related work