ADRA — Adversarial Dev Review Agent
A deterministic-first, adversarial-validation engine for the software lifecycle. Deterministic tools (git, the exact CI command, bundle validation, a language/leak scan, SQL probes) run FIRST and become both the grounding the model may not contradict and the evidence in an immutable provenance log; a blocking adversarial critic tries to refute each artifact instead of blessing it, an LLM-as-judge scores with swap-and-average, and the loop escalates to a human where nothing deterministic backs the verdict. Six skills (code review, PR eval, experiment, improve, document, decide) run the same loop. Runs offline with no API key; published on PyPI as adra.
Business Context
Code review and validation are where engineering risk is priced. A reviewer that emits a confident but false finding costs trust; an autonomous agent that merges on a green suite it did not adversarially challenge costs an incident. The value here is a defensible verdict: every blocker traces to a deterministic tool result recorded in an immutable provenance log, the model can add semantic findings but cannot overturn the floor, and the system says out loud when it has no evidence and hands the decision to a person. That is the discipline a senior engineer already applies, made explicit and reproducible.
Strategic Value
ADRA is a reusable pattern for grounding a language model in deterministic evidence instead of letting its prose be the verdict, and for building an agent that refutes rather than flatters. The engine is client-agnostic: a client is a governance suite (conventions, ADRs, CI standards, glossary, incident cases) the engine grounds on, shipped with a complete fictional example (Northwind Data Platform) and selectable by an environment variable, so the same skills run against any standards without code changes. One connector Protocol lets the same loop run against GitHub, Azure DevOps, Databricks and Azure or against a self-contained offline emulator (synthetic git repos, PRs, wiki, boards, CI and a SQLite warehouse). It is deliberately framework-free (a hand-rolled state machine, no LangChain or LangGraph) and honest about scope: the connector-phase security controls it will need are labelled planned, not claimed as done.
The Challenge
The AI-code-review market splits in two and both halves miss the same spot. Reviewers (CodeRabbit, Greptile, Qodo and the like) feed linters into a language model, but the model prose is the verdict, so hallucinated and "consistently-stated-but-false" findings leak through and the deterministic signals are never the gate. Autonomous coders (Devin, OpenHands, SWE-agent) write code and treat "tests pass" as success rather than adversarially trying to prove the change wrong. Both produce opinions where a senior engineer would want proofs, refutations, and an honest "I cannot verify this, a human should decide".
Our Approach
ADRA occupies the gap with a deterministic spine that grounds a blocking adversarial critic. Its tools run first, in order: git and merge-base health, the exact CI command, bundle validation, a language and secret-leak scan, code discovery, and SQL warehouse probes; each returns a typed ToolResult that becomes both the grounding the model may not overturn and the evidence in the run record. On top of that floor a two-pass critic red-teams every artifact (a deterministic hard-floor pass keyed on a shared typed rubric, then an LLM semantic-attack pass whose job is to refute, not bless), and an LLM-as-judge scores with swap-and-average and reference anchoring so a winner counts only when it is stable under the order swap. Six skills run this same loop, differing only by domain prompt and tools: code_review, pr_eval, experiment, improve, document, decide. Because the deterministic floor carries the verdict, the whole loop and its test suite run offline with no API key via a mock provider; connecting a real provider through a pydantic-ai seam (config only, per-role routing) adds the semantic layer on top. It is read-only by default: writes require an explicit external flag and a human gate, and the loop escalates to a human wherever nothing deterministic backs the call.
Key Performance Indicators
| KPI | Baseline | Result | Impact |
|---|---|---|---|
| What carries the verdict | The model prose is the verdict (false findings leak through) | A deterministic floor (git, CI, bundle, lang/leak, SQL) grounds a blocking critic; every finding carries evidence | You audit the gate, not trust an opinion |
| Autonomy stance | Autonomous agents treat "tests pass" as success and write freely | Adversarial critic tries to refute each artifact; read-only by default, human gate on PR create/push/merge | No unsupervised writes; it escalates when it cannot verify |
| Runs offline | Needs an API key and a cloud service | A deterministic mock lane runs the whole loop and the test suite with no key | Reproducible; a real provider is config-only via pydantic-ai |
| Grounding scope | A reviewer wired to one host | 4 real connectors (GitHub, Azure DevOps, Databricks, Azure) + an offline emulator, one Protocol | The same client-agnostic skills run anywhere |
Architecture
adra pipeline
From an opinion to a proof, a refutation, or an honest escalation
ADRA is a deterministic-first, adversarial-validation engine for the software lifecycle: it reviews diffs and PRs, runs hypothesis-driven validation experiments, proposes minimum-functional improvements, writes documentation back, and routes decisions, all on one loop. It is published on PyPI as adra (pip install adra, Python >= 3.11, Apache-2.0), and a private connection-first web console consumes the same engine live at adra.fasl-work.com.
Deterministic-first, by construction
The tools run first, not last. git and merge-base health, the exact CI command, bundle validate, a language and secret-leak scan, code discovery, and SQL warehouse probes each return a typed result that becomes two things at once: the grounding the model may not contradict, and the evidence written to an immutable provenance log. Because the deterministic floor carries the verdict, the whole loop and its tests run offline with no API key through a mock provider; connecting a real provider adds a semantic layer on top and never replaces the floor.
intake plan ground (deterministic tools) generate CRITIC
revise / accepted / escalate
artifacts + immutable run recordA critic that refutes, a judge that swaps
The critic is two passes: a deterministic hard-floor keyed on a shared typed rubric (so “what we check” never drifts between the check and the prompt), then an LLM semantic-attack pass whose job is to refute an artifact, not bless it. The LLM-as-judge scores with swap-and-average and reference anchoring, so a winner counts only when it stays the winner after the comparison order is reversed. The orchestrator is a hand-rolled, framework-free state machine, no LangChain or LangGraph.
Honest about autonomy and scope
ADRA is read-only by default: writes require an explicit external flag and a human gate, and the loop escalates to a human wherever nothing deterministic backs the verdict, exactly where a senior engineer would stop and ask. It is client-agnostic (a governance suite of conventions, ADRs, CI standards, glossary and incident cases, shipped with a complete fictional example, the Northwind Data Platform, and selectable by environment variable), and its connectors span GitHub, Azure DevOps, Databricks and Azure plus a fully offline emulator. The connector-phase security controls it will still need (dual-LLM capability split, sandboxed egress-filtered execution) are labelled planned, not claimed as done, which is why the package stays on 0.x.
Technology Stack
In action
A short tour of the live app — the real interface, recorded from the deployed site.
Application Screenshots
