# The $0 AI Code-Review Playbook ## Catch bugs + security holes with any models — no paid per-review tool **One-line value:** Build an automated code reviewer that flags *real* bugs and *real* security holes using AI models you already have — for $0 — by chaining a cheap, aggressive "finder" with a skeptical "verifier" that kills false positives before they ever reach you. --- ### The one insight that separates signal from noise A naive *"review this code"* prompt buries you in false positives — and a reviewer that cries wolf gets ignored, then switched off. The frontier tools (Cursor Bugbot, Anthropic's open-source security review) independently converged on the same fix: > **The cost of a false positive vastly exceeds the cost of a false negative.** Make the first pass aggressive, the second pass adversarial, and ship only what survives both. It works because each stage has exactly one job: | Stage | Its only job | Tone | |---|---|---| | **Finder** | Flag *every* suspicious pattern on the diff + nearby context | Aggressive, over-sensitive | | **Verifier** | Try to *refute* each finding; default to "not a real bug" unless proven | Skeptical, adversarial | | **Survivors** | Only confirmed findings reach you | High signal | That's the whole pattern. Cheap, model-agnostic, and free on any flat-rate coding-plan membership. --- ### The flow, end to end 1. **Feed the Finder** the git diff of changed files (plus a little surrounding context — not the whole repo). 2. **Ask for strict-JSON findings** so you can pipe them programmatically. 3. **Pass each finding, one at a time, to the Verifier.** 4. **Keep only `KEEP` verdicts.** Everything else is dropped, silently. 5. **Optional ensemble:** run the Finder *N* times (vary temperature, reorder the diff), then keep only findings that show up in **≥K passes**. Near-free on a membership, and a real recall boost. | Lever | Why it matters | |---|---| | Diff-only input | Cheap, and keeps the model on what actually changed | | Two-pass filter | Finder can be loose; the Verifier does the precision work | | Ensemble (optional) | Flat-rate re-runs cost nothing; majority vote cuts noise | | Confirmed-only blocking | Keeps the reviewer trusted instead of annoying | --- ### Copy-paste: the Finder prompt ``` You are an aggressive code reviewer. Review the diff below and flag EVERY suspicious pattern — bugs, logic errors, edge cases, security issues. Over-flagging is fine; missing a real bug is not. DIFF: <<>> Return STRICT JSON only — no prose, no code fences: { "findings": [ { "file": "path/to/file", "line": 42, "severity": "critical|high|medium|low", "summary": "one-line description", "failure_scenario": "concrete, step-by-step way this breaks", "minimal_fix": "the smallest change that fixes it" } ] } ``` ### Copy-paste: the Verifier (refuter) prompt ``` You are a skeptical senior engineer. A previous pass flagged the finding below. Your job is to REFUTE it — try hard to prove it is NOT a real bug. Default to "not a real bug" unless you can substantiate a concrete failure using the actual code in the diff. DIFF: <<>> FINDING: <<>> Can you construct a concrete input or execution path that triggers the claimed failure? Reply with EXACTLY one line: KEEP — DROP — ``` Run **Finder once → Verifier per finding → ship the survivors.** That's the entire system, and it drops into any chat window or API. --- ### The security lens (same scaffold, new focus) Keep the exact same Verifier. Just point the Finder at security by prepending this line and handing it the taxonomy + exclusion list: ``` You are a security-focused reviewer. Flag ONLY the categories in the taxonomy below. DROP anything on the exclusion list, even if it looks risky. Same STRICT JSON output as before. ``` **Vulnerability taxonomy — what to hunt:** | Category | Look for | |---|---| | **Injection** | SQL, command, LDAP, XPath, NoSQL, XXE | | **Broken auth / authz** | Privilege escalation, IDOR, broken sessions | | **Secrets & data exposure** | Hardcoded credentials, PII, sensitive logging | | **Weak crypto** | Broken algorithms, hardcoded keys/IVs | | **Unsafe input validation** | Only with a *real, demonstrable* exploit path | | **Race conditions** | TOCTOU, concurrent-state bugs | | **Insecure config** | Permissive CORS, missing security headers | | **Supply chain** | Vulnerable or typosquatted dependencies | | **Unsafe deserialization / eval** | RCE via `eval`, pickle, `unserialize` | | **XSS** | Reflected, stored, DOM-based | **False-positive exclusion list — tell BOTH passes to DROP these on sight:** > Do NOT report: DoS / resource exhaustion · missing rate-limiting · generic input-validation without a proven exploit · open redirects · pure style or lint issues. The exclusion list is what makes this usable. Without it you drown in low-value noise; with it, nearly every surviving finding is worth reading. --- ### Wire it into your workflow - **Review the git diff, not the whole repo** — `git diff main...HEAD` piped straight into the Finder. Cheap and focused. - **Run it pre-push or nightly** from a tiny script. No infra, no service. - **Block or flag only on a confirmed `critical`.** Let mediums and lows collect in a digest you skim, not a gate that stops you. --- ### The one metric that matters: resolution rate > Of the issues your reviewer surfaces, how many did you actually FIX? That single number tells you if the reviewer is signal or noise. Track it, and tune your prompts to push it up. Frontier teams hill-climbed this from roughly **~50% → ~75%** — purely by tightening the Verifier and curating the exclusion list. If yours sits below 50%, your Finder is too loose or your exclusion list too short. Fix those two things before touching anything else. --- ### Want the pre-built version? This guide is genuinely enough to build the whole thing today. But if you'd rather skip the assembly: **Grab the complete prompt pack — Finder + Verifier + the security lens + the ensemble script — at the [[Your Business] store](https://yourwebsite.com).** Copy, paste, ship.