Audited product-level coding-agent research updates https://micromilo.github.io/awesome-claude-code-codex-papers/ 2026-08-25T00:00:00Z MicroMilohttps://github.com/MicroMilo https://micromilo.github.io/awesome-claude-code-codex-papers/papers/qlcoder-2026/ 2026-08-25T00:00:00Z

CVE-grounded retrieval, AST/LSP guidance, an MCP interface, and an executable CodeQL validator that feeds back into iterative query repair. Reported evidence: QLCoder synthesizes correct queries for 53.4% of CVEs versus 10% for Claude Code-only; its query F1 is 0.70, versus 0.048 for IRIS and 0.073 for CodeQL suites.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/lean-refactor-2026/ 2026-08-25T00:00:00Z

A version-filtered strategy bank and multi-objective retrieve-and-rerank pipeline steer a frozen planner-refactorer-debugger agent without retraining. Reported evidence: Lean Refactor reports over 70% token-level compression on competition benchmarks, over 20% on research repositories, and up to 60% compilation-time reduction, while stating that it outperforms Claude Code.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/scaling-laws-2026/ 2026-08-25T00:00:00Z

Evolution-based program search jointly optimizes symbolic expressions and fitting procedures, with execution feedback and MAP-Elites selection. Reported evidence: SLDAgent reaches average R2 0.748 with GPT-5 versus 0.550 for Codex GPT-5, and improves over Claude Code rows for both Claude-Haiku-4.5 and Claude-Sonnet-4.5.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/cybergym-2026/ 2026-08-25T00:00:00Z

A scalable vulnerability-reproduction benchmark with difficulty levels, pre/post-patch execution validation, and zero-day discovery analysis. Reported evidence: Codex CLI with GPT-4.1 is directly compared with OpenHands, EnIGMA, and Cybench agent; the union of four agents reaches 18.4% Level-1 success, exposing complementary failure modes.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/deepscientist-2026/ 2026-08-25T00:00:00Z

A three-stage hypothesis, implementation/evaluation, and analysis loop backed by persistent Findings Memory and Bayesian exploration. Reported evidence: Claude Code executes all code implementation and analysis tasks as a host component; the system reports 21 validated innovations and 60% simulated acceptance under the stated review protocol.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/devops-gym-2026/ 2026-08-25T00:00:00Z

A realistic dynamic DevOps environment with standardized tool interfaces, multi-stage workflows, and stage-specific metrics. Reported evidence: Claude Code with Claude-4-Sonnet reports 51.85% build/configuration, 20.56% monitoring, 23.87% issue resolving, and 13.87% test generation success.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/featurebench-2026/ 2026-08-25T00:00:00Z

Execution-traced feature extraction removes target functionality while preserving interfaces, producing realistic feature-completion tasks with visible tests and anti-cheating controls. Reported evidence: Claude Code with Claude Opus 4.5 resolves 11.0% of Full tasks and Codex with GPT-5.1-Codex resolves 12.5%; the paper reports passed rates, token I/O, and failure modes.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/appforge-2026/ 2026-08-25T00:00:00Z

Formal task specifications, expert-verified UI interaction traces, automated compilation/testing, and runtime fuzz testing create a full-app benchmark. Reported evidence: Claude Code with Qwen3-Coder reaches 6.93% functional success; the paper reports that coding-agent gains are marginal and that multi-file integration and lifecycle behavior remain bottlenecks.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/innogym-2026/ 2026-08-25T00:00:00Z

Agent-as-judge extraction and comparison turns complete solution repositories into summaries/pseudocode and scores novelty against known solutions. Reported evidence: Codex with GPT-5 is actually used as the extraction and novelty-evaluation host; the benchmark compares MLAB, CodeAct, and AIDE rather than using Codex as a primary agent baseline.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/rpg-zerorepo-2026/ 2026-08-25T00:00:00Z

A persistent repository planning graph over features, functions, files, interfaces, and data flow guides staged generation, dependency-aware planning, and validation. Reported evidence: ZeroRepo reaches 81.5% functional coverage and 69.7% test accuracy, improving over Claude Code by 27.3 and 35.8 points; it produces about 36K lines and 445K code tokens.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/artemis-2026/ 2026-08-25T00:00:00Z

Supervisor with dynamic expert agents, recursive task decomposition, parallel exploration, context management, triage, and reporting Reported evidence: ARTEMIS finds 9 valid vulnerabilities with an 82% valid-submission rate, ranks second overall, and outperforms 9 of 10 human participants in the reported penetration-testing study.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/formact-2026/ 2026-08-25T00:00:00Z

HTML source editor, template retrieval, rendered-page review agent, iterative refinement, and edit-triggered context compression Reported evidence: Render correctness is 4.81 versus 4.39 for multi-pass Codex; human rank-1 rate is 0.760 versus 0.140. Content alignment is the caveat where Codex is slightly higher.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/terminal-bench-2-2026/ 2026-08-25T00:00:00Z

A benchmark design with realistic terminal environments, human-written solutions, executable tests, standardized Harbor tasks, and failure-mode analysis. Reported evidence: Codex CLI with GPT-5.2 reaches 62.9% resolution; Claude Code with Claude Opus 4.5 reaches 52.1%, with lower Claude Code rows for Sonnet 4.5, Opus 4.1, and Haiku 4.5 reported in Table 2.

https://micromilo.github.io/awesome-claude-code-codex-papers/papers/helmsman-2026/ 2026-08-25T00:00:00Z

Human-in-the-loop planning, modular code generation by collaborative agent teams, and closed-loop sandboxed evaluation and refinement with hierarchical runtime and semantic verification. Reported evidence: Claude Code with Claude Sonnet 4.5 reports 43.75% success and Codex with GPT-5.1-Codex reports 37.50%; Helmsman with Claude Sonnet 4.5 and GPT-5.1 reports 100% success in the paper's 16-task comparison.