Oriel — AI & technology worth noticing, every day.

AI agent fixed the bug, but what else did it change?

Web page · https://blog.trailofbits.com/2026/10/21/your-ai-agent-fixed-the-bug-but-what-else-did-it-change

First seen by Oriel on .

AI summary

Trail of Bits introduces patching-bugs and regression-hunt, plugins that structure agent-led bug fixes and live proof-based regression checks; preliminary tests found 25% fewer false positives than a baseline Codex agent.

  • patching-bugs reproduces reported bugs, develops fixes, and uses a three-arm proof: the original and reverted code must fail while the patched code passes.
  • regression-hunt traces code changes to real interfaces, reproduces candidates on live deployments, and uses a fresh-context agent plus adjudication to validate evidence.
  • Across 21 confirmed regressions, regression-hunt reduced false positives by 25%, with a 95% confidence interval of 14% to 38%.
  • Strict evidence requirements left some known regressions borderline; the evaluation also found 21 regressions among more than 130 merged PRs, and none broke a security protection.
  • Maintainers can guide the plugins with THREAT_MODEL.md and PATCHING_GUIDELINES.md or AGENTS.md; both plugins provide modular steps and defined input and output schemas.

#ai agents #software security #regression testing #open source

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hacker News: 49, latest observation ; +42 over observed period since
  • Lobsters: 2, latest observation ; +2 over observed period since

Comments

Comment count reported by the source.

  • Hacker News: 6, latest observation ; +5 over observed period since
  • Lobsters: 0, latest observation ; 0 over observed period since

Aliases

SourceExternal IDURL
Hacker News50183253https://blog.trailofbits.com/2026/10/21/your-ai-agent-fixed-the-bug-but-what-else-did-it-change/ Read discussion
Lobsters17bp4zhttps://blog.trailofbits.com/2026/10/21/your-ai-agent-fixed-the-bug-but-what-else-did-it-change/ Read discussion