Detecting Flaky Tests and Finding Their Root Causes with Strands Agents and Gemini
A few years ago (three years at most), one of the biggest challenges QA Engineers faced was: how do...
First seen by Oriel on .
AI summary
The article presents flaky-test-detector, a tool that scores inconsistent JUnit test results by commit and optionally uses a Strands Agents SDK agent with Gemini to suggest root causes.
- A test is considered flaky when it both passes and fails against the same commit; skipped results and records without commit_sha are excluded.
- The default ranker requires data from at least five commits and flags scores of 0.1 or higher, calculated as flaky_commits divided by analyzed commits.
- The CLI produces JSON, CSV, and Markdown reports, while agent tools fetch reports by period and export summaries, inferred causes, and recommendations to Markdown.
- The GitHub Actions workflow runs weekly, requires JUnit artifacts, and supplies commit_sha and run_id; other pipelines may need changes, and AI findings still require human review.
#software testing #flaky tests #strands agents #gemini
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Reactions
Total public reactions reported by the source; distinct from points and likes.
- DEV Community: 84, latest observation ; +38 over observed period since
Comments
Comment count reported by the source.
- DEV Community: 29, latest observation ; +29 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| DEV Community | 5012963 | https://dev.to/m4rri4nne/detecting-flaky-tests-and-finding-their-root-causes-with-strands-agents-and-gemini-4kan Read discussion |