Oriel — AI & technology worth noticing, every day.

Detecting Flaky Tests and Finding Their Root Causes with Strands Agents and Gemini

Post · https://dev.to/m4rri4nne/detecting-flaky-tests-and-finding-their-root-causes-with-strands-agents-and-gemini-4kan

A few years ago (three years at most), one of the biggest challenges QA Engineers faced was: how do...

First seen by Oriel on .

AI summary

The article presents flaky-test-detector, a tool that scores inconsistent JUnit test results by commit and optionally uses a Strands Agents SDK agent with Gemini to suggest root causes.

  • A test is considered flaky when it both passes and fails against the same commit; skipped results and records without commit_sha are excluded.
  • The default ranker requires data from at least five commits and flags scores of 0.1 or higher, calculated as flaky_commits divided by analyzed commits.
  • The CLI produces JSON, CSV, and Markdown reports, while agent tools fetch reports by period and export summaries, inferred causes, and recommendations to Markdown.
  • The GitHub Actions workflow runs weekly, requires JUnit artifacts, and supplies commit_sha and run_id; other pipelines may need changes, and AI findings still require human review.

#software testing #flaky tests #strands agents #gemini

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Reactions

Total public reactions reported by the source; distinct from points and likes.

  • DEV Community: 84, latest observation ; +38 over observed period since

Comments

Comment count reported by the source.

  • DEV Community: 29, latest observation ; +29 over observed period since

Aliases

SourceExternal IDURL
DEV Community5012963https://dev.to/m4rri4nne/detecting-flaky-tests-and-finding-their-root-causes-with-strands-agents-and-gemini-4kan Read discussion