Benchmarking the Gemini Decisions API
Post · https://dev.to/gde/benchmarking-the-gemini-decisions-api-l31
The Gemini Decisions API scores a fixed set of answers in one pass and returns probabilities. An ADK triage agent built on it, a 48-ticket benchmark against Gemini 3.8 Flash, Gemini 4 Hydrogen, Gemini 3.5 Flash-Lite and Gemma 4 E2B, and a test of 1 to 100 questions in one request.
First seen by Oriel on .
AI summary
This tutorial benchmarks the Gemini Decisions API against three Gemini models and Gemma for customer-support triage, finding it fastest, inexpensive, confidence-aware, and competitive on fixed-set decisions.
- The Decisions API scores up to 1,000 allowed-answer questions in one forward pass, returns probabilities, generates no text, and charges $0.10 per million input tokens.
- Across 48 tickets, it achieved 46/48 team labels, 47/48 refund labels, and 29/48 tickets with all three answers correct.
- Its 202 ms median latency was 3.5 times faster than Flash-Lite, 9.7 times faster than Hydrogen, and 11.7 times faster than 3.8 Flash.
- A 0.7 confidence threshold identified all 17 urgency errors, enabling automatic handling of higher-confidence results and human review for uncertain ones.
- The API answered 100 yes/no questions in 359 ms, while generative backends produced answers without confidence scores and scaled more slowly.
#gemini decisions api #customer support #model benchmarking #llm agents
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Reactions
Total public reactions reported by the source; distinct from points and likes.
- DEV Community: 21, latest observation ; +5 over observed period since
Comments
Comment count reported by the source.
- DEV Community: 4, latest observation ; 0 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| DEV Community | 5032399 | https://dev.to/gde/benchmarking-the-gemini-decisions-api-l31 Read discussion |