A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost
Web page · https://hightouch.com/blog/llm-judges-identity-resolution
First seen by Oriel on .
AI summary
Hightouch benchmarks LLM and decision-model judges for identity-resolution over-merging, finding that confidence-routed Jev and Claude Opus 5.5 approaches frontier accuracy at substantially lower cost.
- Identity resolution maps records to unique real people, with over-merging occurring when one identity incorrectly combines multiple individuals.
- Hightouch used pseudopeople to generate 1,000 identities, evenly divided between noisy single-person profiles and over-merges, including realistic data errors.
- On the harder test, Jev alone reached 89.1% accuracy and GPT-6 Luna reached 88.8%, compared with 95.1% for Claude Opus 5.5.
- Routing uncertain Jev decisions to Claude Opus 5.5 achieved 95.3% accuracy at $2.94 per 1,000 identities, versus $5.39 for Opus alone.
- Decision models answered in about 0.15–0.20 seconds per identity and cost $0.05–$0.12 per 1,000, but their performance degraded on difficult cases.
#identity resolution #llm judges #model evaluation #decision models
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hacker News: 595, latest observation ; +571 over observed period since
Comments
Comment count reported by the source.
- Hacker News: 153, latest observation ; +152 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hacker News | 50397429 | https://hightouch.com/blog/llm-judges-identity-resolution Read discussion |