Oriel — AI & technology worth noticing, every day.

A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost

Web page · https://hightouch.com/blog/llm-judges-identity-resolution

First seen by Oriel on .

AI summary

Hightouch benchmarks LLM and decision-model judges for identity-resolution over-merging, finding that confidence-routed Jev and Claude Opus 5.5 approaches frontier accuracy at substantially lower cost.

  • Identity resolution maps records to unique real people, with over-merging occurring when one identity incorrectly combines multiple individuals.
  • Hightouch used pseudopeople to generate 1,000 identities, evenly divided between noisy single-person profiles and over-merges, including realistic data errors.
  • On the harder test, Jev alone reached 89.1% accuracy and GPT-6 Luna reached 88.8%, compared with 95.1% for Claude Opus 5.5.
  • Routing uncertain Jev decisions to Claude Opus 5.5 achieved 95.3% accuracy at $2.94 per 1,000 identities, versus $5.39 for Opus alone.
  • Decision models answered in about 0.15–0.20 seconds per identity and cost $0.05–$0.12 per 1,000, but their performance degraded on difficult cases.

#identity resolution #llm judges #model evaluation #decision models

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hacker News: 595, latest observation ; +571 over observed period since

Comments

Comment count reported by the source.

  • Hacker News: 153, latest observation ; +152 over observed period since

Aliases

SourceExternal IDURL
Hacker News50397429https://hightouch.com/blog/llm-judges-identity-resolution Read discussion