Oriel — AI & technology worth noticing, every day.

Using LLM for confidence scoring is a bad idea

Web page · https://rumple.github.io/blog/2026/10/20/asking-llm-for-confidence.html

First seen by Oriel on .

AI summary

The post argues that confidence scores generated as ordinary LLM text are not trustworthy unless calibrated against outcomes, and recommends deriving confidence from observable evidence and recorded results.

  • A useful confidence score reflects available evidence and matches observed accuracy across many predictions; calibration alone is insufficient if the score ignores informative evidence.
  • LLM-generated scores can vary with repeated prompts, wording, and preceding text because they are plausible continuations, not measurements of evidence or correctness.
  • For support agents, track observable signals such as article matches and agreement between repeated replies, then compare them with reviewed outcomes.
  • Use historical correctness rates to estimate confidence for similar cases, and regularly check calibration; the method depends on maintaining a reliable record of outcomes.

#llm agents #calibration #confidence estimation #customer support

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hacker News: 224, latest observation ; +220 over observed period since

Comments

Comment count reported by the source.

  • Hacker News: 102, latest observation ; +102 over observed period since

Aliases

SourceExternal IDURL
Hacker News50176082https://rumple.github.io/blog/2026/10/20/asking-llm-for-confidence.html Read discussion