Oriel — AI & technology worth noticing, every day.

Validating the Comparison, Not Just the Scorer: Why Overall Accuracy Can Mislead in Interactive LLM Evaluation

Paper · https://arxiv.org/abs/2610.31678

arXiv:2610.31678v1 Announce Type: new Abstract: Many LLM studies compare how models respond to different follow-up prompts. For example, one prompt may neutrally ask a model to reconsider, while another insists that an invented film really exists. The difference is used to measure the effect of conversational pressure. An automatic scorer then labels whether each response makes an unsupported factual claim. Researchers commonly validate the scorer with one overall accuracy or $F_1$ score. However, high overall performance can hide prompt-specific mistakes: if the scorer makes more errors after one prompt, the measured difference can be too large, too small, or point in the wrong direction. An audit containing mostly difficult responses can also misrepresent performance in the full study. We derive how such errors alter a prompt comparison and examine the problem using six models, 180 invented-entity questions, and six prompt conditions. In a balanced audit of 288 responses, overall $F_1$ was $.916$. Yet, depending on the prompt, the scorer flagged 2.6%-33.3% of responses that the primary LLM rater judged not to contain an unsupported claim; these prompt-specific estimates were uncertain. In a second audit, the scorer and two separate LLM raters evaluated the same 300 model-question cases under each prompt. An apparent 9.3-point increase from asserting certainty rather than neutrally asking the model to reconsider fell to 2.3 points under both raters. The primary-rater interval included zero, so it did not establish an increase. Two other prompt comparisons kept their direction under both raters, while an attempted prompt-specific correction remained too uncertain for a firm conclusion. Researchers should therefore validate the exact difference they plan to report, not just the scorer as a whole: audit both prompts on matching cases and include scorer and rater uncertainty in the final result.

First seen by Oriel on .

AI summary

The paper shows that overall scorer accuracy can conceal prompt-specific errors that distort measured differences in interactive LLM evaluations, and recommends validating the exact comparison being reported.

  • A balanced audit of 288 responses achieved overall F1 of .916, while prompt-specific false-positive estimates ranged from 2.6% to 33.3% and remained uncertain.
  • Across 300 model-question cases, an apparent 9.3-point increase under an asserting-certainty prompt fell to 2.3 points under two LLM raters.
  • The primary-rater interval included zero, so the comparison did not establish an increase; two other comparisons retained their direction under both raters.
  • The authors recommend auditing both prompts on matching cases and incorporating scorer and rater uncertainty into the final result.

#llm evaluation #measurement #human evaluation

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
arXiv AI Papersoai:arXiv.org:2610.31678v1https://arxiv.org/abs/2610.31678 Read discussion