The Best Second Opinion Is Right Where the Primary Is Wrong: Partner Selection for Selective Span Tagging
Paper · https://arxiv.org/abs/2610.37678
arXiv:2610.37678v1 Announce Type: new Abstract: A span tagger that ranks its outputs by risk can defer the riskiest ones to a person, and a second model can sharpen this ranking. Which second model should it be? Partner selection is conditional, not global: the best second opinion is the model that is right where the primary is wrong. Across eight tagger types, three seeds and three English benchmarks (576 evaluated primary-partner pairings), choosing the partner by held-out accuracy on the primary's errors recovers 82% of the best partner's gain, against 69-70% for span F1 or a random pick. Frozen before a fourth dataset was chosen, the rule replicated there (Spearman 0.40 vs. 0.04 for span F1). Partner value is also directional. The predictive distribution is the second opinion: hard labels discard half of a partner's value, and scoring the served label improves 94% of pairings. Fixed before the fourth dataset was chosen, this score also replicated there. For LoRA-tuned decoders, independently trained partners beat every readout of the primary network. Applied zero-shot to 51 MASSIVE locales, every primary has a helpful trained partner in every locale, and the preferred partner follows competence.
First seen by Oriel on .
AI summary
The paper finds that selecting a second span tagger based on its accuracy specifically on the primary model’s errors improves selective tagging decisions across datasets and locales.
- Across 576 primary-partner pairings, error-conditioned selection recovers 82% of the best partner’s gain, versus 69–70% for selection by span F1 or random choice.
- The selection rule replicated on a fourth dataset held out from the initial analysis, with Spearman correlation 0.40 versus 0.04 for span F1.
- Using a partner’s predictive distribution rather than hard labels preserves more value, and scoring the served label improves 94% of pairings.
- For LoRA-tuned decoders, independently trained partners outperform readouts of the primary network; in 51 MASSIVE locales, every primary has a helpful trained partner.
#span tagging #selective prediction #model ensembles #lora
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| arXiv AI Papers | oai:arXiv.org:2610.37678v1 | https://arxiv.org/abs/2610.37678 Read discussion |