Who May Overrule the Agent? Evidence-Gated Authority in LLM Review Institutions
Paper · https://arxiv.org/abs/2611.05390
arXiv:2611.05390v1 Announce Type: new Abstract: Multi-agent language-model systems often add a reviewer to improve an upstream agent's decision. Yet a reviewer can both repair errors and replace decisions that were already correct. We study this as a problem of institutional jurisdiction: conditional on the same proposal, task information, model, and review output, when is the reviewer authorized to make its recommendation binding? We compare broad override authority with an evidence gate that permits replacement only after the reviewer produces a mechanically valid record of a binding-constraint violation and a compliant repair. Across two locally run open-weight models, 96 spatial-policy tasks, and a 96-task portfolio-transfer benchmark, the gate sharply reduces corruption of correct proposals while retaining most repairs of violations. The directions of this tradeoff are partly implied by the rule; its magnitude depends on reviewer transitions and upstream proposal states. A prospective validation then gives the reviewer an explicit choice to retain, replace, or escalate. This eliminates correct-proposal corruption for Qwen, but Mistral still replaces all 48 correct proposals; Qwen also declines every welfare-only repair that Mistral frequently attempts. Thus interface design changes reviewer behavior but is not a model-independent safeguard. The result is not that restricted review is universally better. It is that reviewer competence, action interface, and binding jurisdiction jointly determine both the correction domain and the corruption surface.
First seen by Oriel on .
AI summary
The paper evaluates evidence-gated authority for LLM reviewers, finding that limiting overrides to documented constraint violations reduces corruption of correct proposals, but results vary by model.
- The gate allows replacement only when the reviewer provides a mechanically valid record of a binding-constraint violation and a compliant repair.
- Tests covered two locally run open-weight models, 96 spatial-policy tasks, and a 96-task portfolio-transfer benchmark.
- The gate retained most repairs while sharply reducing overrides of correct proposals; the size of the tradeoff depended on reviewer transitions and proposal states.
- In prospective validation, Qwen avoided corrupting correct proposals but declined every welfare-only repair, while Mistral replaced all 48 correct proposals.
- The authors conclude that reviewer competence, interface design, and the scope of binding authority jointly shape corrections and corruption.
#llm agents #multi-agent systems #ai governance
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| arXiv AI Papers | oai:arXiv.org:2611.05390v1 | https://arxiv.org/abs/2611.05390 Read discussion |