Oriel — AI & technology worth noticing, every day.

MedToolFault and MedGuard: evaluating and improving clinical agent reliability

Paper · https://arxiv.org/abs/2611.09942

arXiv:2611.09942v1 Announce Type: new Abstract: Clinical agents must retrieve patient-specific evidence and correctly update records despite unreliable tool responses. We introduce MedToolFault, a fault-injection benchmark linking six tool-failure classes to answer correctness, required state changes and forbidden mutations in public clinical-agent tasks. We also introduce MedGuard, a tool-layer method that verifies read evidence and attributes ambiguous writes through public interfaces. Across 127,861 formal episode evaluations, we assess fault recovery, mechanism controls, task expansion, constrained tool budgets and external question answering. Across nine model configurations, unguarded agents fail in 32.2-67.8% of fault-injected evaluations on tasks each configuration solves without faults. MedGuard improves paired safe success by 13.7-46.6 percentage points on these tasks.

First seen by Oriel on .

AI summary

MedToolFault benchmarks clinical-agent reliability under tool failures, while MedGuard verifies evidence and ambiguous writes, improving safe task completion across nine model configurations.

  • The benchmark links six tool-failure classes to answer correctness, required state changes, and forbidden mutations in public clinical-agent tasks.
  • MedGuard operates at the tool layer, verifying read evidence and attributing ambiguous writes through public interfaces.
  • The study reports 127,861 formal episode evaluations spanning fault recovery, mechanism controls, task expansion, constrained tool budgets, and external question answering.
  • Without faults, the evaluated configurations solve the tasks; with injected faults, unguarded agents fail in 32.2-67.8% of evaluations.
  • MedGuard raises paired safe success by 13.7-46.6 percentage points on those tasks.

#clinical agents #agent reliability #fault injection #healthcare ai

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
arXiv AI Papersoai:arXiv.org:2611.09942v1https://arxiv.org/abs/2611.09942 Read discussion