Oriel — AI & technology worth noticing, every day.

From Target Selection to Condition Reuse: Auditing Residual Interventions in Tool-Use Models

Paper · https://arxiv.org/abs/2610.31686

arXiv:2610.31686v1 Announce Type: new Abstract: Activation interventions can make a language model produce a target answer, but can the model use the intended condition in other computations? We study this question in Qwen3.5-27B on tool-use tasks where time or permissions determine eligibility and selection. Across 32 fresh worlds, two case-specific residual-writing variants raise familiar target-selection correctness from 35/128 to 119/128 and from 33/128 to 114/128. Yet these gains disappear when familiar facts and identifiers are recombined within the same selection task, while selection of previously rewarded identifiers increases. The gains persist across output formats but yield little improvement in independent judgments about whether a candidate is eligible or should be selected. Internal edits can nevertheless change how a source fact affects a judgment: in a prespecified secondary comparison, an early-layer source patch changes 50/64 judgments when that source is used, with one adverse change among 64 cases where it is not. This selective effect does not ensure correct complete actions. Together, these results separate target-selection control, condition-sensitive judgments, and complete-action correctness. Evaluating how an edited state is used, rather than only its trained answer, exposes these differences.

First seen by Oriel on .

AI summary

The study finds that residual interventions improve familiar target selection in Qwen3.5-27B, but gains fail under recombination and do not reliably improve eligibility judgments or complete actions.

  • Across 32 fresh worlds, two case-specific residual-writing variants raised familiar selection correctness from 35/128 to 119/128 and from 33/128 to 114/128.
  • When familiar facts and identifiers were recombined, the gains disappeared while selection of previously rewarded identifiers increased.
  • Improvements persisted across output formats but yielded little improvement in independent judgments about eligibility or whether to select a candidate.
  • An early-layer source patch changed 50/64 judgments when the source was used, with one adverse change among 64 cases where it was not.
  • The results distinguish target-selection control, condition-sensitive judgments, and complete-action correctness in tool-use tasks governed by time or permissions.

#language models #activation interventions #tool use #model auditing

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
arXiv AI Papersoai:arXiv.org:2610.31686v1https://arxiv.org/abs/2610.31686 Read discussion