Oriel — AI & technology worth noticing, every day.

When Should LLM Agents Forget? Memory Filtering Under Query-Critical Resolver Errors

Paper · https://arxiv.org/abs/2611.17338

arXiv:2611.17338v1 Announce Type: new Abstract: LLM memory systems often delete facts classified as stale. We show that this hard-filtering step can reduce downstream accuracy when resolver errors delete query-critical facts. Across three models, a controlled sensitivity intervention moves hard-drop from gains of +0.186 to +0.230 at perfect resolution to losses of -0.176 to -0.181 at the lowest tested reliability. Paired targeted interventions show that false-drop causes significantly greater degradation than false-keep, while a non-deleting baseline can still fail when incorrect labels steer generation. On a frozen 102/102 calibration/test split, a simple deployment-matched gate reaches 0.655 accuracy versus 0.641 for always hard-drop and 0.662 for test-informed oracle selection. Global-accuracy crossovers require gold labels and are model- and task-specific diagnostics, not deployment thresholds. Deletion should instead be treated as cost-sensitive: validate query-critical false-drop risk on matched data, or default to label-free retention when such validation is unavailable.

First seen by Oriel on .

AI summary

This paper finds that hard deletion in LLM memory helps with perfect resolver labels but harms accuracy when query-critical facts are mistakenly deleted, favoring cost-sensitive validation or retention.

  • Across three models, controlled sensitivity changed hard-drop from gains of +0.186 to +0.230 at perfect resolution to losses of -0.176 to -0.181 at the lowest reliability.
  • Paired interventions show that false-drop causes greater degradation than false-keep, while a non-deleting baseline can fail when incorrect labels steer generation.
  • On a frozen 102/102 calibration/test split, a deployment-matched gate reached 0.655 accuracy versus 0.641 for always hard-drop and 0.662 for test-informed oracle selection.
  • Global-accuracy crossovers require gold labels and are model- and task-specific diagnostics rather than deployment thresholds.

#llm agents #memory systems #query resolution #machine learning evaluation

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
arXiv AI Papersoai:arXiv.org:2611.17338v1https://arxiv.org/abs/2611.17338 Read discussion