When Should LLM Agents Forget? Memory Filtering Under Query-Critical Resolver Errors
Paper · https://arxiv.org/abs/2611.17338
arXiv:2611.17338v1 Announce Type: new Abstract: LLM memory systems often delete facts classified as stale. We show that this hard-filtering step can reduce downstream accuracy when resolver errors delete query-critical facts. Across three models, a controlled sensitivity intervention moves hard-drop from gains of +0.186 to +0.230 at perfect resolution to losses of -0.176 to -0.181 at the lowest tested reliability. Paired targeted interventions show that false-drop causes significantly greater degradation than false-keep, while a non-deleting baseline can still fail when incorrect labels steer generation. On a frozen 102/102 calibration/test split, a simple deployment-matched gate reaches 0.655 accuracy versus 0.641 for always hard-drop and 0.662 for test-informed oracle selection. Global-accuracy crossovers require gold labels and are model- and task-specific diagnostics, not deployment thresholds. Deletion should instead be treated as cost-sensitive: validate query-critical false-drop risk on matched data, or default to label-free retention when such validation is unavailable.
First seen by Oriel on .
AI summary
This paper finds that hard deletion in LLM memory helps with perfect resolver labels but harms accuracy when query-critical facts are mistakenly deleted, favoring cost-sensitive validation or retention.
- Across three models, controlled sensitivity changed hard-drop from gains of +0.186 to +0.230 at perfect resolution to losses of -0.176 to -0.181 at the lowest reliability.
- Paired interventions show that false-drop causes greater degradation than false-keep, while a non-deleting baseline can fail when incorrect labels steer generation.
- On a frozen 102/102 calibration/test split, a deployment-matched gate reached 0.655 accuracy versus 0.641 for always hard-drop and 0.662 for test-informed oracle selection.
- Global-accuracy crossovers require gold labels and are model- and task-specific diagnostics rather than deployment thresholds.
#llm agents #memory systems #query resolution #machine learning evaluation
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| arXiv AI Papers | oai:arXiv.org:2611.17338v1 | https://arxiv.org/abs/2611.17338 Read discussion |