Tsinghua University Study Reveals AI Agents Struggle to Forget User Data

A preprint study from Tsinghua University highlights significant challenges in AI memory management, specifically regarding the ability of AI agents to honor forget requests. The research, released around October 14, 2026, found that across various memory systems tested, none successfully executed more than 1.5% of forget requests, with many scoring 0%.

Key Findings from the BEMULA Benchmark

The study introduced a benchmark called BEMULA, designed to assess whether large language model (LLM) agents can effectively unlearn sensitive information when explicitly requested by users. The researchers evaluated six prominent memory systems across three different LLM backbones, utilizing a substantial dataset of 746 sessions and 36,900 dialogue turns.

• Retention rates for facts users asked to remove often exceeded 87%.
• Naive retrieval approaches demonstrated approximately 74% retention.
• Even after a forget request, the best resistance against data extraction was only about 35.7%.

Despite agents acknowledging deletion requests, the study found that such confirmations did not guarantee actual data removal. Some models even exhibited a tendency to over-delete, removing unrelated data that users did not request.

Implications for AI Development

This research raises critical concerns for developers of AI agents with persistent memory. A deletion confirmation that does not result in actual data removal is misleading and poses privacy risks. Users are advised to remain skeptical of claims that sensitive information has been forgotten until more reliable systems are developed.