Tell Me What To Learn: Generalizing Neural Memory to be Controllable in Natural Language

Modern machine learning models are deployed in diverse, non-stationary environments where they must continually adapt to new tasks and evolving knowle...

Core Idea

This paper introduces a controllable neural memory system where the model updates what it stores based on natural-language instructions (e.g., “remember user preferences,” “ignore rumors,” “only store medically verified facts”). Instead of assuming a single fixed “learn everything” objective, memory updates become selective and policy-driven, allowing the same agent to adapt differently depending on the context.

Why this paper matters: In real products, information streams are messy (tickets, chats, logs, documents) and requirements change. A memory system that can be steered reduces costly fine-tuning, avoids “remembering the wrong thing,” and supports compliance and safety constraints.

Technical Details

Key innovations (in plain terms)

  • Instruction-conditioned memory updates: Memory isn’t updated automatically; an instruction in natural language controls whether to write, what to write, how strongly to write, and when to overwrite.
  • Heterogeneous source handling: The system can treat different inputs differently (e.g., “customer says X” vs “policy doc says Y”), rather than assuming all tokens are equally trustworthy.
  • Generalized objective for memory: Prior neural memory often optimizes a single goal like “store whatever helps next-token prediction.” This work generalizes the objective to include user intent expressed as text.

Important algorithms / architectures (conceptual)

While details aren’t in the abstract, a typical realization (and what the paper is claiming) looks like:

  1. Base model (LLM) + external neural memory

    • The LLM handles reasoning and language.
    • A memory module stores key-value representations (embeddings) that can be retrieved later.

  2. Memory controller conditioned on instruction

    • Input: (instruction, current observation, optional context like source type)
    • Output: update decisions such as:
      • Write gate: store vs ignore
      • Update rule: append new entry vs edit/overwrite existing
      • Strength/priority: how retrievable/important this memory should be
      • Scope: per-user vs global vs session-only memory

  3. Retrieval integrated into generation

    • At query time, the model retrieves relevant memories (e.g., via similarity search over embeddings) and uses them as additional context.

Novel approaches introduced

  • Natural-language “memory policy” as a first-class control signal: Instead of hardcoding rules (like filters) or retraining for each policy, the system takes instructions directly.
  • Selective continual learning without full fine-tuning: Updates happen through lightweight memory writes/edits, reducing catastrophic forgetting and reducing compute.
  • Controllability over time: The paper’s framing implies you can change instructions later (“from now on, only store verified sources”) and the system adapts its future memory behavior accordingly.

How This Relates to Interviews

System design relevance

  • Designing an AI agent with long-term memory: How to store user preferences, prior interactions, and evolving facts without constantly retraining.
  • Data governance and safety: How to prevent storing sensitive data, misinformation, or low-confidence content; how to enforce retention policies.
  • Operating in non-stationary environments: Handling changing requirements, new products, new policies, and drifting user needs.

Common interview scenarios where this applies

  • Customer support agent memory
    • Remember: account preferences, ongoing issues
    • Ignore: angry rants, unverified claims
    • Update: “issue resolved” should overwrite “issue ongoing”
  • Healthcare assistant
    • Remember: clinician-confirmed facts
    • Ignore: speculative symptoms or unverified sources
    • Scope: per-patient memory with strict privacy constraints
  • Enterprise knowledge assistant
    • Prefer: internal policy docs over chat messages
    • TTL: store some items temporarily (session) and some long-term (org memory)

Key concepts to understand

  • Neural memory: A learned store of vector representations used for retrieval, not just a raw database.
  • Write/forget gating: Mechanisms that decide what becomes persistent.
  • Catastrophic forgetting: Fine-tuned models can lose older capabilities; external memory reduces this risk.
  • Instruction-following as control: Using natural language to specify behavior instead of code-only rules.

Key Takeaways

  • Controllable memory updates: The model can be told what to learn and what to ignore using natural language instructions.
  • Better fit for real-world streams: Supports heterogeneous inputs (different trust levels, different sources) rather than treating all data equally.
  • Cheaper continual adaptation: Avoids frequent fine-tuning by using lightweight memory writes/edits.
  • Safety and compliance leverage: Enables policies like “don’t store PII,” “only store verified facts,” or “store per-user only.”
  • Practical applications: Customer service agents, healthcare assistants, enterprise knowledge tools, and any long-lived agent operating in changing environments.