Research / Architecture
ANNIE V2: Diffusion-Inspired Cognition for a Self-Thinking Local LLM
A local-first AI architecture combining persistent memory, working state, background processing, and diffusion-inspired planning research.
Published on
Focus
Memory retrieval, internal state, motivation signals, and iterative refinement around a local model.
Principle
Keep the system local, observable, and grounded in real user context instead of canned responses.
ANNIE V2 extended the original assistant with persistent memory, a working representation of goals and constraints, and background processing between conversations. The work focused on continuity beyond a single prompt and response.
The architecture brings memory retrieval, compact state summaries, exploration signals, and iterative refinement into a local-first system with observable behavior. Diffusion-inspired planning formed a separate research component within that architecture.
Inside ANNIE V2 Chat
Chat view with the warming/online status indicator and navigation to Memory, Thinkers, and Code Review.
What ANNIE V2 Does
• Local chat with persistent memory: conversations are stored and can be recalled by semantic similarity, not just keywords.
• Belief state: a compact “working model” of goals + constraints that updates over time in the background.
• Background Thinker: an always-on loop that keeps exploring your goal space, generating “ticks” and pivots as it learns.
• Latent Future / forecasting mode: structured scenario thinking where the system organizes possibilities into actionable paths.
• Self-review signals: the system can surface internal notes (e.g., what it’s uncertain about, what it’s trying next) without dumping raw chain-of-thought.
Inside ANNIE V2 Memory and Processing
Memory timeline + long-term personal memory feed, with belief state, belief-vector decode (nearest memory), and optional manual belief injection.
Background Thinker loop: tick stream, curiosity/explore rates, and the active memory + sources panel.
How It Works (High Level)
ANNIE V2 is built around a small set of “cognitive primitives” that cooperate:
- Encode experience. Every new message becomes an embedding and is added to a vector index so it can be recalled later.
- Retrieve context. The system searches memory for relevant context instead of placing the entire archive into a prompt.
- Update beliefs. ANNIE periodically generates a compact, private summary of goals and constraints and stores a vector representation.
- Process in the background. A background loop revisits memory, tracks novelty and progress, and directs further exploration.
- Answer cleanly. The user receives the final response while internal processing remains separate.
The result is a system that behaves less like a one-shot chatbot and more like a tiny workspace: memory, state, and direction keep evolving even between messages.
The Diffusion Process (Planning in Latent Space)
The planning research used a diffusion-inspired approach. It represented a candidate plan as a sequence of latent vectors before translating that trajectory into text.
Conceptually, it looks like this:
• Start from noise (an unformed internal trajectory).
• Condition that trajectory on a blended signal: the current goal, the belief vector, and retrieved memory anchors.
• Run an iterative denoise process that nudges the latent trajectory toward coherence over multiple steps.
• Finally, decode each latent step by mapping it back to human-readable text using nearest-neighbor retrieval from memory.
The process repeatedly refines an internal representation, then translates that structure into candidate steps using retrieved memory. This is a planning experiment within V2, rather than a claim of human-like cognition.
The Engineering Principles
Five principles shaped the architecture and its evaluation:
- Continuity. Memory persists beyond the current context window.
- Self-modeling. State summaries compress goals and constraints into a working representation.
- Exploration. Curiosity signals support exploration beyond the highest-ranked memory result.
- Iterative refinement. Background loops revisit a direction across multiple steps.
- Observability. The interface exposes processing activity and state for inspection.
Where This Goes Next
ANNIE V2 brought long-term memory, background processing, and diffusion-inspired planning research into one local-first architecture. These components informed the later ANNIE work on persistent context, controlled execution, and evaluation. The development priority is making that continuity useful in everyday project work.