admin/obsidian-vault
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
obsidian-vault/Projects/Memory System/Memory System Architecture.md
AlexAI 7ec760e7bb vault backup: 2026-02-25 23:00:07
2026-02-25 23:00:07 -06:00

9.3 KiB
Raw Permalink Blame History

title, category, project, type, status, date, last_updated, tags, source_file
title category project type status date last_updated tags source_file
Memory System Architecture Projects Memory System Architecture/Documentation complete 2026-02-16 2026-02-23
memory
architecture
openclaw
system-design
data-flow
memory_system_architecture.md

Memory System Architecture

Diagram of how information flows and persists in the OpenClaw system

Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                         INFORMATION FLOW                                     │
└─────────────────────────────────────────────────────────────────────────────┘

    User Conversation
           │
           ▼
┌─────────────────────┐     ┌─────────────────────┐     ┌─────────────────────┐
│   ME (Main Agent)   │────▶│   Memory Worker     │────▶│   SQLite Database   │
│   (Real-time)       │     │   (Daily 3 AM)      │     │   (Structured)      │
└─────────────────────┘     └─────────────────────┘     └─────────────────────┘
           │                           │                           │
           │                           │                           │
           ▼                           ▼                           ▼
┌─────────────────────┐     ┌─────────────────────┐     ┌─────────────────────┐
│   Daily Notes       │     │   Query Interface   │     │   Stats/Search      │
│   (memory/*.md)     │◄────│   (On Demand)      │◄────│   (SQL/FTS)        │
└─────────────────────┘     └─────────────────────┘     └─────────────────────┘
           │
           ▼
┌─────────────────────┐
│   MEMORY.md         │
│   (Curated)         │
└─────────────────────┘
           │
           ▼
┌─────────────────────┐
│   Supermemory.ai    │
│   (Cloud Backup)    │
└─────────────────────┘

Storage Layers (By Speed & Persistence)

1. ⚡ Session RAM (Transient)

Aspect Details
What Current conversation context, tool outputs, working memory
Writes Every message I process
When Real-time during conversation
Stored Until session ends or compaction (30 min - 4 hours)
Survives ❌ Session crash ❌ Gateway restart
Size ~100K-250K tokens

Risk: Compaction clears this. The "danger zone" is between last tool use and compaction.

2. 📝 Daily Notes (Short-term)

Aspect Details
What Raw daily activity, decisions, tasks, errors
Writes Pre-compaction flush (automatic) + manual captures
When End of productive sessions, before /compact
Stored memory/YYYY-MM-DD.md
Survives ✅ Session crash ✅ Gateway restart
Retention ~30-90 days (manually reviewed)
Format Free-form markdown

Written by: Me (main agent) during heartbeat or EOD ritual

3. 🧠 MEMORY.md (Long-term)

Aspect Details
What Curated important info, distilled from daily notes
Writes Manual review of daily notes, during heartbeats
When Every few days, or when something critical happens
Stored MEMORY.md in workspace root
Survives ✅ Everything (file-based)
Retention Permanent (manual curation)
Format Human-readable markdown

Written by: Me, after reviewing daily notes

4. 📊 SQLite Database (Structured)

Aspect Details
What Structured: tasks, decisions, facts, projects with salience
Writes Memory Worker (automated daily extraction)
When Daily 3:00 AM (cron job)
Stored ~/.openclaw/memory.db
Survives ✅ File-based
Retention Permanent (until manual deletion)
Format Relational: cells, scenes, FTS index

Written by: Memory Worker agent (spawned via cron)

Schema:

memory_cells: id, scene, cell_type, salience, content, source_file, created_at
scenes: scene, summary, item_count, updated_at
memory_fts: full-text search index

5. 🌐 Supermemory.ai (Cloud)

Aspect Details
What Full backup of all memory files
Writes Supermemory Backup job (automated)
When Daily 2:00 AM
Stored Supermemory.ai cloud service
Survives ✅ Disk failure ✅ Workspace loss
Retention Cloud provider dependent
Format API-uploaded documents

Written by: Python script via cron job

6. 📋 Workspace Context (Session Bridge)

Aspect Details
What Current conversation, in-progress, finished today
Writes Real-time during session
When Continuously updated
Stored workspace-context.md
Survives ✅ Session crash ✅ Channel switch
Retention Cleared nightly (~11 PM)
Format Structured markdown

Special: Survives between channels and session crashes. Cleared daily.

Retention Summary

Layer Retention Cleared When Backup
Session RAM Minutes-hours Compaction ❌
Workspace Context ~24 hours 11 PM nightly ❌
Daily Notes 30-90 days Manual archive Supermemory
MEMORY.md Permanent Manual edit Supermemory
SQLite DB Permanent Manual delete ❌ (local only)
Supermemory Permanent Cloud provider N/A (is backup)

Write Triggers

Every Message
      │
      ├─► Session RAM (immediate)
      │
      └─► If important ┐
                       ▼
              workspace-context.md
                       │
          Pre-compaction ┤
                       ▼
              memory/YYYY-MM-DD.md
                       │
         Periodic review ┤
                       ▼
                   MEMORY.md
                       │
              Daily 2 AM ┤
                       ▼
                 Supermemory.ai
                       │
              Daily 3 AM ┤
                       ▼
                 SQLite Database

Access Patterns

I (Main Agent) Access:

Source When Purpose
MEMORY.md Every session startup Core identity, user prefs, important facts
USER.md Every session startup Who Corey is
SOUL.md Every session startup How I should behave
workspace-context.md Every session startup Current conversation state
memory/*.md During heartbeats Recent context
SQLite DB On demand Structured queries ("what tasks pending?")

Memory Worker Access:

Source When Purpose
IDENTITY.md Daily 3 AM Who it is
SOUL.md Daily 3 AM Its mission
HEARTBEAT.md Daily 3 AM What to do (the script)
memory/YYYY-MM-DD.md Daily 3 AM What to extract
SQLite DB Daily 3 AM Where to write

Failure Recovery

Scenario: Session Crash

  • ✅ Survives: Files (MEMORY.md, daily notes, workspace-context)
  • ❌ Lost: Session RAM (compaction would have cleared anyway)
  • 🔄 Recovery: Read files on restart, reconstruct context

Scenario: Gateway Restart

  • ✅ Survives: All files, SQLite DB
  • ❌ Lost: Session state, cron job state (must recreate jobs)
  • 🔄 Recovery: Gateway restart, verify cron jobs running

Scenario: Disk Failure

  • ✅ Survives: Supermemory.ai (cloud backup)
  • ❌ Lost: Local files, SQLite DB
  • 🔄 Recovery: Restore from Supermemory, recreate DB (re-extract from notes)

Key Insights

  1. Text > Brain — Files persist, my session doesn't
  2. Daily notes = raw, MEMORY.md = curated — Filter noise from signal
  3. Worker = automated structuring — Don't have to manually organize everything
  4. Hybrid = best of both — Human-readable + machine-queryable
  5. Multiple backups — Local files + cloud (Supermemory) + structured DB

Related Files

  • Memory System Evolution — OpenClaw summary (in OpenClaw/ folder)
  • MEMORY.md — Curated long-term memory
  • memory/YYYY-MM-DD.md — Daily notes
  • workspace-context.md — Session bridge
  • ~/.openclaw/memory.db — Structured SQLite database

Generated: 2026-02-16
Copied to Obsidian: 2026-02-23
System Version: Multi-agent with SQLite extraction

Reference in New Issue View Git Blame Copy Permalink