vault backup: 2026-02-25 23:00:07

Projects/Memory System/Memory System Evolution.md

@@ -0,0 +1,144 @@
+#Memory
+
+## Architecture Evolution
+Started as flat file system, evolved to hierarchical hybrid (file + SQL) architecture.
+
+---
+
+## Current Stack (February 2026)
+
+### 1. Hierarchical File Structure
+Based on ucsandman's design — index + drill-down:
+
+`
+memory/
+├── MEMORY.md                 # ~2.4k token index (was 5-10k)
+├── people/
+│   └── corey.md              # User details
+├── projects/
+│   ├── mission-control-python.md
+│   ├── home-assistant.md
+│   ├── memory-system.md
+│   └── [others]
+├── decisions/
+│   └── 2026-02.md            # Monthly decisions
+└── 2026-02-21.md             # Daily notes
+`
+
+**Benefits:**
+- Session start: ~2.4k tokens (was 5-10k)
+- On-demand drill-down: ~1k tokens per file
+- **~70% token savings**
+
+### 2. SQLite Database
+- **Location:** ~/.openclaw/memory.db
+- **Tables:**
+  - memory_cells — Structured data
+  - scenes — Context/grouping
+  - memory_fts — Full-text search
+- **Extraction:** Memory Worker agent (3 AM daily)
+
+### 3. Supermemory.ai Backup
+- **Schedule:** Daily 2 AM
+- **Script:** scripts/backup-memory.py
+- **History:** Document IDs tracked per backup
+
+### 4. MySQL Mirror (Legacy)
+- **Host:** localhost:3306
+- **Database:** clawdbot_projects
+- **Tables:** daily_notes, long_term_memories, action_items, backup_logs, sessions
+- **User:** AlexAI/alexisabignerd
+- **Status:** Functional but not actively used
+
+---
+
+## Key Decisions
+
+| Date | Decision | Status |
+|------|----------|--------|
+| 2026-02-09 | Supermemory backup with cron | Active |
+| 2026-02-12 | workspace-context.md for temp state | Active |
+| 2026-02-16 | Hybrid File + SQLite (not MySQL) | Active |
+| 2026-02-18 | Hierarchical index + drill-down | Live |
+| 2026-02-20 | Memory Worker writes to SQLite | Working |
+
+---
+
+## Ollama Memory Embeddings (2026-02-10)
+
+**Setup:**
+- Installed skill from https://clawhub.ai/vidarbrekke/ollama-memory-embeddings
+- Uses Ollama /v1/embeddings endpoint
+- Replaces node-llama-cpp for local embeddings
+
+**Trade-off:**
+- ✅ Faster, OpenAI-compatible API
+- ⚠️ Two embedding universes: old (node-llama-cpp) vs new (Ollama)
+- Historical memories may search differently
+
+---
+
+## Workers
+
+### Memory Worker Daily
+- **Schedule:** 3 AM
+- **Task:** Extract structured data → SQLite
+- **Extraction source:** memory/YYYY-MM-DD.md files
+- **Output:** memory.db + Discord report
+
+### Job Verifier Daily  
+- **Schedule:** 9 AM
+- **Task:** Check overnight job statuses
+- **Reports to:** #alerts channel
+
+---
+
+## Integration Points
+
+| System | Direction | Method |
+|--------|-----------|--------|
+| OpenClaw Memory Search | Read | semantic search |
+| SQLite Database | Read/Write | memory tool |
+| Supermemory | Write (backup) | API |
+| Obsidian Vault | Write (export) | notesmd-cli |
+| MySQL | Legacy | Direct SQL |
+
+---
+
+## Skills Related
+
+| Skill | Purpose |
+|-------|---------|
+| ollama-memory-embeddings | Ollama-based embeddings |
+| memory-to-obsidian | Export notes to Obsidian |
+| ez-cronjob | Fix cron scheduling issues |
+| self-improving-agent | Log errors/corrections |
+
+---
+
+## Troubleshooting Notes
+
+**Cron deadlock bug:**
+- Issue: cron tool hangs
+- Fix: Use openclaw cron CLI via exec instead
+
+**Context bloat:**
+- Issue: 256k limit hit frequently
+- Fix: Spawn sub-agents for complex tasks
+
+**Session recovery:**
+- workspace-context.md survives crashes
+- Cleaned daily at ~11 PM
+
+---
+
+## Future Ideas
+
+1. SQL → HA Dashboard integration
+2. Automated presence tracking export
+3. Memory stats/analytics in Mission Control
+4. Re-embed historical memories for consistency
+
+---
+
+Status: Live | Last Major Update: 2026-02-18 (Hierarchical restructure)