Self-Curating AI Memory: How My Local Agent Autonomously Writes and Manages Its Own Knowledge Base in Obsidian

By | November 7, 2026
0 Comment

Series Navigation:

In the first two installments of this series, I solved two foundational problems for my local CLI coding and infrastructure assistant, Google Antigravity (agy):

  1. Episodic Archiving: Automatically syncing SQLite session histories (conversation_summaries.db) into Obsidian via systemd --user path watchers.
  2. Sub-Second Retrieval: Bridging Obsidian’s native Omnisearch HTTP API (localhost:51361) with a zero-dependency Python fallback scanner to search 3,000+ notes in under 250 milliseconds.

Those two layers gave my agent eyes into the past. What they did not give it was agency over its own memory.

Early on, my memory workflow still relied on human-in-the-loop triggers: I had to explicitly tell the agent “remember this” or “log that fix to your memory file.” In practice, when you are deep in the middle of debugging a failing Fedora 45 system upgrade on a NAS node at midnight or fixing an LTFS tape mount race condition, you never remember to tell your assistant to take notes.

To truly eliminate agent amnesia, the agent must act as its own autonomous librarian—proactively distilling root causes, updating working state, recording audit trails, and pruning stale context without waiting for human reminders.

Here is the complete architectural breakdown of how Antigravity autonomously writes, structures, and manages its own memory inside my Obsidian vault (/home/gvoina/Vault/George/).

1. The Core Philosophy: Filesystems as Cognitive Architecture

Most agent memory frameworks hide knowledge inside opaque SQLite blobs or remote vector databases. When the agent hallucinates an outdated IP address or repeats a solved mistake, inspecting why requires querying embeddings rather than reading plain text.

By using an Obsidian Vault as the agent’s primary memory substrate, every piece of knowledge is:

  • Human-Inspectable & Editable: Plain Markdown files with YAML frontmatter that render natively in my daily PKM graph.
  • Synced Everywhere: Propagated in real time to my phone (S25Ultra) over Tailscale via Syncthing, and backed up weekly to physical LTO-4 cold-storage tape on nas2.
  • Categorized by Cognitive Lifespan: Instead of dumping everything into a single giant AGENTS.md or CLAUDE.md file (which bloats the prompt on every turn), memory is partitioned across distinct files inside /home/gvoina/Vault/George/Meta/ based on how rapidly the information changes.

2. The 4-Tier Memory Topology in Meta/

Rather than treating “memory” as a monolith, the vault divides agent cognition into four distinct tiers:

Tier 1: Working Memory (Meta/states/antigravity.md)

Modeled after a physical sticky note on a monitor, this file holds short-term operational context. It tracks what the agent is currently focusing on, the last execution timestamp, a bounded list of recent milestones, and active reminders:

---
agent: antigravity
last-run: "2026-11-07T00:03:20+02:00"
---

## Post-it

### Active Working Context
- **Workspace**: `/home/gvoina`
- **Active Focus**: Autonomous Memory Management & Fleet System Maintenance.
- **Recent Milestones**:
  - Activated autonomous memory management directives in `~/.agents/AGENTS.md`.
  - Diagnosed and resolved nas3 Fedora 45 system upgrade (`--allowerasing` & `nodejs-libs` fix).
  - Automated nas1 HP Ultrium 5-SCSI LTO-5 LTFS tape mounting at boot via idempotent `mount-ltfs.service`.
  - Synced 7 newly added/ordered vinyl releases with Discogs & Obsidian (340 total releases).

### Ongoing Tasks & Reminders
- Monitor weekly tape backup runs via `Meta/tape-backups.md`.
- Rotate tape cartridges on a monthly basis.

Tier 2: Semantic Long-Term Memory (Meta/antigravity-memory.md)

This is the agent’s permanent engineering encyclopedia. Instead of raw chat logs, it stores distilled, verified facts organized into five structured sections:

  1. User & Environment Profile: Workspace paths, core OS platform, vault root.
  2. Infrastructure & Fleet Notes: Exact network topology and hardware quirks for nas1 (blog.voina.org), nas2 (192.168.2.22 via jump host, LTO-4 tape drive), nas3 (192.168.1.50, iLO 4 at 192.168.25.174), Meshtastic LoRa nodes, and Wise 2FA signing keys.
  3. Engineering & Operational Principles: Forward-fix-only policy, script registration discipline, financial confirmation rules.
  4. Cross-Session Project Tracking: Pointers to active workflows (Discogs/Obsidian vinyl pipeline, Home Assistant integrations, tape archival).
  5. Notes & Learnings Log: Timestamped one-line technical discoveries distilled from resolved incidents.

Tier 3: Identity & Preference Memory (Meta/user-profile.md)

Captures persistent human attributes—languages, professional role, life areas, hardware/audio gear (such as Meze 99 Classics headphones), and personal preferences—shared across all vault personas.

Tier 4: Episodic Audit Trail (Meta/agent-log.md & Meta/antigravity-session-history.md)

Every state-mutating action taken by the agent is appended chronologically under a ## YYYY-MM-DD header in Meta/agent-log.md. If I ever want to know when a package was removed, when a systemd unit was reloaded, or when a Discogs sync occurred, Meta/agent-log.md provides an immutable paper trail.

3. Encoding Autonomy in ~/.agents/AGENTS.md

An LLM only acts autonomously if its system prompt grants it explicit standing authority and defines clear triggers for when to write memory.

In /home/gvoina/.agents/AGENTS.md, Section 9 establishes five non-negotiable Autonomous Memory Management Directives:

* **Autonomous Memory Management Directives**:
  * **Self-Directed Distillation**: You are explicitly authorized and required to autonomously record permanent engineering discoveries, root-cause resolutions, hardware quirks, and user preferences into `Meta/antigravity-memory.md` without waiting for explicit prompt instructions.
  * **Continuous Context & State Tracking**: Maintain `Meta/states/antigravity.md` with active working context, ongoing tasks, and current infrastructure states across sessions.
  * **Automated Audit Logging**: Every major system change, package upgrade resolution, hardware or service fix, or media sync MUST be autonomously appended to `Meta/agent-log.md` using `python3 /home/gvoina/scripts/obsidian_memory.py log "<message>"`.
  * **Memory Hygiene & Curation**: Autonomously deduplicate, categorize, and prune obsolete state entries in `Meta/` to keep semantic retrieval razor sharp.
  * **Pre-Task Memory Consultation**: When starting or resuming complex tasks involving user preferences, personal projects, or fleet topology, always query relevant memory notes in `Meta/`.

This shifts the interaction paradigm from reactive note-taking (“save this if I tell you to”) to proactive engineering hygiene (“every resolved root cause is incomplete until it is distilled into memory”).

4. The CLI Memory Bridge: obsidian_memory.py

To make reading and updating these files fast, deterministic, and atomic during a session, the agent uses /home/gvoina/scripts/obsidian_memory.py.

# View current working state and latest audit log entries
python3 /home/gvoina/scripts/obsidian_memory.py status

# Read a specific cognitive tier (memory | state | log | profile | dreams)
python3 /home/gvoina/scripts/obsidian_memory.py read memory

# Append an automated milestone to Meta/agent-log.md and bump state timestamp
python3 /home/gvoina/scripts/obsidian_memory.py log "Resolved nas3 Fedora 45 RPM conflict and completed upgrade."

# Append a distilled permanent fact to Meta/antigravity-memory.md
python3 /home/gvoina/scripts/obsidian_memory.py add-note "nas1 LTFS auto-mount requires /usr/local/bin/ltfs and mountpoint -q guard."

# Search across the entire 3,000+ note vault via Omnisearch + local fallback
python3 /home/gvoina/scripts/obsidian_memory.py search "Slipknot vinyl"

When cmd_log runs, it automatically locates or creates today’s ## YYYY-MM-DD section in Meta/agent-log.md, inserts the entry at the top of the day’s ledger, and updates the last-run ISO-8601 timestamp inside Meta/states/antigravity.md in a single pass:

def cmd_log(args):
    ensure_meta_dirs()
    message = args.message.strip()
    today = get_today_str()
    header = f"## {today}"
    entry = f"- **Antigravity**: {message}"

    content = LOG_FILE.read_text() if LOG_FILE.exists() else "# Agent Log\n\n"
    if header in content:
        idx = content.find(header)
        line_end = content.find("\n", idx)
        content = content[: line_end + 1] + f"{entry}\n" + content[line_end + 1 :]
    else:
        match = re.search(r"(## \d{4}-\d{2}-\d{2})", content)
        if match:
            idx = match.start()
            content = content[:idx] + f"{header}\n{entry}\n\n" + content[idx:]
        else:
            content += f"\n{header}\n{entry}\n"

    LOG_FILE.write_text(content)

5. Three Real-World Case Studies from This Week

To see how autonomous memory works in practice, here are three real tasks from a single multi-day session and how the agent used and updated its Obsidian memory across them.

Case Study 1: Diagnosing a Fedora 45 Upgrade Failure on nas3

When upgrading nas3 (192.168.1.50) from Fedora 44 to Fedora 45, the offline transaction test failed after downloading 4,145 packages (6.7 GB).

  1. Pre-Task Consultation: The agent read its fleet topology rules to connect directly to gvoina@192.168.1.50 without asking for an IP address.
  2. Live Diagnosis: It discovered two distinct traps:
    • DNF5 Flag Syntax: In DNF5, --allowerasing must be placed after the subcommand (dnf system-upgrade download --releasever=45 --allowerasing), otherwise dnf5 aborts with Unknown argument "--allowerasing" for command "dnf5".
    • Orphaned Package File Collision: Fedora 45 split Node.js into versioned packages (nodejs22-libs), colliding on /usr/lib64/libnode.so.127 with an orphaned Fedora 43 package (nodejs-libs-22.22.2-2.fc43).
  3. Autonomous Distillation: After verifying that nothing critical required nodejs* (rpm -q --whatrequires) and resolving the upgrade via dnf remove -y "nodejs*", the agent recorded both the DNF5 syntax rule, the nodejs-libs collision fix, and the 7 GB / partition headroom requirement directly into Meta/antigravity-memory.md. Next April, when Fedora 46 arrives, the agent will already know these exact constraints.

Case Study 2: Engineering an Idempotent LTFS Tape Mount on nas1

After rebooting nas1 (blog.voina.org:2222), the HP Ultrium 5-SCSI LTO-5 tape drive (/dev/st0) was not mounting at /media/lto5.

  1. Live Inspection: The agent inspected systemctl status mount-ltfs.service on nas1 and found two issues:
    • First, ExecStart pointed to /usr/bin/ltfs (status=203/EXEC), whereas HPE StoreOpen LTFS was installed at /usr/local/bin/ltfs.
    • Second, when the tape was already mounted by an existing ltfs process (PID 6302), starting the systemd unit failed with LTFS20035E Unable to lock device (Resource temporarily unavailable).
  2. Idempotent Fix: The agent engineered an idempotent unit file using mountpoint -q /media/lto5:[Service] Type=oneshot RemainAfterExit=yes ExecStart=/usr/bin/bash -c 'if ! mountpoint -q /media/lto5; then /usr/local/bin/ltfs -o devname=/dev/st0 /media/lto5; fi' ExecStop=/usr/bin/bash -c 'if mountpoint -q /media/lto5; then /usr/bin/umount /media/lto5; fi'
  3. Autonomous Distillation: It immediately updated nas1‘s hardware profile in Meta/antigravity-memory.md to document the HP Ultrium 5-SCSI /dev/st0 drive, /media/lto5 mount point, and mount-ltfs.service configuration.

Case Study 3: Multimodal Vinyl Shopping Cart Analysis & Discogs Sync

While browsing a flash sale on Niche Records, I dropped a screenshot of my shopping cart (3 Vitalic 2026 represses and 3 Slipknot limited color vinyl editions) and asked if I should buy them.

  1. Vault Cross-Reference: The agent queried /home/gvoina/Vault/George/03-Resources/Music/ and confirmed I only owned Dissidænce (Episode 2) and Disco Boy by Vitalic, and .5: The Gray Chapter, We Are Not Your Kind, and The End For Now… by Slipknot—proving zero duplicates and showing that the cart completed Vitalic’s core studio trilogy and Slipknot’s classic run.
  2. Barcode & Market Verification: Using the EAN barcodes extracted from the screenshot (3516628520914, 0075678645730, etc.), it queried the Discogs API and European retailers (jpc.de, HHV.de, Cărturești, eMAG) to verify that the 15% discounted prices beat EU import costs once shipping was factored in.
  3. Zero-Touch Post-Order Sync: When I replied “OK I did the order”, the agent automatically sent POST requests to the Discogs API to add all 6 release IDs to my collection, ran /home/gvoina/scripts/import_discogs.py to generate the Obsidian notes in 03-Resources/Music/, detected that the brand-new 2026 Rave Age release (#38641539) lacked cover art on its sub-release page, fetched the primary artwork from the Discogs Master release (#489502), embedded it into Vitalic - Rave Age.md, and logged the new 340-release catalog milestone to Meta/agent-log.md and Meta/antigravity-memory.md.

6. The Hidden Bottleneck of In-Band Memory (And What Comes Next)

Having an agent autonomously manage its own Markdown memory files mid-session is a massive leap forward over stateless chat. However, as Lamis Mukta from Anthropic’s Applied AI team highlighted in her landmark talk “Learning While You Sleep: Beyond Memory to Dreaming”, relying exclusively on in-band memory writes (where the agent updates memory while simultaneously trying to solve your task) eventually runs into three structural walls:

  1. Attention Splitting & Missed Writes: In the middle of a 3,900-step marathon debugging session, the active agent is focused on solving the immediate user prompt. Important side-discoveries (such as SMB share paths or SSO port numbers) often remain buried in raw transcript.jsonl files without ever being written to Meta/antigravity-memory.md.
  2. The “Junk Drawer” Effect (Proactive Interference): Append-only in-band logs accumulate near-duplicate notes, outdated status lines, and superseded configurations over weeks of operation.
  3. Concurrency Collisions: If multiple sessions (e.g., an interactive terminal session, a Signal Bot daemon, and a Meshtastic LoRa gateway) write to Meta/antigravity-memory.md at the same time, naive file writes can clobber each other.

How do biological brains solve this? They don’t consolidate long-term semantic memory while running away from a predator—they do it out-of-band while sleeping.

👉 Continue to Part 4: Learning While You Sleep: Implementing Anthropic’s Out-of-Band Memory “Dreaming” for Local AI Agents in Obsidian

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.