Local RAG Index

The local RAG index is the first retrieval layer over canonical Markdown memory.

Purpose

It helps AI clients quickly find relevant snippets without loading the whole project knowledge vault into context.

project-knowledge/ Markdown
        ↓
scripts/aiw/indexer.py
        ↓
.aiw/indexes/<profile>/project-knowledge.jsonl
        ↓
MCP tool: memory_hybrid_search

Current Implementation

The current indexer is dependency-free and lexical. It is intentionally simple so it can run on a new machine without a vector database.

Build:

python3 scripts/aiw/indexer.py build --profile fidelity

Status:

python3 scripts/aiw/indexer.py status --profile fidelity

Search:

python3 scripts/aiw/indexer.py search "dismissal lifecycle" --profile fidelity

What It Stores

source path;
heading;
text chunk;
mtime;
content hash;
chunk id.

What It Does Not Do

It does not replace Markdown.
It does not write project facts.
It does not index templates as real notes.
It does not send data to a cloud service.

Future Options

Future phases may add:

better full-text ranking;
semantic embeddings;
Qdrant or Chroma as optional local vector stores;
hybrid lexical + semantic search;
index status in the menu bar app.

Keep this as a derived layer. The project knowledge vault remains canonical.

1.4 KiB Raw Blame History