---
type: service-design
status: active
updated: 2026-05-21
tags:
  - ai-workspace
  - rag
  - index
---

# Local RAG Index

## Goal

Add retrieval over canonical workspace memory without replacing the human-readable `project-knowledge/` vault.

The local index is derived and disposable. If the index disagrees with Markdown, the Markdown wins.

---

## Current Implementation

The first implementation is dependency-free and lexical:

```text
scripts/aiw/indexer.py
```

It reads:

```text
project-knowledge/**/*.md
```

and writes:

```text
.aiw/indexes/<profile>/project-knowledge.jsonl
.aiw/indexes/<profile>/manifest.json
```

It skips:

```text
project-knowledge/09-templates/
```

so Obsidian templates do not appear as real memory.

---

## Commands

Build the index:

```bash
python3 scripts/aiw/indexer.py build --profile fidelity
```

Check index status:

```bash
python3 scripts/aiw/indexer.py status --profile fidelity
```

Search the index:

```bash
python3 scripts/aiw/indexer.py search "dismissal lifecycle" --profile fidelity
```

---

## MCP Exposure

`aiw-context-mcp` exposes:

```text
memory_hybrid_search
```

Current behavior:

- searches the derived local index when it exists
- returns cited paths, headings, snippets, scores, hashes, and mtimes
- falls back to live Markdown search when no index exists
- remains read-only

---

## Future Upgrade Path

This layer can later add:

- full-text ranking
- embeddings
- Qdrant or Chroma as a local vector store
- hybrid lexical + semantic search
- reranking
- Mattermost evidence indexing with strict source filters

Do not make the vector store canonical. It should remain rebuildable from Markdown and selected evidence.