feat: implement local indexer for project-knowledge and add memory hybrid search functionality

core/services/local-rag-index.md

@@ -0,0 +1,102 @@
+---
+type: service-design
+status: active
+updated: 2026-05-21
+tags:
+  - ai-workspace
+  - rag
+  - index
+---
+
+# Local RAG Index
+
+## Goal
+
+Add retrieval over canonical workspace memory without replacing the human-readable `project-knowledge/` vault.
+
+The local index is derived and disposable. If the index disagrees with Markdown, the Markdown wins.
+
+---
+
+## Current Implementation
+
+The first implementation is dependency-free and lexical:
+
+```text
+scripts/aiw/indexer.py
+```
+
+It reads:
+
+```text
+project-knowledge/**/*.md
+```
+
+and writes:
+
+```text
+.aiw/indexes/<profile>/project-knowledge.jsonl
+.aiw/indexes/<profile>/manifest.json
+```
+
+It skips:
+
+```text
+project-knowledge/09-templates/
+```
+
+so Obsidian templates do not appear as real memory.
+
+---
+
+## Commands
+
+Build the index:
+
+```bash
+python3 scripts/aiw/indexer.py build --profile fidelity
+```
+
+Check index status:
+
+```bash
+python3 scripts/aiw/indexer.py status --profile fidelity
+```
+
+Search the index:
+
+```bash
+python3 scripts/aiw/indexer.py search "dismissal lifecycle" --profile fidelity
+```
+
+---
+
+## MCP Exposure
+
+`aiw-context-mcp` exposes:
+
+```text
+memory_hybrid_search
+```
+
+Current behavior:
+
+- searches the derived local index when it exists
+- returns cited paths, headings, snippets, scores, hashes, and mtimes
+- falls back to live Markdown search when no index exists
+- remains read-only
+
+---
+
+## Future Upgrade Path
+
+This layer can later add:
+
+- full-text ranking
+- embeddings
+- Qdrant or Chroma as a local vector store
+- hybrid lexical + semantic search
+- reranking
+- Mattermost evidence indexing with strict source filters
+
+Do not make the vector store canonical. It should remain rebuildable from Markdown and selected evidence.