Building a Complete AI Development Ecosystem for Claude Code: Persistent Memory + Spec-Driven Development

The Problem

Claude Code is incredibly powerful for individual tasks, but two fundamental limitations emerge in long-running, complex projects:

1. No persistent memory — Every session starts from zero. Debugging insights, architecture decisions, and learned patterns are lost between conversations.
2. No structured development workflow — Claude Code can write great code, but complex features need a structured pipeline: requirements → design → implementation → review → verification. Without this, quality depends entirely on prompt engineering.

I built solutions for both.

---

What I Built: Three Interconnected Systems

┌─────────────────────────────────────────────────────────────────┐
│  1. RAG Memory Server (MCP)                                    │
│     Persistent semantic memory across all sessions              │
├─────────────────────────────────────────────────────────────────┤
│  2. Bulk Knowledge Pipeline                                    │
│     Ingest debugging knowledge from any GitHub repository       │
├─────────────────────────────────────────────────────────────────┤
│  3. Spec-Driven Development (SDD) Workflow                     │
│     10-phase orchestrated pipeline with quality gates           │
└─────────────────────────────────────────────────────────────────┘
         ↕ All three systems feed each other ↕

---

1. Self-Hosted RAG Memory Server

An MCP server backed by Qdrant + Ollama running on a VPS. Five tools: mem_save, mem_search, mem_delete, mem_context, mem_stats.

Key design decisions:

• Hybrid search: Dense vectors (nomic-embed-text, 768D) + sparse vectors (BM25-like) with Reciprocal Rank Fusion. Neither pure semantic nor pure keyword — both combined
• Auto-chunking: 1500-char chunks with 200-char overlap for long memories
• Project namespacing: Memories isolated per project (mem_search(query, project="my-app"))
• Summary mode: mem_context(project, mode="summary") returns 150-char previews instead of full content — dramatically reduces token usage for context loading

Performance hardening (learned the hard way):

| Problem | Root Cause | Solution |
|---------|-----------|----------|
| Ollama OOM on bulk operations | No concurrency control | Semaphore (max 1 concurrent request) + micro-batching (max 8 texts/request) |
| 3-minute query latency | Embedding model cold starts | Pre-warm on server boot + LRU embedding cache (128 entries) |
| 92% of search time was embedding | Every query re-embeds | Cache hit → 7ms vs 87ms (10x improvement) |

Benchmarks (AMD EPYC 16-core VPS):

Warm embed (single):    87ms
Full search (embed+RRF): 79ms  →  7ms on cache hit
Qdrant search alone:     6ms
Save (1 chunk):          88ms

---

2. Bulk Knowledge Ingestion Pipeline

This is where it gets interesting. Instead of only saving my own decisions manually, I built a pipeline that extracts debugging knowledge from any GitHub repository and makes it searchable.

Pipeline

GitHub Search API → Targeted extraction (by label, keyword, topic)
        ↓
Qwen 2.5 1.5B (local, via Ollama) → Summarize each into ~400 char debugging notes
        ↓
nomic-embed-text → Embed summaries
        ↓
Qdrant → Indexed and searchable via MCP

Real Example

I targeted a large open-source AI assistant repository (~41K issues, 289K stars) with these search filters:

• Memory system labels (extensions: memory-core, extensions: memory-lancedb)
• MCP server issues
• Crash bugs
• Ollama-related issues
• Performance/timeout problems
• Embedding/vector issues

Result: 2,237 unique issues extracted, deduplicated across overlapping searches, each summarized with a debugging-focused prompt:

"Problem: what went wrong? Root cause: why? Fix: how was it resolved? Component: what was affected?"

Why This Matters

When Claude Code encounters a similar problem in my project, it searches this knowledge base:

mem_search("Ollama crash under bulk embedding load", project="external-repo")

And gets back real solutions from a similar project — instead of reasoning from scratch.

It's the difference between a junior dev thinking through every problem from first principles, and a senior dev saying "I've seen this before, here's what works."

Ingestion Safeguards

The pipeline is designed to never crash Ollama (the original problem that motivated all of this):

• Three independent phases: extract → summarize → ingest (each resumable via JSONL checkpoints)
• Configurable delays between operations
• Qwen model explicitly unloaded before embedding phase starts
• Embedding model re-warmed after Qwen memory pressure
• Progress tracking files for crash recovery

---

3. Spec-Driven Development (SDD) Workflow

The third piece is a 10-phase multi-agent orchestration framework that turns feature development into a structured, auditable pipeline. Every feature, bugfix, and refactor flows through:

init → explore → propose → spec + design (parallel) → tasks → apply → review → verify → clean → archive

How It Works

Claude Code acts as an orchestrator — it never writes code directly. Instead, it:

1. Validates preconditions via a contract system (PARCER) before each phase
2. Launches sub-agents that read phase-specific skill files (~/.claude/skills/sdd/sdd-{phase}/SKILL.md)
3. Collects standardized envelopes (A2A schema) with metrics, build health, and phase-specific data
4. Tracks quality deltas via an append-only quality-timeline.jsonl — an immutable audit trail of every phase
5. Handles failures autonomously via auto-negotiation loops before escalating to the developer

Key Innovations

Semantic Code Review — The review phase doesn't use generic linting rules. It dynamically generates a review rubric from:

• Actual spec scenarios (GIVEN/WHEN/THEN)
• Design decisions from the design doc
• Project conventions from CLAUDE.md and AGENTS.md

The same console.log gets flagged as CRITICAL in a payment handler (spec says "no logging of secrets") but only SUGGESTION in a utility file.

Auto-Negotiation Loop — When review or verify gates fail:

1. Classifies each issue as AUTO_FIXABLE or HUMAN_REQUIRED
2. If all are auto-fixable → dispatches apply agent in fix mode → re-runs the gate
3. Max 2 automatic fix iterations before escalating to the developer
4. Prevents infinite loops via Early Termination triggers

Speculative Parallelization — Spec and design phases run simultaneously (both depend only on the proposal), cutting planning time.

PARCER Contracts — Each phase declares preconditions and postconditions in its skill file. During sdd-init, these are auto-assembled into a live manifest. The orchestrator validates preconditions before launching any phase — catches "forgot to run X first" errors automatically.

Real Output: Archived Change

Here's a real archived change (mem-context-summary-mode) from this project:

openspec/changes/archive/2026-03-09-mem-context-summary-mode/
├── exploration.md          ← Codebase analysis, risk assessment
├── proposal.md             ← Intent, scope, approach, rollback plan
├── specs/                  ← 5 requirements, 15 GIVEN/WHEN/THEN scenarios
├── design.md               ← Architecture decisions, interfaces, data flow
├── tasks.md                ← 11 tasks across 5 phases, all checked off
├── apply-report.md         ← Files created/modified, build results
├── review-report.md        ← Semantic rubric scores, verdict: PASS
├── verify-report.md        ← Typecheck PASS, tests PASS, 100% completeness
├── clean-report.md         ← Dead code scan (none found)
├── archive-manifest.md     ← Change summary, key decisions
└── quality-timeline.jsonl  ← 10 snapshots, one per phase (audit trail)

Every decision is recorded. Every quality metric is tracked. Nothing is lost.

---

How All Three Systems Integrate

SDD Workflow                         Memory Server
────────────                         ─────────────
explore phase ──── mem_search ──────▶ "Have we solved this before?"
                                     Returns prior decisions & patterns
        ↓
propose → spec → design → tasks
        ↓
apply → review → verify → clean
        ↓
archive phase ──── mem_save ────────▶ Key decisions & learnings persisted
                                     Available in ALL future sessions

Bulk Ingestion Pipeline
───────────────────────
External repos ──── extract → summarize → ingest ──────▶ Memory Server
                                                         "How did others
                                                          solve this?"

The cycle: SDD produces learnings → Memory stores them → Future SDD phases retrieve them → Better decisions → Better learnings. Each project gets smarter over time.

External repo knowledge adds a second dimension: not just "what did I learn" but "what did the ecosystem learn."

---

Stack

• Runtime: Bun
• MCP SDK: @modelcontextprotocol/sdk
• Vector DB: Qdrant (hybrid dense + sparse, RRF fusion)
• Embeddings: Ollama + nomic-embed-text (768D)
• Summarization: Ollama + Qwen 2.5 1.5B (for bulk ingestion)
• Deployment: Docker Compose on a self-managed VPS (30GB RAM, 16 cores)
• SDD Skills: 12 phase-specific skill files, auto-discovered via PARCER contracts

---

Why This Matters for Claude Code

1. Memory is the missing layer. Forgetting everything between sessions is the biggest limitation for complex projects. The RAG approach with hybrid search, project namespacing, and summary mode is a production-tested pattern.

1. Learning from the ecosystem is unexplored territory. No coding assistant currently ingests resolved issues from relevant open-source projects to inform its work. The pattern of "search 2,000 resolved issues from a similar project before debugging" is powerful and generalizable.

1. Structured workflows produce better code. SDD's 10-phase pipeline with semantic review, auto-negotiation, and quality tracking shows that Claude Code can reliably handle complex features when given structure — not just write one-off functions.

1. Self-hosted is the right model. Developers want control over their knowledge base. A VPS-local solution avoids the privacy concerns of cloud-hosted memory.

1. The performance patterns are transferable. The semaphore, micro-batching, embedding cache, and resumable pipeline patterns would apply directly to any native implementation.

---

Would love to hear if persistent memory, structured development workflows, or external knowledge ingestion are on the Claude Code roadmap. Happy to discuss architectural decisions or share more detailed implementation notes.