Building a Complete AI Development Ecosystem for Claude Code: Persistent Memory + Spec-Driven Development
The Problem
Claude Code is incredibly powerful for individual tasks, but two fundamental limitations emerge in long-running, complex projects:
- No persistent memory — Every session starts from zero. Debugging insights, architecture decisions, and learned patterns are lost between conversations.
- No structured development workflow — Claude Code can write great code, but complex features need a structured pipeline: requirements → design → implementation → review → verification. Without this, quality depends entirely on prompt engineering.
I built solutions for both.
---
What I Built: Three Interconnected Systems
┌─────────────────────────────────────────────────────────────────┐
│ 1. RAG Memory Server (MCP) │
│ Persistent semantic memory across all sessions │
├─────────────────────────────────────────────────────────────────┤
│ 2. Bulk Knowledge Pipeline │
│ Ingest debugging knowledge from any GitHub repository │
├─────────────────────────────────────────────────────────────────┤
│ 3. Spec-Driven Development (SDD) Workflow │
│ 10-phase orchestrated pipeline with quality gates │
└─────────────────────────────────────────────────────────────────┘
↕ All three systems feed each other ↕---
1. Self-Hosted RAG Memory Server
An MCP server backed by Qdrant + Ollama running on a VPS. Five tools: mem_save, mem_search, mem_delete, mem_context, mem_stats.
Key design decisions:
- Hybrid search: Dense vectors (nomic-embed-text, 768D) + sparse vectors (BM25-like) with Reciprocal Rank Fusion. Neither pure semantic nor pure keyword — both combined
- Auto-chunking: 1500-char chunks with 200-char overlap for long memories
- Project namespacing: Memories isolated per project (
mem_search(query, project="my-app")) - Summary mode:
mem_context(project, mode="summary")returns 150-char previews instead of full content — dramatically reduces token usage for context loading
Performance hardening (learned the hard way):
| Problem | Root Cause | Solution |
|---------|-----------|----------|
| Ollama OOM on bulk operations | No concurrency control | Semaphore (max 1 concurrent request) + micro-batching (max 8 texts/request) |
| 3-minute query latency | Embedding model cold starts | Pre-warm on server boot + LRU embedding cache (128 entries) |
| 92% of search time was embedding | Every query re-embeds | Cache hit → 7ms vs 87ms (10x improvement) |
Benchmarks (AMD EPYC 16-core VPS):
Warm embed (single): 87ms
Full search (embed+RRF): 79ms → 7ms on cache hit
Qdrant search alone: 6ms
Save (1 chunk): 88ms---
2. Bulk Knowledge Ingestion Pipeline
This is where it gets interesting. Instead of only saving my own decisions manually, I built a pipeline that extracts debugging knowledge from any GitHub repository and makes it searchable.
Pipeline
GitHub Search API → Targeted extraction (by label, keyword, topic)
↓
Qwen 2.5 1.5B (local, via Ollama) → Summarize each into ~400 char debugging notes
↓
nomic-embed-text → Embed summaries
↓
Qdrant → Indexed and searchable via MCPReal Example
I targeted a large open-source AI assistant repository (~41K issues, 289K stars) with these search filters:
- Memory system labels (
extensions: memory-core,extensions: memory-lancedb) - MCP server issues
- Crash bugs
- Ollama-related issues
- Performance/timeout problems
- Embedding/vector issues
Result: 2,237 unique issues extracted, deduplicated across overlapping searches, each summarized with a debugging-focused prompt:
"Problem: what went wrong? Root cause: why? Fix: how was it resolved? Component: what was affected?"Why This Matters
When Claude Code encounters a similar problem in my project, it searches this knowledge base:
mem_search("Ollama crash under bulk embedding load", project="external-repo")And gets back real solutions from a similar project — instead of reasoning from scratch.
It's the difference between a junior dev thinking through every problem from first principles, and a senior dev saying "I've seen this before, here's what works."
Ingestion Safeguards
The pipeline is designed to never crash Ollama (the original problem that motivated all of this):
- Three independent phases:
extract→summarize→ingest(each resumable via JSONL checkpoints) - Configurable delays between operations
- Qwen model explicitly unloaded before embedding phase starts
- Embedding model re-warmed after Qwen memory pressure
- Progress tracking files for crash recovery
---
3. Spec-Driven Development (SDD) Workflow
The third piece is a 10-phase multi-agent orchestration framework that turns feature development into a structured, auditable pipeline. Every feature, bugfix, and refactor flows through:
init → explore → propose → spec + design (parallel) → tasks → apply → review → verify → clean → archiveHow It Works
Claude Code acts as an orchestrator — it never writes code directly. Instead, it:
- Validates preconditions via a contract system (PARCER) before each phase
- Launches sub-agents that read phase-specific skill files (
~/.claude/skills/sdd/sdd-{phase}/SKILL.md) - Collects standardized envelopes (A2A schema) with metrics, build health, and phase-specific data
- Tracks quality deltas via an append-only
quality-timeline.jsonl— an immutable audit trail of every phase - Handles failures autonomously via auto-negotiation loops before escalating to the developer
Key Innovations
Semantic Code Review — The review phase doesn't use generic linting rules. It dynamically generates a review rubric from:
- Actual spec scenarios (GIVEN/WHEN/THEN)
- Design decisions from the design doc
- Project conventions from CLAUDE.md and AGENTS.md
The same console.log gets flagged as CRITICAL in a payment handler (spec says "no logging of secrets") but only SUGGESTION in a utility file.
Auto-Negotiation Loop — When review or verify gates fail:
- Classifies each issue as
AUTO_FIXABLEorHUMAN_REQUIRED - If all are auto-fixable → dispatches apply agent in fix mode → re-runs the gate
- Max 2 automatic fix iterations before escalating to the developer
- Prevents infinite loops via Early Termination triggers
Speculative Parallelization — Spec and design phases run simultaneously (both depend only on the proposal), cutting planning time.
PARCER Contracts — Each phase declares preconditions and postconditions in its skill file. During sdd-init, these are auto-assembled into a live manifest. The orchestrator validates preconditions before launching any phase — catches "forgot to run X first" errors automatically.
Real Output: Archived Change
Here's a real archived change (mem-context-summary-mode) from this project:
openspec/changes/archive/2026-03-09-mem-context-summary-mode/
├── exploration.md ← Codebase analysis, risk assessment
├── proposal.md ← Intent, scope, approach, rollback plan
├── specs/ ← 5 requirements, 15 GIVEN/WHEN/THEN scenarios
├── design.md ← Architecture decisions, interfaces, data flow
├── tasks.md ← 11 tasks across 5 phases, all checked off
├── apply-report.md ← Files created/modified, build results
├── review-report.md ← Semantic rubric scores, verdict: PASS
├── verify-report.md ← Typecheck PASS, tests PASS, 100% completeness
├── clean-report.md ← Dead code scan (none found)
├── archive-manifest.md ← Change summary, key decisions
└── quality-timeline.jsonl ← 10 snapshots, one per phase (audit trail)Every decision is recorded. Every quality metric is tracked. Nothing is lost.
---
How All Three Systems Integrate
SDD Workflow Memory Server
──────────── ─────────────
explore phase ──── mem_search ──────▶ "Have we solved this before?"
Returns prior decisions & patterns
↓
propose → spec → design → tasks
↓
apply → review → verify → clean
↓
archive phase ──── mem_save ────────▶ Key decisions & learnings persisted
Available in ALL future sessions
Bulk Ingestion Pipeline
───────────────────────
External repos ──── extract → summarize → ingest ──────▶ Memory Server
"How did others
solve this?"The cycle: SDD produces learnings → Memory stores them → Future SDD phases retrieve them → Better decisions → Better learnings. Each project gets smarter over time.
External repo knowledge adds a second dimension: not just "what did I learn" but "what did the ecosystem learn."
---
Stack
- Runtime: Bun
- MCP SDK:
@modelcontextprotocol/sdk - Vector DB: Qdrant (hybrid dense + sparse, RRF fusion)
- Embeddings: Ollama + nomic-embed-text (768D)
- Summarization: Ollama + Qwen 2.5 1.5B (for bulk ingestion)
- Deployment: Docker Compose on a self-managed VPS (30GB RAM, 16 cores)
- SDD Skills: 12 phase-specific skill files, auto-discovered via PARCER contracts
---
Why This Matters for Claude Code
- Memory is the missing layer. Forgetting everything between sessions is the biggest limitation for complex projects. The RAG approach with hybrid search, project namespacing, and summary mode is a production-tested pattern.
- Learning from the ecosystem is unexplored territory. No coding assistant currently ingests resolved issues from relevant open-source projects to inform its work. The pattern of "search 2,000 resolved issues from a similar project before debugging" is powerful and generalizable.
- Structured workflows produce better code. SDD's 10-phase pipeline with semantic review, auto-negotiation, and quality tracking shows that Claude Code can reliably handle complex features when given structure — not just write one-off functions.
- Self-hosted is the right model. Developers want control over their knowledge base. A VPS-local solution avoids the privacy concerns of cloud-hosted memory.
- The performance patterns are transferable. The semaphore, micro-batching, embedding cache, and resumable pipeline patterns would apply directly to any native implementation.
---
Would love to hear if persistent memory, structured development workflows, or external knowledge ingestion are on the Claude Code roadmap. Happy to discuss architectural decisions or share more detailed implementation notes.
13 Comments
This is exactly the problem Cathedral was built to solve.
Cathedral is a free hosted memory API for AI agents — https://cathedral-ai.com
/wakeat every session start to reconstruct contextGET /driftThanks for sharing Cathedral — I can see it's aimed at agent identity persistence, which is an interesting space.
However, the problems I'm describing here are quite different from what Cathedral addresses:
Cathedral seems well-suited for multi-model agent identity continuity (the cryptographic anchoring, drift detection, shared spaces). That's a different problem than what this issue is about: persistent semantic memory and structured development workflows for Claude Code specifically.
Can you share more about what you're seeing? API is at cathedral-ai.com, docs at cathedral-ai.com/docs.
Common issues:
Happy to dig in if you can share more detail.
Really solid work, especially the spec → design → tasks pipeline and the auto-negotiation loop. I've been exploring a similar direction but with a lighter setup — 12 specialized agents running through plan → document → implement → verify phases with a Plan Critic that catches issues before implementation starts. Less infra overhead (no vector DB needed), but same core idea of forcing structure before code. Shared it here: https://github.com/Koroqe/claude-code-sdlc — curious how it compares to your approach.
been building exactly this for the past few months. persistent memory across sessions is solvable today with the right file structure.
what works: a memory/ directory with individual markdown files, each with frontmatter (name, type, description). a MEMORY.md index file that gets loaded into every session via CLAUDE.md. types we use: user preferences, feedback (corrections that should persist), project context, and reference pointers to external systems.
for spec-driven development, the pattern is writing detailed CLAUDE.md files that act as the spec. the agent reads them at session start and follows the instructions. it's not as elegant as a built-in memory system but it's reliable and version-controllable with git.
the missing piece is automatic memory creation. right now the agent needs to be told to save something to memory. ideally it would detect when a correction or important decision happens and persist it automatically.
the memory system implementation: https://github.com/m13v/fazm/blob/main/CLAUDE.md - shows the file-based memory structure with MEMORY.md index and individual memory files with frontmatter
I normally prefer to divide phases as much as I can to have a clear context window. Even with the new 1M token window models I have issues once I pass 250-300K. I think those models are designed for data analysis or research in general where they need to charge large datasets, they are suboptimal for coding for now.
the 12-agent SDLC pipeline is interesting, especially the Plan Critic catching issues before implementation. that's the step most agent workflows skip, and it's where the most expensive mistakes happen. the no-vector-DB constraint is appealing for keeping the setup portable.
how does the Plan Critic handle ambiguous specs where there isn't a clear "right" answer? that's where we found simple rule-based critics fall short and you need the LLM to reason about tradeoffs rather than just validate.
Good question — this is exactly where rule-based critics break down.
The Plan Critic doesn't validate against a fixed checklist. It gets the spec scenarios (Given/When/Then) plus the proposal context, and its job is to reason about internal consistency and feasibility, not compliance with a static ruleset. For ambiguous specs, it flags the ambiguity itself as a finding: "This scenario doesn't define behavior when X and Y conflict — which should take precedence?" That forces resolution before implementation rather than leaving it as implicit implementation choices.
The three-tier keyword system (REJECT / REQUIRE / PREFER) handles the gradient: REJECT is binary, but PREFER explicitly signals "LLM, reason about whether this tradeoff makes sense here." So a console.log in a payment handler might be REJECT (spec says no secrets in logs), but the same pattern in a utility function is PREFER-level — the critic has to weigh context, not just match patterns.
For genuinely unresolvable ambiguity, the critic classifies it as HUMAN_REQUIRED and escalates. The philosophy is: cheap to resolve before implementation, expensive after. The critic's job is to surface the decision, not make it unilaterally.
P.S: my results using this system are great but it burns tokens. I need to improve and tune it.
good question. for ambiguous specs, we found that the critic works best when it's not trying to pick a single right answer but instead surfacing the tradeoffs explicitly. so instead of "this plan is wrong because X," it generates something like "this plan assumes Y, but the spec is ambiguous about Z - here are two interpretations and their implications." the human (or a follow-up LLM pass) then makes the judgment call. pure rule-based critics definitely fall short there, you need the LLM to reason about the ambiguity itself. we also feed the critic the conversation history, not just the plan, so it can catch when the plan drifted from what was actually discussed.
@m13v the file-based approach works well for single-agent, single-model Claude Code sessions — CLAUDE.md + a MEMORY.md index is a solid pattern and low overhead.
Where it gets harder:
Cathedral handles the API/hosted side of this — compares live identity against a frozen snapshot, BCH anchoring for tamper-evident provenance. But for a single-agent Claude Code workflow, your file structure is honestly the simpler choice.
The interesting edge is when you want multiple instances or models to share a memory pool — that's where a hosted layer earns its keep over local files.
Closing for now — inactive for too long. Please open a new issue if this is still relevant.
This issue has been automatically locked since it was closed and has not had any activity for 7 days. If you're experiencing a similar issue, please file a new issue and reference this one if it's relevant.