Vectorizer

Vectorizer — Semantic Memory Server

A lightweight, self-hosted memory server for AI agents. Stores messages as embeddings in ChromaDB with optional LLM-powered summarization and Q&A. Each agent gets isolated memory via workspace namespaces.

Features

• Workspace isolation — ws_ collections, no cross-talk between agents
• Semantic + hybrid search — vector cosine (HNSW) + BM25 RRF, temporal filters, grep
• Peers + peer cards — agent identity management within workspaces
• Agentic dialectic chat — observer/observed modes, reasoning levels, SSE streaming, 5 built-in tools
• Reasoning graph + deriver — premise edges, BFS reasoning chains, async derivation
• Conclusions + surprisal dreamer — offline summarization with surprisal gating
• Optional LLM brain — summarization & RAG Q&A via /chat/completions
• Auth — API key or JWT with peer scoping
• Layered config — env > .env > config.toml > defaults
• Docker-ready — one docker compose up (ChromaDB + Qwen embeddings + Vectorizer)
• MCP + Skills + SDKs — 13 MCP tools, TypeScript & Python SDKs
• Evals — LongMemEval-style recall + reasoning-grounded benchmarks

Architecture

Agent → Vectorizer API → ChromaDB (vectors) + Embedding Service
                              │
                              ├─ Qwen3-Embedding-4B (1536d MRL)
                              ├─ nomic-embed-text (768d fallback)
                              └─ text-embedding-3-small (OpenAI)

Tech Stack

• Language: Go
• Storage: ChromaDB
• Embeddings: Qwen3-Embedding-4B (1536d), nomic-embed-text, OpenAI
• Communication: REST API, gRPC, MCP
• Deployment: Docker Compose
• LLM Integration: OpenAI-compatible endpoints (qwen3:8b, gpt-4o-mini, etc.)