Architectural evaluation criteria, capabilities matrix (XMemo vs Mem0 vs Zep), scenario-based selection, reproducible benchmarks, and migration guide.
Architectural evaluation dimensions
Evaluating agent memory systems requires looking beyond surface vector retrieval to core production architecture: data model, tenancy boundaries, consistency guarantees, protocol compliance, latency, and machine readability.
- Storage model: Hybrid relational metadata with vector embeddings versus vector-only flat stores. Relational grounding ensures exact filter pushdown, immutable audit logs, and deterministic scope partitioning.
- Multi-tenant isolation: Hardware- or server-enforced tenant partitions versus application-level metadata filtering. Shared agent spaces require cryptographically signed space bindings.
- Consistency guarantees: Read-your-writes consistency across distributed tool calls versus eventual consistency windows that cause hallucinations in continuous agent loops.
- Protocol compliance: Native Model Context Protocol (MCP) Streamable HTTP endpoints versus proprietary wrappers requiring per-framework SDK adapters.
- Latency boundaries: Sub-50ms p95 recall overhead to prevent blocking interactive LLM generation loops.
- Machine readability: Structured, deterministic JSON responses adhering strictly to tool schemas without prose tail or conversational wrapper text.
Capabilities comparison matrix
Direct capability comparison across XMemo, Mem0, and Zep based on production architectural evidence and documented system capabilities as of September 2026. Third-party capabilities not independently verified are explicitly designated.
- Scopes: XMemo provides 4-tier orthogonal scopes (user, session, agent, team/space); Mem0 provides user and session scopes; Zep provides user and thread/session scopes.
- Projects & Namespaces: XMemo provides first-class project fact isolation with cross-client sharing; Mem0 relies on custom metadata tagging; Zep organizes by user/thread sessions.
- MCP Native Tools: XMemo provides 12 frozen standard MCP tools with streamable HTTP; Mem0 offers community MCP wrappers; Zep offers community MCP wrappers.
- Conflict Resolution: XMemo provides versioned update trees, optimistic locking, and conflict signals; Mem0 overwrites or appends; Zep reconciles temporal graph edges.
- Episodic-to-Semantic Reflection: XMemo features scheduled background Dream reflection with decay policies; Mem0 extracts facts inline on write; Zep performs graph extraction asynchronously.
- Deployment Options: XMemo supports managed SaaS and self-hosted Docker/VPS; Mem0 supports cloud and open-source pip library; Zep supports cloud and self-hosted Community Edition.
- Note on third-party metrics: Competitor benchmarks and internal performance figures from third-party marketing materials are marked NOT VERIFIED and excluded from our baseline assertions.
{
"comparison_matrix": {
"dimensions": [
{
"feature": "Scope hierarchy",
"xmemo": "User, session, agent, team space (4 tiers)",
"mem0": "User, session, custom metadata",
"zep": "User, thread/session"
},
{
"feature": "MCP Protocol support",
"xmemo": "Native Streamable HTTP (12 tools)",
"mem0": "Adapter / community wrapper",
"zep": "Adapter / community wrapper"
},
{
"feature": "Audit log & tombstones",
"xmemo": "Immutable append-only audit log + tombstones",
"mem0": "Metadata update without full audit log",
"zep": "Temporal graph history"
},
{
"feature": "Conflict management",
"xmemo": "Versioned revisions with CAS resolution",
"mem0": "Last-write-wins overwrite",
"zep": "Edge update in temporal graph"
}
]
}
}
Scenario-based selection guide
Select the memory system that matches your workload topology, team structure, and governance requirements.
- Choose XMemo when: You orchestrate autonomous multi-agent teams across multiple environments (Claude Code, Cursor, ChatGPT, Codex, Gemini CLI) requiring shared project context, auditable memory retention, strict tenant isolation, and native MCP protocol integration.
- Choose Mem0 when: You are building a consumer conversational chatbot that needs lightweight per-user personalization in Python without complex team workspaces or multi-IDE synchronization.
- Choose Zep when: Your application requires temporal reasoning over multi-turn conversational dialogue histories using temporal knowledge graph structures.
Reproducible benchmark methodology
XMemo provides an open, repository-owned benchmark harness to evaluate retrieval accuracy and governance constraints under controlled conditions. Benchmarks distinguish strictly between internal synthetic evaluation, production latency, and verified public claims.
- Evaluation Methodology: Benchmarks follow frozen method contract docs/reports/T06_REPRODUCIBLE_BENCHMARK_METHOD_CONTRACT_2026-09-16.json with deterministic golden set datasets and recorded verification traces.
- Internal Synthetic Benchmarks: Run hermetically via scripts/benchmark_runner.py using tests/data/recall_golden_set.jsonl (SHA-256 7533FBDC...). These measure in-memory retrieval algorithm quality and ranking logic without network or database I/O.
- Real Production Latency: End-to-end latency includes TLS handshake, network transit, database transactions, and tenant isolation overhead. Production latency varies by hosting topology (cloud SaaS vs self-hosted Docker) and geographical distance, and is not represented by in-memory synthetic figures.
- Public Performance Claims: All reported metrics must cite an immutable benchmark_run_id and raw verifier report under docs/reports/t06_reproducibility/. Unverified vendor claims and synthetic shortcuts are excluded from baseline assertions.
- Failure Mode Analysis: Documented edge failure observed when ambiguous entity names ('config.py' across two distinct project scopes) competed for ranking without explicit project scope qualification; resolved by adding project scope constraints.
# Execute reproducible retrieval benchmark against golden set
python scripts/benchmark_runner.py \
--track standard \
--dataset tests/data/recall_golden_set.jsonl \
--cache-state COLD_CACHE \
--output docs/reports/t06_reproducibility/custom_run_raw.json
Migration to XMemo
Migrating from legacy key-value, vector-only, or external memory systems to XMemo follows a four-stage structured transition.
- Stage 1: Export legacy memory items as JSON Lines containing text, user_id, session_id, and created_at timestamps.
- Stage 2: Map identifiers to XMemo scope hierarchy — map user_id to owner, session_id to agent_instance_id or project_id, and categorize facts by durability.
- Stage 3: Ingest via the batch remember tool or REST API (/v1/memories) with appropriate scope and provenance attributes.
- Stage 4: Validate recall accuracy and isolation boundaries using recall_context before enabling agent write permissions.
import requests
def migrate_legacy_record(legacy_item, api_key):
payload = {
"content": legacy_item["content"],
"path": f"migration/{legacy_item.get('id', 'item')}",
"scope": "project" if legacy_item.get("project_id") else "user",
"metadata": {"source": "legacy_migration", "original_id": legacy_item.get("id")}
}
response = requests.post(
"https://xmemo.dev/v1/memories",
headers={"Authorization": f"Bearer {api_key}"},
json=payload
)
response.raise_for_status()
return response.json()
Known limitations and operational boundaries
To prevent misapplication, understand the operational boundaries and explicit design trade-offs of XMemo.
- Not a general relational database: XMemo is optimized for agent facts, state, and skills. It does not replace application primary databases for arbitrary SQL transactions.
- Not an unstructured file dump: Documents exceeding 32 KB should be indexed in Knowledge Bases rather than stored as single inline memory entries.
- Dream reflection frequency: Periodic consolidation runs in batches (default: hourly or upon idle trigger); immediate consolidation is not synchronous with writes.
- Rate limits: Hosted endpoints enforce standard tier token buckets (100 req/min for free pilots, 2,400 req/min for enterprise workspaces) to protect multi-tenant availability.
Persona flows
ChatGPT user
Give ChatGPT durable access to your XMemo preferences, project facts, decisions, and TODOs without pasting bearer tokens into a chat.
Save a synthetic preference or project note, start a new chat, then ask ChatGPT to recall it through XMemo before continuing work.
Copilot / Codex developer
Carry repo decisions, coding conventions, bug-fix notes, and task history between IDE and CLI agents.
Record a codebase decision or bug fix, then ask the next IDE or CLI agent to recall the relevant XMemo context before editing.
Team / enterprise pilot owner
Evaluate shared memory with account controls, source attribution, export/delete workflows, and reviewer-safe setup evidence.
Have a pilot member save a synthetic team memory, confirm source attribution in XMemo, then review delete/export and support paths.