open-context open-context
October 15, 2026 · 6 min read

How open-context Tests 15 Database Backends With One Suite

open-context's MCP context store runs on 15 different databases, and every one of them is checked against the exact same file: tests/store/conformance.ts, roughly 50 tests that define what "correct" means for a context store. A backend either passes all of them or it isn't shipped as supported — the tests are the contract, not the documentation.

Key takeaways

  • One conformance suite of ~50 tests is the single definition of correct behavior for every open-context context store backend, covering contexts, bubbles, search, and lifecycle.
  • npm test runs it for free against json, sqlite, and memory — no external services needed, since those only need a temp file or RAM.
  • The other 12 backends run through npm run test:backends once their connection string is set, usually against the services defined in docker-compose.test.yml.
  • The suite encodes concrete invariants: createdAt-then-id ordering, case-insensitive substring search, undefined (never null) for unset optional fields, and read-your-writes consistency.
  • If a backend can't satisfy a test, the backend is documented as limited — the suite itself is never weakened to make it pass.

Why one interface across 15 backends needs one contract

open-context's MCP server lets Claude save and recall context through six tools — save_context, recall_context, list_contexts, search_contexts, update_context, delete_context — and none of those tools know or care which database is underneath. That only works if every backend genuinely behaves the same way. A context store that silently drops a tag, returns null where another returns undefined, or sorts results in a different order isn't a drop-in replacement, it's a bug waiting to surface the first time someone switches backends. The conformance suite exists to catch exactly that class of bug before a user does.

What the suite actually checks

tests/store/conformance.ts exports runStoreConformance(), a function that takes a harness — setup, create, teardown — and runs the full behavioral contract against whatever adapter the harness hands back. It's organized into 13 groups that mirror the store's own methods:

Individual tests get specific: round-tripping content with embedded quotes and a literal DROP TABLE string (so every SQL driver has to prove it's actually parameterizing, not concatenating), round-tripping Unicode, and reopening a fresh adapter instance mid-test to prove a write really persisted rather than living only in an in-memory cache.

How it runs against three different adapter families

open-context doesn't implement all 15 backends separately. As the project's own BYODB architecture overview explains, CRUD logic is written once per family — a shared adapters/sql.ts for sqlite, libsql, d1, duckdb, postgres, cloudsql, mysql, and mssql, each contributing only a small Dialect (placeholder style, DDL, string concat); a shared adapters/document.ts for memory, mongodb, redis, firestore, and dynamodb, each contributing a DocumentDriver; and a bespoke adapters/surreal.ts for surrealdb. The conformance suite runs against the family implementation, not against each individual database — which means passing it for sqlite already proves most of what postgres and mysql need, since they share the same code path and differ only in their Dialect.

That's also why the suite runs in two tiers. npm test exercises json (a flat file), sqlite (a temp file), and memory (plain RAM) on every run, including CI, because none of them need a service running. The remaining 12 need a real database, so tests/store/backends.test.ts skips each one unless its connection string is present in the environment:

docker compose -f docker-compose.test.yml up -d
OPENCONTEXT_TEST_POSTGRES_URL="postgres://opencontext:[email protected]:55432/opencontext" \
  npm run test:backends
docker compose -f docker-compose.test.yml down -v

docker-compose.test.yml spins up Postgres, MySQL, and SQL Server (among others) with health checks, so the same 50 tests run against a live engine instead of a mock.

The invariants that actually matter

A handful of rules in the suite aren't obvious from the interface alone, which is exactly why they're enforced in code instead of left to each driver's judgment:

None of these are edge cases a casual integration test would catch. They surface specifically because the same 50 assertions run against every backend, so a driver that technically "works" but diverges on one of these points fails the exact same way sqlite would if it made the same mistake.

When a backend can't pass

The project's own guidance is explicit about which direction the fix goes: the conformance suite "must never be weakened to make a backend pass — if a backend cannot satisfy it, that is a finding to document, not a test to relax." That's a meaningful constraint to hold, because the easy failure mode for any shared-contract test suite is death by a thousand exceptions — one skip() for Redis here, one loosened assertion for Firestore there — until the suite no longer actually guarantees interchangeability for anyone. Treating a failed test as a backend limitation instead keeps the contract meaningful: if postgres passes all 50 tests and mysql passes all 50 tests, a user can actually trust that switching between them with db migrate won't change behavior underneath their saved contexts.

This pattern — one reusable suite, a thin per-implementation adapter, run against both fakes and real services — isn't unique to open-context. dbt Labs proposed the same shape for dbt-core's database adapters: bundle shared test cases into a package so adapter maintainers prove behavior matches without copy-pasting tests per database. It's the standard way to make "drop-in replacement" a verifiable claim instead of a marketing one.

Adding a backend without breaking the contract

Adding a 16th backend to open-context means wiring it into an existing family, not building a parser for the conformance suite itself. The short version: add the scheme to DbScheme and SUPPORTED_SCHEMES, parse any backend-specific DSN fields, write a Dialect or DocumentDriver, register it in ADAPTERS and the build() switch, load the client library with importOptional() so a missing package produces an install instruction instead of a crash, and add it as an optional peer dependency. Then the suite does the verification work: run it against the new backend until all 50 tests pass, and only then is it a backend open-context actually supports — not one that merely compiles.

Point open-context's MCP server at any of its 15 supported databases.

Get started with open-context →

FAQ

What is the open-context conformance suite?

It's a single file, tests/store/conformance.ts, containing about 50 tests that every one of open-context's 15 database backends must pass. It is the only definition of correct behavior for the context store — not a README, not a comment, the tests themselves.

How many database backends does open-context support?

Fifteen: json, sqlite, libsql, d1, duckdb, postgres, cloudsql, mysql, mssql, memory, mongodb, redis, firestore, dynamodb, and surrealdb. They share three adapter implementations — SQL, document/KV, and a bespoke SurrealDB adapter — rather than fifteen separate codebases.

Do I need to run every backend's database to test open-context's store?

npm test runs the full conformance suite against json, sqlite, and memory with zero external services, since those need only a temp file or RAM. The other 12 backends run via npm run test:backends once their connection strings are set, typically against the services in docker-compose.test.yml.

What happens if a backend can't pass all 50 conformance tests?

The test stays as written. If a backend genuinely cannot satisfy an invariant, that becomes a documented limitation of that backend, not a reason to loosen the suite — weakening the contract to fit one backend would let every other backend regress silently.

Does the conformance suite check credentials or connection strings?

Indirectly, through the adapter contract: every driver receives dsn.canonical, never dsn.raw, and everything user-visible about a connection string is expected to go through redactDsn() so passwords never leak into logs or error messages.