How open-context Tests 15 Database Backends With One Suite
open-context's MCP context store runs on 15 different databases, and every one of them is
checked against the exact same file: tests/store/conformance.ts, roughly 50
tests that define what "correct" means for a context store. A backend either passes all of
them or it isn't shipped as supported — the tests are the contract, not the documentation.
Key takeaways
- One conformance suite of ~50 tests is the single definition of correct behavior for every open-context context store backend, covering contexts, bubbles, search, and lifecycle.
npm testruns it for free againstjson,sqlite, andmemory— no external services needed, since those only need a temp file or RAM.- The other 12 backends run through
npm run test:backendsonce their connection string is set, usually against the services defined indocker-compose.test.yml. - The suite encodes concrete invariants:
createdAt-then-idordering, case-insensitive substring search,undefined(nevernull) for unset optional fields, and read-your-writes consistency. - If a backend can't satisfy a test, the backend is documented as limited — the suite itself is never weakened to make it pass.
Why one interface across 15 backends needs one contract
open-context's MCP server lets Claude save and recall context through six tools —
save_context, recall_context, list_contexts,
search_contexts, update_context, delete_context — and
none of those tools know or care which database is underneath. That only works if every
backend genuinely behaves the same way. A context store that silently drops a tag, returns
null where another returns undefined, or sorts results in a
different order isn't a drop-in replacement, it's a bug waiting to surface the first time
someone switches backends. The conformance suite exists to catch exactly that class of bug
before a user does.
What the suite actually checks
tests/store/conformance.ts exports runStoreConformance(), a function
that takes a harness — setup, create, teardown — and
runs the full behavioral contract against whatever adapter the harness hands back. It's
organized into 13 groups that mirror the
store's own methods:
- Contexts —
saveContext,getContext,recallContext,searchContexts,listContexts,updateContext,deleteContext - Bubbles —
createBubble,listBubbles/getBubble,listContextsByBubble,updateBubble,deleteBubble - Lifecycle — connecting, closing, and reconnecting without losing data
Individual tests get specific: round-tripping content with embedded quotes and a literal
DROP TABLE string (so every SQL driver has to prove it's actually parameterizing,
not concatenating), round-tripping Unicode, and reopening a fresh adapter instance mid-test
to prove a write really persisted rather than living only in an in-memory cache.
How it runs against three different adapter families
open-context doesn't implement all 15 backends separately. As the project's own
BYODB architecture overview
explains, CRUD logic is written once per family — a shared adapters/sql.ts for
sqlite, libsql, d1, duckdb,
postgres, cloudsql, mysql, and mssql, each
contributing only a small Dialect (placeholder style, DDL, string concat);
a shared adapters/document.ts for memory, mongodb,
redis, firestore, and dynamodb, each contributing a
DocumentDriver; and a bespoke adapters/surreal.ts for
surrealdb. The conformance suite runs against the family implementation, not
against each individual database — which means passing it for sqlite already
proves most of what postgres and mysql need, since they share the
same code path and differ only in their Dialect.
That's also why the suite runs in two tiers. npm test exercises json
(a flat file), sqlite (a temp file), and memory (plain RAM) on every
run, including CI, because none of them need a service running. The remaining 12 need a real
database, so tests/store/backends.test.ts skips each one unless its connection
string is present in the environment:
docker compose -f docker-compose.test.yml up -d
OPENCONTEXT_TEST_POSTGRES_URL="postgres://opencontext:[email protected]:55432/opencontext" \
npm run test:backends
docker compose -f docker-compose.test.yml down -v
docker-compose.test.yml spins up Postgres, MySQL, and SQL Server (among others)
with health checks, so the same 50 tests run against a live engine instead of a mock.
The invariants that actually matter
A handful of rules in the suite aren't obvious from the interface alone, which is exactly why they're enforced in code instead of left to each driver's judgment:
- Ordering — every list method returns results
createdAt ASC, id ASC, identically across backends, including ones like DynamoDB that don't sort natively. - Search semantics —
recallContextandsearchContextsdo a case-insensitive substring match over content, tags, and source, andsearchContextsrequires every term to match, not just one. - Absent vs. null — an unset
bubbleIdordescriptionmust come backundefined, nevernull. Several document drivers need explicit configuration to honor this, since their native clients default to storing and returningnull. - Read-your-writes — a write has to be visible to the very next read. DynamoDB in particular needs
ConsistentRead: trueset, or its default eventually-consistent reads can fail this test intermittently.
None of these are edge cases a casual integration test would catch. They surface specifically
because the same 50 assertions run against every backend, so a driver that technically "works"
but diverges on one of these points fails the exact same way sqlite would if it
made the same mistake.
When a backend can't pass
The project's own guidance is explicit about which direction the fix goes: the conformance
suite "must never be weakened to make a backend pass — if a backend cannot satisfy it, that
is a finding to document, not a test to relax." That's a meaningful constraint to hold,
because the easy failure mode for any shared-contract test suite is death by a thousand
exceptions — one skip() for Redis here, one loosened assertion for Firestore
there — until the suite no longer actually guarantees interchangeability for anyone. Treating
a failed test as a backend limitation instead keeps the contract meaningful: if
postgres passes all 50 tests and mysql passes all 50 tests, a user
can actually trust that switching between them with db migrate won't change
behavior underneath their saved contexts.
This pattern — one reusable suite, a thin per-implementation adapter, run against both fakes and real services — isn't unique to open-context. dbt Labs proposed the same shape for dbt-core's database adapters: bundle shared test cases into a package so adapter maintainers prove behavior matches without copy-pasting tests per database. It's the standard way to make "drop-in replacement" a verifiable claim instead of a marketing one.
Adding a backend without breaking the contract
Adding a 16th backend to open-context means wiring it into an existing family, not building a
parser for the conformance suite itself. The short version: add the scheme to
DbScheme and SUPPORTED_SCHEMES, parse any backend-specific DSN
fields, write a Dialect or DocumentDriver, register it in
ADAPTERS and the build() switch, load the client library with
importOptional() so a missing package produces an install instruction instead of
a crash, and add it as an optional peer dependency. Then the suite does the verification work:
run it against the new backend until all 50 tests pass, and only then is it a backend
open-context actually supports — not one that merely compiles.
Point open-context's MCP server at any of its 15 supported databases.
Get started with open-context →FAQ
What is the open-context conformance suite?
It's a single file, tests/store/conformance.ts, containing about 50 tests that every one of open-context's 15 database backends must pass. It is the only definition of correct behavior for the context store — not a README, not a comment, the tests themselves.
How many database backends does open-context support?
Fifteen: json, sqlite, libsql, d1, duckdb, postgres, cloudsql, mysql, mssql, memory, mongodb, redis, firestore, dynamodb, and surrealdb. They share three adapter implementations — SQL, document/KV, and a bespoke SurrealDB adapter — rather than fifteen separate codebases.
Do I need to run every backend's database to test open-context's store?
npm test runs the full conformance suite against json, sqlite, and memory with zero external services, since those need only a temp file or RAM. The other 12 backends run via npm run test:backends once their connection strings are set, typically against the services in docker-compose.test.yml.
What happens if a backend can't pass all 50 conformance tests?
The test stays as written. If a backend genuinely cannot satisfy an invariant, that becomes a documented limitation of that backend, not a reason to loosen the suite — weakening the contract to fit one backend would let every other backend regress silently.
Does the conformance suite check credentials or connection strings?
Indirectly, through the adapter contract: every driver receives dsn.canonical, never dsn.raw, and everything user-visible about a connection string is expected to go through redactDsn() so passwords never leak into logs or error messages.