The Normalized Conversation Schema Every Export Uses
open-context's normalized conversation schema is a single shared shape —
NormalizedConversation and NormalizedMessage, defined once in
src/parsers/types.ts — that every provider parser has to convert its raw export
into before anything else happens. The markdown formatter, the Ollama preference analyzer,
and every future provider parser all read this one shape, and none of them ever touch a
provider's original JSON directly.
Key takeaways
- Every parser (ChatGPT today, Gemini planned) converts into the same
NormalizedConversation/NormalizedMessageshape, so the formatter and analyzer never touch provider-specific JSON. - Messages collapse to three roles —
user,assistant,system— even though ChatGPT's raw export has a fourth role,tool, which the normalizer folds intoassistant. - The ChatGPT parser walks a tree of linked
MessageNodeobjects breadth-first from the root, so branched or edited conversations come out as one flat, ordered list of messages. - Images and attachments become plain string references (asset pointers, attachment names) rather than binary data, keeping normalized output small and diffable.
- Adding a new source provider means writing a parser that emits this schema — nothing downstream, including the formatter, needs to change.
The problem one schema solves
Every AI provider exports chat history in its own shape. ChatGPT's conversations.json
doesn't store messages as a simple array — it stores a mapping: Record<string, MessageNode>,
a tree of nodes keyed by id, each one holding a parent pointer and a children
array. That structure exists because ChatGPT lets you edit a message and branch the conversation,
so the raw export is technically a graph, not a list. A future Gemini export (via Google Takeout)
will look nothing like that.
If the markdown formatter and the preference analyzer each had to understand both formats directly, every new provider would mean touching code throughout the pipeline. open-context avoids that by defining one intermediate shape once, in the conversion pipeline's normalize stage, and requiring every parser to produce it.
What the schema actually contains
At the conversation level, NormalizedConversation has four fields:
| Field | Type | Notes |
|---|---|---|
id | string | Taken from the provider's own conversation id (e.g. ChatGPT's conversation_id) |
title | string | Falls back to "Untitled Conversation" if the source has no title |
created | Date | Parsed from the provider's Unix timestamp |
updated | Date | Same, for the last-modified timestamp |
messages | NormalizedMessage[] | Flattened, chronologically ordered |
Each entry in messages is a NormalizedMessage, with five fields:
| Field | Type | Notes |
|---|---|---|
role | 'user' | 'assistant' | 'system' | Collapsed from whatever roles the provider uses |
content | string | Trimmed text content |
images | string[] (optional) | Asset references, not binary data |
timestamp | Date | Parsed from the message's own Unix timestamp |
metadata | { model?, attachments? } (optional) | Model slug and attachment names, when present |
Four roles become three
ChatGPT's raw author.role can be user, assistant,
system, or tool. The normalizer's normalizeRole() function
collapses tool into assistant, and defaults any unrecognized role to
assistant as well. Downstream code — the markdown formatter, the preference
analyzer — never has to special-case a tool call; it just reads as another assistant turn.
How the ChatGPT parser fills in that shape
Flattening a tree into a list
walkMessageTree() in src/parsers/chatgpt.ts finds the one node whose
parent is null, then runs a breadth-first traversal from there,
queueing each node's children and pushing every valid message it meets into an
array in visit order. The result is a single, linear Message[] built from what
is, in the raw export, a branching graph.
What gets filtered out first
Not every node becomes a message. isValidMessage() drops system messages with no
meaningful content, along with any message that has neither parts nor non-empty
text. One level up, ConversationNormalizer.isValidConversation() drops
whole conversations that end up with zero messages, or where every remaining message is empty —
a chat that was started and abandoned before any real content was typed, for example.
Images and attachments become references, not files
extractImages() pulls asset_pointer values out of multimodal content
parts, plus the id of any attachment whose mimeType starts with
image/, and returns them as a plain string[] on
NormalizedMessage.images. Non-image attachments become their name alone,
via message.metadata?.attachments?.map(a => a.name). The schema never carries
binary data — that keeps a normalized conversation small enough to serialize straight to JSON or
markdown, and it matches open-context's local-only design: nothing has to be uploaded anywhere
just to track that an image existed.
Why this is the extension point for new providers
CLAUDE.md's "Adding a New Source Provider" guide lays out six steps, and step 3 is the one
that matters here: "Map the provider's roles, content types, and metadata to the normalized
schema." Because the markdown formatter
(src/formatters/markdown.ts) and the Ollama preference analyzer both import only
from parsers/types.ts — never from chatgpt.ts directly — a Gemini
parser, or a parser for any other provider, can ship without the formatter or analyzer changing
at all. The contract is the schema, not the code that produces it. If you're writing one, see
how to write a new AI provider parser
for the full walkthrough, including fixtures and tests.
See the normalized schema in action on your own export.
Get started with open-context →FAQ
What is open-context's normalized conversation schema?
It's the shared intermediate shape — NormalizedConversation and NormalizedMessage, defined in src/parsers/types.ts — that every provider parser converts its raw export into. The markdown formatter and preference analyzer only ever read this shape, never a provider's original JSON.
Does normalizing discard the original ChatGPT export data?
No. The parser reads your original conversations.json and produces normalized objects in memory for conversion; your original export ZIP is never modified or deleted.
What happens to ChatGPT's "tool" role messages?
The normalizer's normalizeRole() function folds the tool author role into assistant, since the normalized schema only recognizes three roles: user, assistant, and system.
Will a future Gemini parser use the same schema?
Yes. Any new provider parser, including the planned Gemini support, is expected to output the same NormalizedConversation and NormalizedMessage shape so the rest of the pipeline needs no changes.
Can I write a parser for a provider open-context doesn't support yet?
Yes. CLAUDE.md's "Adding a New Source Provider" section walks through it: study the export format, create a parser in src/parsers/<provider>.ts, map its roles and content to the normalized schema, add fixtures, and write tests.