Skip to main content

Architecture

@dortdb/core is a small, language-neutral engine. It owns the query lifecycle, from registration through planning, optimization, and execution, but ships no query language itself. Languages, functions, indices, and optimizer rules are all plug-ins that attach to the core, and each is a separate package, so a deployment bundles only what it uses.

The query pipeline​

Every query follows the same path:

  1. Register. In-memory data is registered as a source (an O(1)O(1) operation). Optionally, it is indexed.
  2. Parse. The selected language plug-in parses the query text into an abstract syntax tree (AST).
  3. Build a plan. The AST is lowered into a logical plan of unified-algebra operators. Because every language targets the same algebra, a query that mixes languages still becomes one plan.
  4. Optimize. The rule-based optimizer rewrites the plan, pushing down selections, using indices, and so on.
  5. Execute. The executor evaluates the plan lazily over the registered sources.
  6. Serialize. The executor works with an internal representation, so a language serializer converts results back into ordinary JavaScript values before they are returned.

The DortDB methods map onto these stages, and you can stop between any two of them; see Running Queries and Inspecting Plans.

Extending the algebra: the visitor pattern​

The core processes plans with an extended visitor pattern. A plan operator's accept() method receives a dictionary of visitors keyed by language:

accept<Ret, Arg>(visitors: Record<string, PlanVisitor<Ret, Arg>>, arg?: Arg): Ret;

Dispatch therefore keys on two things: the operator's type, which decides which visitXxx method runs, and the operator's lang tag, which decides whose visitor runs it. That tag is the language that instantiated the operator, not the language the operator type comes from. A Selection built by the SQL plan builder is tagged sql and handled by SQL's visitor; the very same core operator type built by XQuery is tagged xquery and handled by XQuery's, all within one traversal.

This is what lets a language grow the algebra without changes to the core. A language implements the visitor interface, usually by subclassing the core visitor and overriding only the methods for the operators it adds. That is how XQuery contributes a TreeJoin operator for path navigation. It is also the seam every deeper extension point sits on; see Extension Model and, for the passes a language must cover, Plan Visitors.

Execution model​

Execution is lazy, synchronous, and single-threaded: iterating a result pulls rows through the operator tree on the calling thread, so functions and aggregates must be synchronous.

The engine is also schema-free. Sources are plain in-memory values read through data adapters, which is what keeps registration free and languages decoupled from data shape, and what makes a few schema-dependent language features unavailable. See Design Principles for why these trades were made, and Limitations for what they cost.