Skip to main content

Authoring a Language

Adding a language is the largest extension point, and it is the one the provided packages exist to demonstrate: a new language registers exactly like they do, as mainLang or in additionalLangs.

This page is the checklist, in the order you will build it. There is no end-to-end example here, because the smallest honest one is a parser plus a plan builder plus a visitor set, several thousand lines. Read @dortdb/lang-sql as the relational reference and @dortdb/lang-xquery as the reference for a language that adds its own operators.

1. A parser that can be interrupted​

The parser turns query text into an AST. Because any language can be nested in any other, it must handle language switches in both directions: hand off to another parser at a LANG <name> block, and know when to stop even with input remaining, at an explicit LANG EXIT, or when it would leave its own scope, for example at a closing parenthesis.

So the parser returns the parsed AST and any unparsed remainder, letting the outer parser resume. Parsing the nested block in

SELECT t1.attr1 FROM (
  LANG newlang
  -- new language here...
) AS t1 WHERE attr1 > 3

the nested parser should stop at the ) and hand ) AS t1 WHERE attr1 > 3 back to SQL.

2. A logical plan builder​

From the AST, build a plan of unified-algebra operators. This is the one visitor with no core default. It is what makes the language registerable at all. See how the provided languages map their syntax onto operators for the patterns to follow.

3. Schema inference across scopes​

The builder receives an IdSet of identifiers defined in the outer context. Some may end with the toInfer symbol, meaning they belong to a source whose schema is still being inferred. If your language references any of them, record them and return them alongside the plan, so inference can complete across the language boundary.

4. Visitors for every pass your operators reach​

If the unified algebra is not enough, define new plan operators. Every pass over the plan must then know how to handle them: dependency analysis, renaming, equality, variable mapping, execution. The full list, what each pass does, and which ones you can inherit is in Plan Visitors.

5. A data adapter and a serializer​

The data adapter decides how the language reads registered values; the serializer converts the executor's internal representation back into plain JavaScript values before results are returned.

6. Optional: the rest of the extension points​

A language package commonly also ships functions and aggregates, index types, and optimizer rules for its own operators.