The array language
markets run on.
Amber is a small, fast, self-contained engine written in portable C99: columnar,
vectorised, in-memory, with the working vocabulary of q/kdb+ — dictionaries,
tables and keyed tables, the full join family including a native as-of kernel, qSQL
select/by, and column attributes implemented in C that turn search from
O(n) into O(log n).
Built for the shape data actually has
A tick is not a row. It is one value in each of nine columns, and every question you ask of a trading day is a question about columns. Amber is columnar all the way down — storage, evaluation, the wire, and the handoff to Arrow.
Zero-copy memory layout
A column is one contiguous, cache-line-aligned payload. amber_get_vector_ptr()
hands you that pointer — not a copy of it. NumPy's base, Arrow's data buffer and the
engine's column are the same address, asserted by pointer equality in the test suites.
Native TCP wire protocol
amberd is ~450 lines of C: one engine, many connections, a one-line
request and a length-prefixed reply in text, json, jsonc or
raw. Small enough to reimplement in an afternoon — which Go, TypeScript and Python
clients all have.
Array evaluation, not row loops
Every expression compiles to a flat opcode array and runs on a real stack VM.
Vector ops dispatch into AVX2 / NEON kernels and split across cores above 100,000 elements.
\disasm decodes the bytecode the compiler actually emitted.
Columnar storage & attributes
All four kdb-style attributes are implemented in C — `s sorted,
`u unique, `p parted, `g grouped. Sorted turns
? into a binary search: 1.73 s → 1.4 ms on 2M rows, same answer.
Native as-of join
aj matches each trade to its most recent quote with a branch-free
lower_bound over each symbol group's sorted nanosecond slice, off a thread-local
16 MB bump arena that keeps malloc jitter out of the hot path.
Diagnostics that read like Rust
An error[E0104] line, a --> locator, a gutter-aligned
source line, ^^^ underlines and a = help: note. One report, never two —
and switchable at runtime so a trap loop stays quiet.
q vocabulary, K notation
If you have written q, you already know most of Amber: select … by … from … where …,
aj, wj, xbar, wavg, meta,
([]…) table literals and keyed tables all mean what you expect. Bare qSQL works at the
prompt with no sel"…" wrapper.
The notation underneath is terse array grammar. Dyadic library functions take brackets
(aj[c;x;y]), there is no >= or <=, and symbols cannot
contain _ — because _ is a verb.
/ tables are first-class and render without `show`
t:([]sym:`AAPL`MSFT`AAPL; px:187.3 411.2 187.4; sz:100 250 50)
/ qSQL, typed bare at the prompt
select vwap:wavg[sz;px] by sym from t
/ 1-minute OHLCV bars — the classic tickerplant query
tb:+@[+trades; ,`time; minbar[1]@]
qby[tb; `sym`time;
`o`h`l`c`v!({first x`px};{max x`px};{min x`px};{last x`px};{sum x`sz})]
/ TAQ: the prevailing quote for every print
m:aj[`sym`time; trades; quotes]
/ sorted attribute => O(log n) lookups
v:asc 2000000?1000000000
`at v / `s
v ? 12345 67890 / binary search, ~1244x faster
/ zero-copy Arrow C Data Interface, no libarrow linkage
p:arrow.export t / (schemaAddr; arrayAddr)A whole market, generated and stored
amber-tick writes a partitioned, splayed column store the engine reads natively — 38M prints, 89M quotes, ten-level book snapshots and LULD halts across five sessions and 500 names, from a simulator whose realism is checked: twenty-one published stylized facts, all passing on every generated store.
- arrivals
- discrete-time Hawkes process — bursts, not flat Poisson
- quotes first
- so a print can never fall outside the NBBO
- epoch
- nanoseconds since 2000-01-01 — Amber's own, not Unix
- files
- each column is one
-8!value, byte-identical to the engine's
Simulated tape: mean-reverting tick returns (bid-ask bounce) with clustered arrivals.
One engine, three seams
Everything that consumes Amber lives
outside the engine repository and reaches it through libamber.so, the in-process
ext/ registry, or a TCP socket. The engine gained one build flag, one export map and one
header section for all of it.
Measured against C, and against everyone else
Ten engines, four workloads, one specification. Every answer is
an integer that fits float64 exactly, so summation order cannot change it — the harness compares
answers bit-exactly against the C reference and prints WRONG instead of a
time for any engine that disagrees.
| Workload (10M rows unless noted) | C -O3 | Amber | Amber qSQL | CBQN | NumPy | DuckDB | ngn/k |
|---|---|---|---|---|---|---|---|
| Vector arithmetic + mask | 8.22 | 54.56 | 65.73 | 30.91 | 27.47 | 22.00 | 254.82 |
| Reductions — sum + max + dot | 21.92 | 15.40 | 52.46 | 4.09 | 8.62 | 29.00 | 237.93 |
| Group-by — 100 groups | 6.41 | 61.45 | 119.63 | 28.83 | 11.54 | 14.00 | 276.51 |
| Inner join — 1M × 1,000 sparse keys | 0.83 | 3.42 | 9.85 | 1.80 | 11.34 | 9.00 | 364.13 |
Median of 5 timed runs after 2 warm-ups, milliseconds, kernel time only. Amber is reported twice on purpose: array-primitive code is the fair peer of ngn/k and CBQN; the qSQL layer is the fair peer of DuckDB's planner. The gap between the rows is the query layer's overhead, and it is meant to be visible. Full results →
Nothing here patches the engine
Each satellite reaches Amber through a published seam and nothing else. Delete any of them and the engine is byte-for-byte what it was.
python-amber
libamber.soThe engine in your Python process. pip install amber, then NumPy views
that point at the engine's own column buffers — no server, no socket, no serialisation.
amber-arrow
libamber.soAn ArrowArrayStream whose batches are windows onto one export,
not slices of it. Plus amberd, the TCP query server, and an Arrow Flight daemon.
amber-jupyter
python-amberAmber cells and %%python cells in one process, so a column crosses the
boundary as a pointer. HTML grids with type tags, capped at 500 rows on purpose.
amber-tick
store + runtimeA US-equity simulator, a splayed partitioned store, TAQ analytics, and a live tickerplant → RDB → HDB pipeline with a replayable sequenced log.
grafana-amber
amberd socketLive dashboards over bare qSQL, column-oriented on the wire. A Go backend, a
mage build, and time macros that speak Amber's epoch rather than Unix's.
vscode-amber
LSPqSQL-aware completion where the clause decides the vocabulary, hovers that explain
idioms like x@<x, and diagnostics that never evaluate your code.
amber-ai
ext/ seamA schema-aware co-pilot that reads your live tables before it answers, talks only to
127.0.0.1:11434, and cannot hang or break the REPL.
Amber Notepad
WebAssemblyThe real C interpreter compiled to wasm32 with plain clang — editor,
AST and bytecode panes, twelve bundled programs, entirely offline.
Tutorials
start hereThree end-to-end walkthroughs: a 5-million-row tick store, a Grafana dashboard pipeline, and Arrow record batches into Python without a copy.
One command. No install.
Amber compiles from source on first run and
installs nothing system-wide. There is no bin/, no QHOME, no dotfile —
deleting the folder uninstalls it completely.
git clone https://github.com/BonucciAndrea/amber.git
cd amber
chmod +x a build.sh install.sh
./a # builds with -O3, then opens the REPL