nbxgrammar 0.1.1, from the
v0.1.1tag of github.com/NeuroBrix/nbxgrammar.
NeuroBrix's grammar engine for constrained decoding. It compiles a JSON Schema, or a description of a tool-call
format, into a matcher that gives at every decoding step the mask of the tokens a model may write next — so the output
can only be what the schema or the format allows. It is the engine behind response_format: json_schema and forced
tool_choice in NeuroBrix's OpenAI-compatible server.
Written in Rust with the standard library only, loaded from Python with ctypes (no binding library). One wheel per
platform serves every Python 3.9 or later.
pip install nbxgrammar
import json, numpy as np, nbxgrammar as G
grammar = G.Grammar.json_schema(json.dumps({"type": "object", "properties": {"city": {"type": "string"}},
"required": ["city"]}))
vocab = G.Vocab(token_bytes, eos_ids) # each token id's bytes (None for a special token), the end-of-sequence ids
matcher = G.Matcher(grammar)
mask = matcher.mask(vocab) # int32 words, ceil(vocab / 32): bit v % 32 of word v // 32 = token v allowed
matcher.advance(vocab, token_id) # the token the sampler chose
neurobrix serve constrains a decode through nbxgrammar in two cases, on its OpenAI-compatible API: a request with response_format: {"type": "json_schema", ...}, and a forced tool call (tool_choice: "required" or a named function). parallel_tool_calls: false is enforced only with a forced call, and refused by name otherwise. With tool_choice: "auto" the decode is not constrained. The engine pins nbxgrammar on the platforms its wheels cover.
properties, required; no other property is generated), arrays (items,minItems, maxItems), strings, numbers, integers, booleans, null, enum, const, anyOf whose alternatives start$ref into $defs. Any other constraining keyword is refused by name (a ValueError), never ignored;title, description, default, examples, ...) are read as annotations.until, optionally ended by a special token), and exact token ids — what a chat template's tool-callMeasured on Qwen3's vocabulary (151 936 tokens, Apple M4 Pro), two different statistics: by kind of position on
generated answers, a mask takes about 15 µs median inside JSON structure and 73 µs in free text; the benchmark of
examples/bench.rs (random walks over the test schemas, every step counted) reads about 92 µs median a step.
json reads a schema's text. schema compiles the supported subset (a JSON Schema, or a description) to a node arena
and refuses what it cannot enforce. matcher is the automaton: the state of one output against one compiled grammar,
stepped by a byte or by a token step; its frames are private — the other modules ask it questions and step it.
vocab is the token tree and the masks. ffi is the C ABI. Each imports only the ones before it, and none of them
decides what to constrain or when: that is the caller's. No function is longer than 53 code lines (clippy.toml,
denied in CI): the limit is this crate's own 95th percentile when it was set, and it only goes down.
The crate pins its toolchain (rust-toolchain.toml). pip install . builds the library with cargo and places it in
the package; cargo test --release runs the oracle tests (every mask against a brute force over the vocabulary, every
completed output against an independent validator).
Apache License 2.0 (LICENSE); see NOTICE. Contributions are made under the
NeuroBrix Contributor Assignment Agreement
(CONTRIBUTING.md).
Project: neurobrix.org · documentation: docs.neurobrix.org ·
[email protected]
© 2025–2026 Hocine Benkelaya. NeuroBrix engine released under the Apache License 2.0.
Developed by Hocine Benkelaya · Supported and funded by Neural Network Holding Ltd · Commercial distribution by WizWorks OÜ