nbxtensor 0.0.8, from the
v0.0.8tag of github.com/NeuroBrix/nbxtensor. Files nameddocs/nbxtensor/…in this text are not part of the nbxtensor repository at this version.
Mirrored from CHANGELOG.md at v0.0.8. Also at neurobrix.org/nbxtensor/releases.
All notable changes to nbxtensor are documented in this file.
The format is based on Keep a Changelog,
and this project adheres to Semantic Versioning.
backend.level_zero.gpu_facts reads an Intel GPU's integrated flag,maxMemAllocSize) from Level Zero; backend.device_files.for_pci_idbackend.install_help.pci_display_devices lists the display controllers on the PCIautotune_certified keeps an entry's (or memory-classproof and adds the proofs of the same configuration under other generators in also_proven; where thealternates. A re-proof under a new generator therefore adds its proof and never un-serves anotherproof alone reads a file as before. New names: certification_proofs, class_certifications,entry_certifications, entry_proofs, certification_for_generator, class_proven_since, reproof_source,ALSO_PROVEN, ALTERNATES.nbxtensor.memory.paged_kv.PagedTritonKVCache (per-layer pagetruncate, gather_pages), moved fromon_reallocate(uid, generation) hook in place of its call into the engine's replay plans;chunk_attention, chunk_mask and require_causal_mask: a prefill chunk at a cached length attending over thenbxtensor.memory.pages (moved from the engine): the page pool, page tables, reference counts and fencednbxtensor.memory.kv_session (moved from the engine): the positions a serving session's cache holds betweenKVSession, held_ids, forgetting_on_failure), and chunk_rows, the rows of eachpaged_kv_gather (paged_kv_gather_kernel): a layer's K or V of every sequence as whole pages, contiguous,autotune_certified.served_certification: the certification served one key on a memory class, read by a plannergenerator= takes the running generator a caller pricing many keys asked once (on anbxtensor.terminal, the one terminal rule of a command line built on the library (python -m nbxtensor,neurobrix): one stderr handler, INFO by default, -v the developer lines, -vv an error's traceback too, andNBX_LOG=debug|info|warning|error from the environment (a name that is no level is refused by name).backend.install_help.compiler_missing: names a machine without the C (or C++) compiler Triton builds a kerneltriton/runtime/build.py, _find_compiler), never from the text.backend.install_help.hidden_gpus (the GPUs the backend's visibility variable — the table's new visible_env,CUDA_VISIBLE_DEVICES on CUDA, ROCm's three on HIP — hides from this process, against the PCI bus) andbackend.cpu_backend.triton_cpu_installed.io.read_rate.read_seconds (how long this storage takes to read so many bytes, measured now) andmemory.trace_replay.first_replay / vmm_granularity (the chunk probe's first replay made ahead, its costpointer_tables.TablePins and end_call_pins: a holder for the pins of per-call pointer tables built for a cache-v / NBX_LOG=debug), and the plan's explanation still counts the kernels served that way.*.cpython-312-*.so, *.cp312-*.pyd) is built per Python and generates no code, so the Metal wheels for Pythonneurobrix autotune certify --reprove-generator).logging, one logger per module, and importing itpython -m nbxtensor doctor lists the GPUs the process sees: it listed every display controller on the host'spython -m nbxtensor install-metal downloads nothing when the pinned wheels are already installed, and says so inNBX_LOG=debug.masked_fill: a uint8 mask reached tl.where as its condition, whichTablePins, they end when theos.preadv (Windows' ReadFile with annumpy.libs) instead of through /proc, and thebin, nvml.dll,python -m nbxtensor doctor on a Mac without Triton exits 1: it named the fix and exited 0, so a script couldos.sysconf, which Windows does not have; and the code/ paths with its .pyd and .dll binaries, so the same codepython -m nbxtensor install-metal installs the Metal wheels of thenbx-triton-apple, triton-lang/triton-ext#126), each by the release URL and sha256 written in the library,--no-deps --require-hashes --no-index — never an extra index nor a package name resolved on PyPI. Thepython -m nbxtensor doctor on a Macpyobjc-framework-Metal, through which it opens the Metal device; without it the run fell to a CPU profile and wastriton on every platform, and PyPI publishes no Triton for macOS, sopip install nbxtensor — and with it the NeuroBrix engine, even its PyTorch engine on MPS — could not resolve on abackend.launcher.prewarm_compiler(): computes Triton's cache key (triton_key, the sha256 of libtriton and of Triton's sources, about 1.3 s on a V100 host) in a daemon thread, so a process's first kernel compile no longer waits for it; an application calls it at start, while it loads weights.backend.level_zero, backend.level_zero_driver; the xpu backend, chosen when the system's Level Zero loader reports a GPU): a runtime over Level Zero that serves the allocator (device and page-locked host memory, in-order command lists as streams, timed events), and a launch driver for the SPIR-V kernels Intel's Triton build compiles, with Intel's own argument ABI; the compile target is built without torch or SYCL (Level Zero's counts, the OpenCL ICD's extensions, the device file's architecture name). Proven against stand-ins of libze_loader and the OpenCL ICD: no Intel card has run it yet.backend.install_help, python -m nbxtensor doctor): for every NVIDIA, AMD or Intel GPU on the PCI bus, whether its runtime library and its Triton backend are installed, with the vendor's install commands when one is not; the refusal when no GPU runtime answers carries the same steps instead of a bare error. The README's "Install on AMD / Intel GPUs" section gives them.backend.launcher.HipDriver, chosen when the runtime detects HIP): the HIP module API through the HIP runtime library the runtime already loaded, the launch Triton's own AMD launcher performs (hsaco code objects, hipModuleLaunchKernel, the kernel's arguments then the scratch pointers, a block of the code object's wavefront times the kernel's warps), one module per device and code object, the argument count read from the code object's own metadata and a list of another length refused, shared memory past the device's per-block maximum refused as Triton's own AMD launcher refuses it, the wavefront read from the code object and the device file (never defaulted), the launcher contract's compile surface shared with the CUDA driver, every missing symbol refused by name; page-locked host memory is asked of hipHostMalloc with its flags word. Its tests compile real Triton kernels for gfx90a, gfx942, gfx950, gfx1100 and gfx1201 on a machine without an AMD card and launch them through a stand-in HIP library: no AMD card has run it yet.NBX_MM_MAX_OUTPUT_ELEMS (the matrix product's output split, fixed by an int32 element offset), NBX_FLASH_BLOCK_M_VOLTA (Volta's flash Q tile, 64, the measured floor of correctness), NBX_CONV3D_EAGER_FREE_BYTES (when a 3-D convolution frees its buffers early), NBX_FLASH_NUM_WARPS and NBX_FLASH_NUM_STAGES (the flash kernel's launch, which its device file's flash rows state), NBX_DGEMV_ADAPT (the dequantising GEMV's width adaptation, set by dequant_gemv.min_programs), NBX_D128_DETOUR (the power-of-two head-dim detour, decided by flash.pow2_head_dim_correct; launch_keys.flash_headdim_detour loses its enabled argument), NBX_DEPTHWISE_DISABLE (the depthwise convolution route) and NBX_GC_ON_OOM (a garbage-collecting retry of a refused allocation, which is raised, never rescued). A value a kernel needs is a fact of its device file or of a certificate, never a host setting.ops.certified_index): the directory stays JSON, the source contributors edit; the first launch after a change validates a file and writes, per memory class, a binary index of the entries that class is served (length-prefixed keys and certifications, a SHA-256 trailer, never a pickle), named by the fingerprint of everything the validation read; later launches read the index and parse a certification only when its key is asked. A corrupt index is refused by name.backend.launcher.set_launch_recorder: fn(prepared, grid, params, extents)): the byte extent of each pointer's tensor that a driver binding by extent is handed, None for a driver that binds by address — a replay can now launch exactly what was launched.pip install nbxtensor: the 0.0.1 README began its installation with pip install -e nbxtensor from the engine's repository, which PyPI showed as the package's description; it now opens with its title, what it does, the install from PyPI (the repository install is below, for contributors), a GPU example its tests run, why the library exists (no PyTorch, any Triton GPU, the V100's tensor cores back in use with their measured gains, planned memory, certified tuning) and the hardware tested and expected, and it links its API and backend references in the NeuroBrix repository, where they live (they were relative links to files the library's repository does not hold). The package's one-line description no longer names a CPU backend, which the library does not have.The first release of nbxtensor: an inference-first tensor library whose kernels are written in Triton and which never
imports torch. NeuroBrix's Triton engine is its first client: on 2026-11-05 the engine's final gate ran its 49 catalogue
models through the library on four V100 cards (both card classes, every output judged by eye), and the library's tests,
five reference models and an uncertified autotune sweep ran in an environment where torch is not installed, with no
attempt to import it.
What the library holds
nbxtensor.Tensor with its dtype table (dtypes.yml: every width, safetensors code and__dlpack__, from_dlpack): a torch user writes torch.from_dlpack(t) on their side.memory.slab), and the capture and replayHardware: NVIDIA V100 (CUDA) and Apple M4 Pro (Metal through triton-ext) are proven; NVIDIA Ampere and Hopper and
the other Apple silicon chips have device files and are not proven on our machines; AMD Instinct (HIP) and Intel GPUs
(Level Zero) have their interfaces written and are not tested for lack of hardware; there is no CPU kernel backend.
Named refusals: a missing fact is an error that names it, never a default — a dtype absent from the table, a
device fact absent from its device file, a strided slice, a signed integer promoted with uint64, a Python integer
outside an integer array's range, a pointer Metal declines to bind in place, a per-row bias of more than four rows in
addmm, an uncertified key where certified settings are required.
Known limitations: inference only (no autograd); slicing has step 1; no array namespace (__array_namespace__)
yet; conv_transpose2d runs on the FMA kernel only (no matrix-unit path; no catalogue model forms it); on Apple silicon,
two of NeuroBrix's video models do not complete on the M4 Pro yet — mochi-1-preview (the PyTorch engine's memory at its
decoder's temporal stream) and Allegro / Allegro-TI2V on the Triton engine with its replay (a Metal command buffer out of
memory; the fix is in test).
nbxtensor.memory.slab.first_fit_extent: the slab a run needs, the highest offset the slab's own first fitbackend/devices/nvidia/ (Volta, Ampere, Hopper) andbackend/devices/amd/ (CDNA, CDNA2, CDNA3) hold the facts its kernels read — tiles, shared memory, registers,fused_moe_m8n8k4_kernel, moe_wrappers.invoke_fused_moe_m8n8k4): the FMA kernel's contract — the alignment's blocks, the expert pointer table, the top-k division, the routed weight — on the m8n8k4 unit at the profile's tiles, the routing aligned at the tile's rows. Its certificate decides per key against the FMA kernel (fused_moe_unit_tile); on a V100 the unit measured 2.5-5x faster on every prefill projection timed (DeepSeek-Coder-V2-Lite's 20-token gate 2.49 -> 0.96 ms, Qwen3-30B-A3B's 512-token gate 7.24 -> 1.40 ms).nbxtensor.backend.device_files.in_force(), for_arch(arch) and forget(), and nbxtensor.backend.launcher.bind_target(target) for a derivation that launches nothing; the engine's key census and paged KV cache read them instead of the library's private configuration module.backend.launcher.set_launch_recorder: fn(prepared, grid, params, extents)): the byte extent of each pointer's tensor that a driver binding by extent is handed, None for a driver that binds by address — a replay can now launch exactly what was launched.autotune_screen_rtol.int64: 0.0 — met exactly (the MoE alignment's routing tables); the certifier invents no tolerance and refused every alignment key without it.moe_align_stage{1,2,3}_kernel ran per-backend literal widths (128 or the experts' count on CUDA, 32-64 on Metal) at 4 warps; each is now certified per key (the block, the padded expert count, the layer's pairs in the profile's elements classes), its candidates pruned to the blocks that fit the profile's memory.max_shared_memory_per_block. Every candidate computes the same routing; the choice is speed only.nbxtensor.memory.pointer_tables.build_ptr_tables is public: the expert pointer tables a MoE kernel reads (on Metal captured under a pinned scope), built by the engine's MoE layer and by its certifier alike; it was the private _build_ptr_tables.flash_decode_reduce_kernel ran at a literal 4 warps after both split decodes (vector and paged); it is now certified per key (the splits, the value head dim, the output dtype) through attention_wrappers.decode_reduce, and decode_attn_vec.decode_vec_splits names the vector decode's split rule for the wrapper, the census and the certifier alike.paged_decode_attention ran the judged tile (32, PAGED_DECODE_JUDGED_TILE, removed) at 4 warps; the tile is now certified per key among the candidates that divide the page, and the call takes no tile argument. A split's positions are whole pages' worth of the largest tile (PAGED_DECODE_SPLIT_ALIGN), so every candidate cuts the splits alike. The certificate is keyed by the splits the longest sequence makes (paged_decode_attn.paged_decode_splits), the same count a replayed decode step is bucketed by, so a replayed step and the step it is verified against run one configuration.mm and addmm took the m8n8k4 unit whenever its smallest tile filled, at the first tile of the profile that fit, a launch the certification never compared with anything. The FMA launch now keeps its autotune key on the unit too; certifying it times every unit tile the profile declares beside the FMA configurations, against the same oracle, and the entry names the unit's tile only when it was faster (a 80 x 4096 x 4096 fp16 product: FMA 0.404 ms, the first tile 0.207, the second 0.191 — the second is served). A run with no certificate keeps the declared unit; a confirmation run refuses the missing key.conv_transpose_wrapper, groups 1, dilation 1, no output padding): x^T [L, C_in] @ W [C_in, C_out*K] through mm (the length on the rows, which the autotune key classes) — the matrix unit where the hardware profile declares one — then col2im1d_kernel gathers each output position's taps. The scatter kernel looped over every input channel and tap per output element; the vocoders' upsamplers ran 20-307x slower on a V100. Same output dtype; the waveform of a chatterbox sentence agrees with the scatter kernel's at 49.5 dB SNR, its transcription exact.decode_attn_vec_split_kernel ran 32 keys a step and 4 warps, read from environment variables (NBX_DV_BLOCK_N, NBX_DV_WARPS) with those defaults; both are now candidates certified per key (the keys' bucket, the head dims, the GQA group, the bias). The split (256 keys a program, at most 64) stays the kernel's: it defines the partials.moe_decode_vec) ran a hand table (32 x 128, 4 warps; 256 / 2 warps for the sum); each is now certified per key over the ladder the table was measured from (Qwen3-30B-A3B's 2048 x 768 experts on a V100: gateup 16 x 64, down 16 x 256, sum 1024).fused_moe_kernel's column and reduction tiles, warps and stages were literals (64 x 32, 4 warps, 2 stages, moe_wrappers._MOE_BN/_MOE_BK); they are now candidates the certification measures per key — the GEMM's dims, the valid (token, expert) pairs' bucket and the layer's experts (launch_keys.fused_moe_launches). The row block stays the routing alignment's. A 512-token Qwen3-30B-A3B prefill's gate projection takes 128 columns.conv_transpose_wrapper, groups 1, dilation 1, no output padding): W^T [C_out*K, C_in] @ x through mm — the matrix unit where the hardware profile declares one — then col2im1d_kernel gathers each output position's taps. The scatter kernel looped over every input channel and tap per output element; the vocoders' upsamplers ran 20-307x slower on a V100. Same output dtype; the waveform of a chatterbox sentence agrees with the scatter kernel's at 49.5 dB SNR, its transcription exact.nbxtensor.ops.linalg_wrappers.mm_cost / addmm_cost, beside bmm_cost): the casts, the decode rows' GEMV and the weight copy a row-major weight costs there, the bias made contiguous, the output at its store dtype, an epilogue's output; each equal to the allocator's measured peak on the device, over the dtypes, the decode and tiled row counts, both weight layouts and the bias forms.libcudart.so.12), which the driver does not ship; on a clean Linux machine no GPU runtime was found and the card was read as a CPU. nbxtensor now depends on nvidia-cuda-runtime-cu12 on Linux.set_store(None) reset the certified store every later test of that process read (a test of the livemoe_gateup_vec_half_kernel and moe_down_split_vec_half_kernel prune their tiles to the [BLOCK_K, BLOCK_N] fp32 blocks memory.max_shared_memory_per_block holds before any compile (on Apple's 32 KB a 256 x 64 block aborted the Metal emitter); the door is _configs.fits_shared_memory.memory.max_registers_per_block and the like) were read through the profile and found absent, so a reduction orDeviceOOMError with the requested, live, pool-cached and driver-free bytes, as the driver path does, so aBLOCK_HW, BLOCK_C) were served to the new one (BC, TH, TW). Such a file is nowKeyNotCertified naming the kernel and the exception.expandable_segments; the chunk is a measured fact of the generated profile (vmm_chunk_bytes: the smallest multiple of the granularity whose map costs no more than an ordinary allocation, 2 MiB on V100), refused by name when absent; a device without VMM keeps its driver segments. The small class and the weight arenas are unchanged; memory.pool.simulate replays the same rules.memory.pool.simulate, M3's replay) completes with the least held of every pooled rule tried, on Allegro 16g, mochi-1-preview 32g and Kokoro alike.addmm refuses a per-row bias [M, N] above four rows instead of computing it wrong: its kernel adds one bias row to every output row, so such a bias was applied with its first row only. It is now refused by name (a bias [N] or [1, N], which every catalogue model uses, is unchanged; a per-row bias at up to four rows is still served).decode_attn_vec_grouped_kernel and the environment switches of the vector decode (NBX_DV_GROUPED, NBX_DV_SEG, NBX_DV_BLOCK_N, NBX_DV_WARPS): the grouped variant ran only behind its switch.flash_decode_split_kernel and its route: armed only by an environment switch (NBX_FLASH_DECODE, with NBX_FD_BLOCK_N and NBX_FD_SEG) and measured slower than the vector decode kernels; flash_decode.py keeps the fixed-order reduce they share.conv1d_forward_kernel: launched by no route (a 1-D convolution runs through conv2d).