nbxtensor 0.0.8, from the
v0.0.8tag of github.com/NeuroBrix/nbxtensor. Files nameddocs/nbxtensor/…in this text are not part of the nbxtensor repository at this version.
Run AI models without PyTorch, on any GPU Triton supports: tensors, Triton kernels, planned memory and certified tuning. Inference first; training is not in this release.
¶ nbxtensor — the torch-free tensor library for inference
Run AI models without PyTorch, on any GPU Triton supports: tensors, Triton kernels, planned memory and certified tuning.
Runs wherever Triton runs — NVIDIA, AMD, Intel and Apple Silicon GPUs, on Linux, Windows and macOS. Inference first; training is not in this
release. Found a bug on your hardware? Open an issue — we fix fast.
pip install nbxtensor
import numpy as np
from nbxtensor.tensor import NBXTensor
from nbxtensor.ops.linalg_wrappers import mm
rng = np.random.default_rng(0)
a = rng.standard_normal((256, 512)).astype(np.float16)
b = rng.standard_normal((512, 128)).astype(np.float16)
c = mm(NBXTensor.from_numpy(a), NBXTensor.from_numpy(b)) # a Triton kernel on the GPU, no torch
ref = a.astype(np.float32) @ b.astype(np.float32)
assert c.shape == (256, 128) and np.allclose(c.numpy().astype(np.float32), ref, rtol=2e-2, atol=2e-1)
¶ Fast: tensor cores back on the V100
Still running V100s? Triton leaves their tensor cores idle; nbxtensor turns them back on. On sm_70 Triton compiles tl.dot to plain FMA; nbxtensor ships its own mma.m8n8k4 kernels, and a kernel runs on the matrix unit only where its certificate measured it faster.
| V100 kernel | FMA | matrix unit |
|---|---|---|
| GEMM 80 x 4096 x 4096, fp16 | 0.404 ms | 0.191 ms (x2.1) |
| fused MoE, DeepSeek-Coder-V2-Lite, 20-token gate | 2.49 ms | 0.96 ms |
| fused MoE, Qwen3-30B-A3B, 512-token gate | 7.24 ms | 1.40 ms |
| 1 x 1 convolution, fp32, video VAE shapes | x1.26 to x1.95 |
Kernel timings inside nbxtensor, FMA against the matrix unit, as recorded in CHANGELOG.md at v0.0.1. These timings compare nbxtensor's own kernels with each other. Faster than PyTorch on many models.
¶ Lightweight
Lightweight: nbxtensor itself is 1 MB; the whole install, Triton included, is about 1 GB — PyTorch's is 5.6 GB. Measured on a fresh container, Python 3.10.
¶ Devices
Runs wherever Triton runs — NVIDIA, AMD, Intel and Apple Silicon GPUs, on Linux, Windows and macOS. Found a bug on your hardware? Open an issue — we fix fast.
¶ Certified tuning
Kernel settings measured once per GPU class and shipped — no autotuning at run time for the shapes we cover. We are certifying every model we can, so you never have to autotune. Certified your GPU? Share your certificates.
¶ Concepts
Plain data at the edges. A dtype is named by its table name ("float16", "bfloat16" …), a shape is a tuple of
ints, strides are a tuple of ints counted in ELEMENTS (never bytes), a device is a kind and an index. Nothing in the
API takes a duck-typed object or keeps state in a module global: the same API is meant to be implemented later by a
Rust core.
One table. Every width, safetensors code, launch-key spelling and kind of a dtype is one entry of dtypes.yml.
A name the table does not carry is refused by name; no width is ever guessed.
Zero fallback. A missing fact is an error that names it, never a default.
¶ Limits
- Inference only: no autograd.
- Slicing has step 1 (
narrow); a strided slice is refused by name. - A dtype absent from the table is refused; add it to
dtypes.ymlwith its source.
¶ Versions
The first publication was 0.0.1; this is 0.0.5. Each fix published after it increments the number (0.0.6,
…). The rewrite of the core in Rust will be 2.0.0. nbxtensor.__version__ is the
installed version.
¶ License
Apache 2.0.
© 2025–2026 Hocine Benkelaya. NeuroBrix engine released under the Apache License 2.0.
Developed by Hocine Benkelaya · Supported and funded by Neural Network Holding Ltd · Commercial distribution by WizWorks OÜ.
¶ In this section
Product page: neurobrix.org/nbxtensor