basetenlabs/nixl

basetenlabs/nixl

Description: NVIDIA Inference Xfer Library (NIXL)

License: NOASSERTION

Stars: 0

Forks: 0

Open issues: 1

Created: 2026-04-07T21:32:07Z

Pushed: 2026-04-13T22:21:37Z

Default branch: main

Fork: yes

Parent repository: ai-dynamo/nixl

Archived: no

README:

NVIDIA Inference Xfer Library (NIXL)

NVIDIA Inference Xfer Library (NIXL) is targeted for accelerating point to point communications in AI inference frameworks such as NVIDIA Dynamo, while providing an abstraction over various types of memory (e.g., CPU and GPU) and storage (e.g., file, block and object store) through a modular plug-in architecture.

Documentation and Resources

• NIXL overview - Core concepts/architecture overview (docs/nixl.md)

• Python API - Python API usage and examples (docs/python_api.md)

• Backend guide - Backend/plugin development guide (docs/BackendGuide.md)

• Telemetry - Observability and telemetry details (docs/telemetry.md)

• Doxygen guide - API/class diagrams overview (docs/doxygen/nixl_doxygen.md)

• Doxygen images - Diagram assets (docs/doxygen/)

• NIXLBench docs - Benchmark usage guide (benchmark/nixlbench/README.md)

• KVBench docs - KVBench workflows and tutorials (benchmark/kvbench/docs/)

Supported Platforms

NIXL is supported on a Linux environment only. It is tested on Ubuntu (22.04/24.04) and Fedora. macOS and Windows are not currently supported; use a Linux host or container/VM.

Pre-build Distributions

PyPI Wheel

The nixl python API and libraries, including UCX, are available directly through PyPI. For example, if you have a GPU running on a Linux host, container, or VM, you can do the following install:

It can be installed for CUDA 12 with:

pip install nixl[cu12]

For CUDA 13 with:

pip install nixl[cu13]

For backwards compatibility, pip install nixl installs automatically nixl[cu12], continuing to work seamlessly for CUDA 12 users without requiring changes to downstream project dependencies.

If both nixl-cu12 and nixl-cu13 are installed at the same time in an environment, nixl-cu13 takes precedence.

Prerequisites for source build (Linux)

Ubuntu:

$ sudo apt install build-essential cmake pkg-config

Fedora:

$ sudo dnf install gcc-c++ cmake pkg-config

Python

$ pip3 install meson ninja pybind11 tomlkit

UCX

NIXL was tested with UCX version 1.20.x.

GDRCopy is available on Github and is necessary for maximum performance, but UCX and NIXL will work without it.

$ git clone https://github.com/openucx/ucx.git
$ cd ucx
$ git checkout v1.20.x
$ ./autogen.sh
$ ./contrib/configure-release-mt \
--enable-shared \
--disable-static \
--disable-doxygen-doc \
--enable-optimizations \
--enable-cma \
--enable-devel-headers \
--with-cuda= \
--with-verbs \
--with-dm \
--with-gdrcopy=
$ make -j
$ make -j install-strip
$ ldconfig

ETCD (Optional)

NIXL can use ETCD for metadata distribution and coordination between nodes in distributed environments. To use ETCD with NIXL:

ETCD Server and Client

$ sudo apt install etcd etcd-server etcd-client

# Or use Docker
$ docker run -d -p 2379:2379 quay.io/coreos/etcd:v3.5.1

ETCD CPP API

Installed from https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3

$ sudo apt install libgrpc-dev libgrpc++-dev libprotobuf-dev protobuf-compiler-grpc
$ sudo apt install libcpprest-dev
$ git clone https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3.git
$ cd etcd-cpp-apiv3
$ mkdir build && cd build
$ cmake ..
$ make -j$(nproc) && make install

Additional plugins

Some plugins may have additional build requirements, see them here:

• [Mooncake](src/plugins/mooncake/README.md)
• [POSIX](src/plugins/posix/README.md)
• [GDS](src/plugins/cuda_gds/README.md)

Getting started

Build & install

$ meson setup
$ cd
$ ninja
$ ninja install

Build Options

Release build (default)

$ meson setup

Debug build

$ meson setup --buildtype=debug

NIXL-specific build options

# Example with custom options
$ meson setup \
-Dbuild_docs=true \ # Build Doxygen documentation
-Ducx_path=/path/to/ucx \ # Custom UCX installation path
-Dinstall_headers=true \ # Install development headers
-Ddisable_gds_backend=false # Enable GDS backend

Common build options:

• build_docs: Build Doxygen documentation (default: false)
• ucx_path: Path to UCX installation (default: system path)
• install_headers: Install development headers (default: true)
• disable_gds_backend: Disable GDS backend (default: false)
• cudapath_inc, cudapath_lib: Custom CUDA paths
• static_plugins: Comma-separated list of plugins to build statically
• enable_plugins: Comma-separated list of plugins to build (e.g. -Denable_plugins=UCX,POSIX). Cannot be used with disable_plugins.
• disable_plugins: Comma-separated list of plugins to exclude (e.g. -Ddisable_plugins=GDS). Cannot be used with enable_plugins.

Environment Variables

There are a few environment variables that can be set to configure the build:

• NIXL_NO_STUBS_FALLBACK: If not set or 0, build NIXL stub library if the library build fails

Building Documentation

If you have Doxygen installed, you can build the documentation:

# Configure with documentation enabled
$ meson setup -Dbuild_docs=true
$ cd
$ ninja

# Documentation will be generated in /html
# After installation (ninja install), documentation will be available in /share/doc/nixl/

Python Interface

NIXL provides Python bindings through pybind11. For detailed Python API documentation, see [docs/python_api.md](docs/python_api.md).

The preferred way to install the Python bindings is through pip from PyPI:

pip install nixl[cu12]

Or for CUDA 13 with:

pip install nixl[cu13]

Installation from source

Prerequisites:

• uv:…