NVIDIA/NeMo-Speech.cpp

NVIDIA/NeMo-Speech.cpp

Description: NeMo-Speech.cpp is a GGML optimized C++ inference solution for Speech models

Language: C++

License: Apache-2.0

Stars: 7

Forks: 2

Open issues: 1

Created: 2026-07-15T10:46:56Z

Pushed: 2026-08-06T06:57:23Z

Default branch: main

Fork: no

Archived: no

README:

NeMo-Speech.cpp

A lightweight native C++ runtime for NVIDIA Nemotron Speech models built on ggml. Runs speech models in realtime and in batch mode across platforms/backends.

Contents

• [Installation](#installation)
• [Quick start](#quick-start)
• [Command line](#command-line)
• [Local server and playground](#local-server-and-playground)
• [Native SDK](#native-sdk)
• [Build from source](#build-from-source)
• [Documentation](#documentation)
• [License](#license)
• [Contributing](#contributing)

Installation

From a source checkout, install the CLI, HTTP API, and browser playground for the detected platform and backend:

scripts/install.sh --source
export PATH="$HOME/.local/bin:$PATH" # current shell; future shells are updated

The source build requires Git, CMake 3.26 or newer, Ninja, a C++17 compiler, and the toolkit for the selected GPU backend. See [Installation](docs/install.md) for platform-specific prerequisites and options. The same installer will support native release archives once their public URL is configured.

Quick start

The runtime consumes GGUF models. Until the preconverted GGUF is published, use the included converter to download the public .nemo checkpoint and produce a portable Q8 model. Complete the one-time [conversion setup](docs/model-conversion.md) first; it does not install NeMo.

python3 convert_model.py nvidia/nemotron-speech-streaming-en-0.6b \
--outfile nemotron-speech-streaming-en-0.6b.q8_0.gguf

nemo-speech transcribe test_files/asr/wav/test/jfk.wav \
--model nemotron-speech-streaming-en-0.6b.q8_0.gguf

The converter downloads only the .nemo checkpoint through the standard Hugging Face cache. The CLI selects an available backend and handles common mono or stereo PCM WAV sample rates automatically. Substitute your own WAV file after verifying the bundled sample.

Command line

The CLI is the primary interface. Run nemo-speech --help to see the capabilities included in your build. The [CLI guide](docs/cli.md) covers model selection, GPU controls, directory transcription, subtitles, diarization, translation, synthesis, structured output, and benchmarking when you need them.

Local server and playground

Start the same runtime as a local HTTP service and open the playground:

nemo-speech serve \
--asr-model nemotron-speech-streaming-en-0.6b.q8_0.gguf \
--open

The server binds to by default and also provides a documented OpenAI-compatible audio API subset and realtime WebSocket transcription. A separately built riva_server binary provides the Riva-compatible gRPC interface. See the [server guide](docs/server.md) when you are ready to integrate either interface.

Native SDK

Release archives include stable C headers, shared libraries, and an exported CMake package. An installed application can link only the capability it uses:

find_package(NeMoSpeech REQUIRED COMPONENTS ASR)
target_link_libraries(my_app PRIVATE NeMoSpeech::ASR)

See [native SDK integration](docs/sdk.md) for in-process C/C++ usage, or [client integration](docs/clients.md) for OpenAI SDK, curl, and Riva-compatible gRPC usage.

Build from source

For a CUDA ASR and TTS server with the playground from an initialized checkout:

Requires CMake 3.26 or newer, Ninja, C and C++17 compilers, and a supported CUDA toolkit.

git submodule update --init ggml third_party/cpp-httplib
scripts/configure.sh cuda-server
cmake --build --preset cuda-server

The configuration helper validates required submodules and applies the pinned ggml patch series for CUDA builds. CPU, Metal, Vulkan, server, component, Windows, and container instructions are in [Build from source](docs/build.md).

Documentation

| Start here | What it covers | |---|---| | [Installation](docs/install.md) | Native releases, Windows, upgrades, and manual verification | | [CLI guide](docs/cli.md) | Transcription, subtitles, directories, diarization, NMT, TTS, and tooling | | [Model conversion](docs/model-conversion.md) | Convert NeMo and Hugging Face checkpoints to runtime GGUF files | | [Servers](docs/server.md) | HTTP playground/realtime serving and the separate Riva-compatible gRPC server | | [Native SDK](docs/sdk.md) | CMake components, C ABI lifetimes, threading, and examples | | [Client integration](docs/clients.md) | OpenAI SDKs, curl, and Riva gRPC clients | | [Troubleshooting](docs/troubleshooting.md) | doctor output and common runtime failures | | [Build from source](docs/build.md) | Presets, optional components, dependencies, containers, and artifacts | | [All documentation](docs/README.md) | ASR, TTS, NMT, configuration, and developer references |

License

NVIDIA-authored code is released under the [Apache License 2.0](LICENSE), with the project copyright notice in [NOTICE](NOTICE). Third-party components retain their respective terms; see [Third-Party Notices](THIRD_PARTY_NOTICES.md).

Contributing

External contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for the contribution terms and Developer Certificate of Origin sign-off process.