microsoft/VibeASR.cpp

microsoft/VibeASR.cpp

Language: C

License: MIT

Stars: 33

Forks: 5

Open issues: 3

Created: 2026-07-23T06:09:54Z

Pushed: 2026-07-27T06:31:21Z

Default branch: main

Fork: no

Archived: no

README: VibeASR.cpp

---

VibeASR.cpp is the official inference runtime for VibeVoice-ASR-BitNet — enabling real-time multilingual speech recognition on CPU through heterogeneous quantization (I8\_S for VAE + I2\_S for LM).

To enable efficient edge CPU deployment, we replace the original Qwen2.5-7B language model with Qwen2.5-1.5B, achieving only modest accuracy degradation (1–4% absolute WER increase) while reducing the total model size from 4.62 GB to 1.58 GB. Combined with custom SIMD kernels and operator fusion in the ggml framework, VibeVoice-ASR-BitNet achieves 1.6–2.3× faster inference than Whisper.cpp at comparable model sizes, with real-time capability (RTF

📄 Tech Report | 🤗 Models | 🏠 GeneralAI

---

Key Results

Model Size

Inference Performance

> RTF (Real-Time Factor) on 20s audio input. Bold = RTF ⚠️ Architecture Note: The LM (I2\_S) GGUF uses platform-specific bit-packing layouts — ARM (NEON) and x86 (AVX) versions are not interchangeable. The VAE (I8\_S) GGUF is architecture-independent. > > When downloading from HuggingFace: > - x86 (Intel/AMD): use vibeasr-lm-i2_s-embed-q6_k.gguf > - ARM (Apple Silicon / aarch64): use vibeasr-lm-i2_s-embed-q6_k_arm.gguf > > If you build from source with llama-quantize, the output GGUF will automatically match your host architecture.

CLI Inference

# x86 (Intel / AMD)
./build/bin/asr_infer \
--vae-model models/vibeasr/vibeasr-vae-encoder-i8_s.gguf \
--lm-model models/vibeasr/vibeasr-lm-i2_s-embed-q6_k.gguf \
--audio input.wav -t 4

# ARM (Apple Silicon / aarch64)
./build/bin/asr_infer \
--vae-model models/vibeasr/vibeasr-vae-encoder-i8_s.gguf \
--lm-model models/vibeasr/vibeasr-lm-i2_s-embed-q6_k_arm.gguf \
--audio input.wav -t 4

Web Demo (Gradio)

pip install gradio soundfile numpy

# Use the LM GGUF matching your architecture (see note above)
python demo/gradio_asr_demo.py --port 7860 \
--vae-model models/vibeasr/vibeasr-vae-encoder-i8_s.gguf \
--lm-model models/vibeasr/vibeasr-lm-i2_s-embed-q6_k.gguf # or *_arm.gguf on ARM

---

Model Conversion

For most users, downloading pre-quantized models from HuggingFace is recommended. To convert from SafeTensors yourself:

Step 1: SafeTensors → F32 GGUF

# LM (BitNet) — handles weight preprocessing and config flattening automatically
python utils/convert_lm_to_gguf.py

# VAE Tokenizer
python utils/convert_vae_to_gguf.py

Step 2: F32 GGUF → Quantized GGUF

# VAE Tokenizer: F32 → I8_S
./build/bin/llama-quantize \
/vibeasr-vae-encoder-f32.gguf \
/vibeasr-vae-encoder-i8_s.gguf \
I8_S 1 1

# LM: F32 → I2_S (with Q6_K embeddings)
./build/bin/llama-quantize --token-embedding-type Q6_K \
/vibeasr-lm-f32.gguf \
/vibeasr-lm-i2_s-embed-q6_k.gguf \
I2_S 1 1

---

Citation

@article{xu2025vibeasrbitnet,
title={VibeVoice-ASR-BitNet Technical Report},
author={Xu, Songchen and Song, Ting and Huang, Shaohan and Peng, Zhiliang and Xia, Yan and Tu, Yujie and Huang, Xin and Yu, Jianwei and Dong, Li and Wei, Furu},
journal={arXiv preprint arXiv:2607.21075},
year={2025}
}

---

License

This project is licensed under the [MIT License](LICENSE).