inclusionAI/Ming-3.0-audio-flash

inclusionAI/Ming-3.0-audio-flash

Language: Shell

License: MIT

Stars: 36

Forks: 1

Open issues: 0

Created: 2026-10-15T03:50:29Z

Pushed: 2026-10-19T16:54:28Z

Default branch: main

Fork: no

Archived: no

README:

---

https://github.com/user-attachments/assets/2accce07-c451-4132-9fc4-1ed375888a24

🧩 Model Variants

• [Ming-3.0-audio-flash](https://huggingface.co/inclusionAI/Ming-3.0-audio-flash) — A 126B-parameter Mixture-of-Experts model with 6.5B activated parameters. This is the model used by the examples and benchmark results below.
• [Ming-3.0-audio-tiny](https://huggingface.co/inclusionAI/Ming-3.0-audio-tiny) — A compact 9.3B-parameter Mixture-of-Experts model with 2.7B activated parameters. See its model card for checkpoint details and usage guidance.

✨ Highlights

• 🎙️ Complex speech recognition — Recognizes Chinese and English speech across dialects, accents, noisy environments, and multi-speaker meetings.
• ⏱️ Long-audio understanding — Comprehends and analyzes long-form recordings.
• 🎧 Multi-dimensional perception — Understands fine-grained speech, non-speech sounds, acoustic scenes, and music.
• 💡 Audio reasoning — Analyzes and reasons directly over audio content.
• 🗣️ Audio interaction — Supports end-to-end spoken dialogue with optional synthesized speech output.

Ming-3.0-audio-flash closes the loop from hearing the sound, to understanding the information, to getting the task done.

🏗️ Architecture

📊 Benchmarks

All scores are reported as percentages. ↑ means higher is better, while ↓ means lower is better. Ming-3.0-audio-flash is evaluated in non-thinking mode. Bold indicates the best result, underlined indicates the second-best result, and an asterisk (*) indicates results obtained in our own evaluation.

Speech Recognition


Audio Intelligence


Text Intelligence

> [!NOTE] > Results for comparison models are reproduced under our evaluation setup and may differ from their official reports due to differences in prompts, decoding settings, model versions, and evaluation pipelines.

🧭 Inference Options

| | vLLM-Omni | SGLang | Transformers | | --- | --- | --- | --- | | Best for | Serving and spoken interaction | High-throughput text serving | Local evaluation and research | | Input | Audio + text instruction | Audio + text instruction | Audio + text instruction | | Output | Text, or text + speech | Text | Text, or text + speech | | Interface | OpenAI-compatible API | OpenAI-compatible API | Hugging Face AutoClass | | Dependencies | requirements-vllm-omni.txt | requirements-sglang.txt | requirements-transformers.txt |

🚀 Quick Start

1️⃣ Download the Model

pip install -U huggingface_hub
hf download inclusionAI/Ming-3.0-audio-flash \
--local-dir inclusionAI/Ming-3.0-audio-flash

export MODEL_PATH="$PWD/inclusionAI/Ming-3.0-audio-flash"

Download from ModelScope

pip install -U modelscope
modelscope download --model inclusionAI/Ming-3.0-audio-flash \
--local_dir inclusionAI/Ming-3.0-audio-flash \
--revision master

export MODEL_PATH="$PWD/inclusionAI/Ming-3.0-audio-flash"

The model directory contains the tokenizer, processor, model implementation, checkpoint shards, and Talker assets. Large model files are distributed through Hugging Face and ModelScope rather than this GitHub repository.

2️⃣ Start a Server

vLLM-Omni

Install the Ming-3.0-audio-flash branch of inclusionAI/vllm-omni:

python3 -m venv .venv-vllm-omni
source .venv-vllm-omni/bin/activate
pip install -r requirements-vllm-omni.txt

This branch contains the bailing_moe_v3_audio pipeline and the current Ming Talker request interface. It supports IP and registered voice IDs through Chat Completions audio.voice, and voice cloning through ref_audio plus prompt_text.

Start text-output serving for audio understanding and reasoning:

MING_OUTPUT_MODE=text ./scripts/serve_vllm_omni.sh "$MODEL_PATH"

Use the ASR profile when reproducing recognition evaluations with 10-gram blocking support:

MING_OUTPUT_MODE=asr ./scripts/serve_vllm_omni.sh "$MODEL_PATH"

Or enable both text and synthesized speech output:

MING_OUTPUT_MODE=audio ./scripts/serve_vllm_omni.sh "$MODEL_PATH"

For Ming-3.0-audio-tiny, point MODEL_PATH to the Tiny checkpoint and select the Tiny deployment profiles:

export MODEL_PATH="$PWD/inclusionAI/Ming-3.0-audio-tiny"
MING_MODEL_VARIANT=tiny MING_OUTPUT_MODE=text ./scripts/serve_vllm_omni.sh "$MODEL_PATH"

scripts/serve_vllm_omni.sh also detects Tiny automatically when the model directory name contains tiny. Set MING_MODEL_VARIANT explicitly when using a custom directory name.

> [!NOTE] > The default Flash profiles use tensor parallelism across eight GPUs. This is required for the default 128K context length on 80 GB H800 GPUs because vLLM reserves KV-cache memory from the configured context length at startup. In the full-audio profile, the Talker reuses GPU 7 with a separate memory budget. The Tiny full-audio profile uses one GPU per stage. Adjust devices, context and sequence limits, and gpu_memory_utilization in deploy/*.yaml for your hardware.

SGLang

Install the audio branch of inclusionAI/sglang_ling_v3 in a separate environment:

python3 -m venv .venv-sglang
source .venv-sglang/bin/activate
pip install -r requirements-sglang.txt

When the sglang_ling_v3 source checkout is available next to this repository, install that exact checkout in editable mode instead:

pip install -e ../sglang_ling_v3/python

Start Ming-3.0-audio-flash with the validated four-GPU profile:

./scripts/serve_sglang.sh "$MODEL_PATH"

For Ming-3.0-audio-tiny, use its checkpoint and the single-GPU profile:

export MODEL_PATH="$PWD/inclusionAI/Ming-3.0-audio-tiny"
MING_MODEL_VARIANT=tiny CUDA_VISIBLE_DEVICES=0 \
./scripts/serve_sglang.sh "$MODEL_PATH"

The SGLang example currently covers audio input with text output. Use vLLM-Omni or direct Transformers inference when synthesized speech output is required.

3️⃣ Send a Request

vLLM-Omni

pip install -r...