inclusionAI/Ming-3.0-asr

inclusionAI/Ming-3.0-asr

Language: Python

License: MIT

Stars: 17

Forks: 0

Open issues: 0

Created: 2026-10-19T02:38:13Z

Pushed: 2026-10-19T14:04:25Z

Default branch: main

Fork: no

Archived: no

README:

Ming-3.0-asr-tiny: Context-Aware Speech Recognition

Chinese and English · Dialects · Hotwords and Dialogue Context

Hugging Face · ModelScope · Project Blog · Quick Start · Cookbook · Evaluation

Beyond Word-by-Word Transcription

Real speech comes with accents, background noise, unfamiliar names, and specialized terminology. Ming-3.0-asr-tiny combines the Ling-tiny-v3 language model with an audio encoder for Chinese and English transcription, dialect recognition, and Context ASR. Hotwords or preceding dialogue can help identify domain-specific expressions.

• Chinese and English ASR: transcribe single-turn speech through one Transformers interface.
• Dialect recognition: evaluated on 15 test sets covering Hunan, Minnan, Cantonese, Chuanyu, Shanghai, Anhui, Northeast, Henan, Hubei, Jiangsu, Hakka, Shaanxi, Shandong, Tianjin, and Yunnan.
• Context ASR: include relevant names, places, products, or technical terms in the transcription prompt.
• Challenging speech: evaluation covers chat, government, health, knowledge, local-life, and high-noise scenarios.

Introduction Video

https://github.com/user-attachments/assets/11ca7c5f-f827-4ed4-ab57-b4315f410cf5

Watch the release film

Architecture

Evaluation

ASR Overview

Chinese and English ASR

Dialect and Domain Recognition

Context ASR

Quick Start

1. Create an Environment

Use Python 3.12 on Linux with a CUDA GPU. The Transformers path uses the model package's pinned dependencies; SGLang uses a separate environment in Step 5.

git clone https://github.com/inclusionAI/Ming-3.0-asr.git
cd Ming-3.0-asr
python3.12 -m venv .venv
source .venv/bin/activate
unset PYTHONPATH

The inference stack uses Transformers 5.12.1, fla-core 0.5.2, and PyTorch 2.11.0. For a different CUDA version, install matching PyTorch and torchaudio 2.11.0 builds using the official PyTorch instructions.

2. Download the Model

The model weights, tokenizer, configuration, and custom Transformers implementation are distributed together on Hugging Face:

python -m pip install "huggingface_hub[cli]"
hf download inclusionAI/Ming-3.0-asr-tiny --local-dir ./Ming-3.0-asr-tiny-model
export MODEL_DIR="$(cd ./Ming-3.0-asr-tiny-model && pwd)"
python -m pip install -r "$MODEL_DIR/requirements.txt"
python tools/verify_repository.py --checkpoint "$MODEL_DIR"

3. Transcribe an Audio Clip

python examples/transcribe.py \
--model "$MODEL_DIR" \
--audio examples/cases/audio/chinese_common_voice.wav \
--output outputs/transcript.json

Sample output: 漓江和湘江有秦代开凿的灵渠沟通. Replace --audio with your own WAV file. Ordinary ASR and Context ASR use the same audio input format. The default path uses CUDA, BF16, SDPA, greedy decoding, a 10-gram repetition limit, and up to 256 new tokens. For topic and hotword prompting, see the [Context ASR guide](cookbook/context-asr.en.md).

4. Native PyTorch / Transformers Inference

The following is the core call used by the [runnable example](examples/transcribe.py). Run it from the repository root:

import os

import soundfile as sf
import torch
from transformers import AutoModelForCausalLM, AutoProcessor
from examples.hf_runtime import decode_transcript, prepare_asr_request

checkpoint = os.environ["MODEL_DIR"]
processor = AutoProcessor.from_pretrained(checkpoint, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
checkpoint, trust_remote_code=True, dtype=torch.bfloat16,
device_map="cuda:0", attn_implementation="sdpa",
).eval()
waveform, sample_rate = sf.read(
"examples/cases/audio/chinese_common_voice.wav", dtype="float32", always_2d=True,
)
inputs = prepare_asr_request(processor, (waveform.T, sample_rate)).to(model.device)

with torch.inference_mode():
eos = model.config.text_config.eos_token_id
output = model.generate(
**inputs, max_new_tokens=256, do_sample=False,
eos_token_id=eos, pad_token_id=eos, no_repeat_ngram_size=10,
)

print(decode_transcript(
processor.tokenizer, output[0, inputs.input_ids.shape[1]:].tolist(),
))

5. SGLang Accelerated Serving

Use the Ming SGLang adapter branch at the pinned commit below. This repository includes the model-class alias patch and serving examples.

Use a separate Python 3.12 environment for SGLang. Keep the model downloaded in Step 2, then run these commands from the repository root:

python3.12 -m venv .venv-sglang
source .venv-sglang/bin/activate
unset PYTHONPATH
python -m pip install -r cookbook/sglang/requirements.txt
python -m pip check
git clone --single-branch --branch ling_v3_audio_for_pr \
https://github.com/inclusionAI/sglang_ling_v3.git ../sglang_ming_asr
export SGLANG_DIR="$(cd ../sglang_ming_asr && pwd)"
git -C "$SGLANG_DIR" checkout bba862771a9e3413a062dad33e8b494ea9f7755b
export GUIDE_DIR="$(pwd)/cookbook/sglang"
export PYTHON_BIN="$(command -v python)"
SGLANG_DIR="$SGLANG_DIR" bash "$GUIDE_DIR/scripts/apply_patch.sh"
export PYTHONPATH="$SGLANG_DIR/python:$MODEL_DIR"
"$PYTHON_BIN" "$GUIDE_DIR/scripts/check_env.py" \
--model "$MODEL_DIR" --sglang-dir "$SGLANG_DIR"

Start the server in this terminal and run the client in another terminal after /health returns HTTP 200:

export GPU_ID=0 PORT=8000 HOST=127.0.0.1 SERVED_MODEL=ming-asr
PYTHON_BIN="$PYTHON_BIN" MODEL_DIR="$MODEL_DIR" SGLANG_DIR="$SGLANG_DIR" \
bash "$GUIDE_DIR/scripts/start_server.sh"

curl -fsS http://127.0.0.1:8000/health
python cookbook/sglang/scripts/transcribe.py \
--audio examples/cases/audio/chinese_common_voice.wav \
--base-url http://127.0.0.1:8000 --model ming-asr

The sample transcript is 漓江和湘江有秦代开凿的灵渠沟通. See the [complete SGLang guide](cookbook/sglang/README.en.md) for remote serving, English audio, and dataset evaluation settings.

Cookbook: From Transcription to Context

Open each WAV sample to listen, then compare the reference with the model output. Punctuation is removed, and English is lowercased for display.

###...