siliconflow/SenseVoice-1

siliconflow/SenseVoice-1

Description: Open-source SenseVoiceSmall model for Mandarin, Cantonese, English, Japanese, and Korean ASR, language ID, emotion recognition, and audio event detection.

License: MIT

Stars: 0

Forks: 0

Open issues: 0

Created: 2026-08-24T07:55:06Z

Pushed: 2026-08-18T02:13:34Z

Default branch: main

Fork: yes

Parent repository: QwenAudio/SenseVoice

Archived: no

README: ([简体中文](./README_zh.md)|English|[日本語](./README_ja.md))

Introduction

SenseVoice is a speech foundation model with multiple speech understanding capabilities, including automatic speech recognition (ASR), spoken language identification (LID), speech emotion recognition (SER), and audio event detection (AED).

[//]: # ()

> Released checkpoint scope: SenseVoiceSmall supports ASR and language ID for Mandarin, Cantonese, English, Japanese, and Korean, together with emotion and audio-event tags. Speaker diarization is a composed FunASR pipeline using separate FSMN-VAD and CAM++ models; it is not an output of the SenseVoiceSmall checkpoint itself.

Highlights 🎯

SenseVoice focuses on high-accuracy multilingual speech recognition, speech emotion recognition, and audio event detection.

• Research scope vs released checkpoint: The broader SenseVoice work reports training on more than 400,000 hours and support for more than 50 languages. The released SenseVoiceSmall checkpoint linked above supports Mandarin, Cantonese, English, Japanese, and Korean; the benchmark comparisons below are task- and language-specific.
• Rich transcribe:
• Possess excellent emotion recognition capabilities, achieving and surpassing the effectiveness of the current best emotion recognition models on test data.
• Offer sound event detection capabilities, supporting the detection of various common human-computer interaction events such as bgm, applause, laughter, crying, coughing, and sneezing.
• Efficient Inference: SenseVoiceSmall uses a non-autoregressive end-to-end framework for low-latency inference; see the benchmark setup and comparison below.
• Convenient Finetuning: Provide convenient finetuning scripts and strategies, allowing users to easily address long-tail sample issues according to their business scenarios.
• Service Deployment: Offer service deployment pipeline, supporting multi-concurrent requests, with client-side languages including Python, C++, HTML, Java, and C#, among others.

What's New 🔥

• 2026/07: FunASR 1.3.29 restores SenseVoice VAD segment timestamps — when token timestamps and a punctuation model are unavailable, sentence_timestamp=True now returns every VAD region through sentence_info, so subtitle and clipping clients receive usable segment boundaries instead of an empty timeline. Install with pip install -U "funasr==1.3.29". Release notes · PyPI
• 2026/07: FunASR 1.3.27 adds detected-language metadata for SenseVoice — the OpenAI-compatible endpoint now reports detected zh, en, yue, ja, or ko in verbose_json.language. Install with pip install -U "funasr==1.3.27". Release notes · API guide · PyPI
• 2026/06: SenseVoice on llama.cpp / GGUF — run it on CPU/edge as a single self-contained binary (whisper.cpp-style), built-in VAD, no Python at runtime. The q8 model is only ~254 MB with the same accuracy. [runtime/llama.cpp/](./runtime/llama.cpp/) · Releases · GGUF on Hugging Face
• 2026/05: FunASR can compose SenseVoiceSmall with separate FSMN-VAD, CAM++, and punctuation models to produce per-sentence speaker labels. Diarization is not a native SenseVoiceSmall checkpoint output. Requires installing FunASR from source: pip install git+https://github.com/modelscope/FunASR.git
• 2024/11: Add support for timestamp based on the CTC alignment.
• 2024/7: Added Export Features for [ONNX](./demo_onnx.py) and [libtorch](./demo_libtorch.py), as well as Python Version Runtimes: funasr-onnx-0.4.0, funasr-torch-0.1.1
• 2024/7: The SenseVoice-Small voice understanding model is open-sourced, which offers high-precision multilingual speech recognition, emotion recognition, and audio event detection capabilities for Mandarin, Cantonese, English, Japanese, and Korean and leads to exceptionally low inference latency.
• 2024/7: The CosyVoice for natural speech generation with multi-language, timbre, and emotion control. CosyVoice excels in multi-lingual voice generation, zero-shot voice generation, cross-lingual voice cloning, and instruction-following capabilities. CosyVoice repo and CosyVoice space.
• 2024/7: FunASR is a fundamental speech recognition toolkit that offers a variety of features, including speech recognition (ASR), Voice Activity Detection (VAD), Punctuation Restoration, Language Models, Speaker Verification, Speaker Diarization and multi-talker ASR.

Benchmarks 📝

Multilingual Speech Recognition

We compared the performance of multilingual speech recognition between SenseVoice and Whisper on open-source benchmark datasets, including AISHELL-1, AISHELL-2, Wenetspeech, LibriSpeech, and Common Voice. In terms of Chinese and Cantonese recognition, the SenseVoice-Small model has advantages.

Speech Emotion Recognition

Due to the current lack of widely-used benchmarks and methods for speech emotion recognition, we conducted evaluations across various metrics on multiple test sets and performed a comprehensive comparison with numerous results from recent benchmarks. The selected test sets encompass data in both Chinese and English, and include multiple styles such as performances, films, and natural conversations. Without finetuning on the target data, SenseVoice was able to achieve and exceed the performance of the current best speech emotion recognition models.

Furthermore, we compared multiple open-source speech emotion recognition models on the...