openbmb/MiniCPM4.1-8B

GitHub Repo | Technical Report | Join Us

👋 Contact us in Discord and WeChat

What's New

• [2025.09.29] [InfLLM-V2 paper](https://arxiv.org/abs/2509.24663) is released! We can train a sparse attention model with only 5B long-text tokens. 🔥🔥🔥
• [2025.09.05] MiniCPM4.1 series are released! This series is a hybrid reasoning model with trainable sparse attention, which can be used in both deep reasoning mode and non-reasoning mode. 🔥🔥🔥
• [2025.06.06] MiniCPM4 series are released! This model achieves ultimate efficiency improvements while maintaining optimal performance at the same scale! It can achieve over 5x generation acceleration on typical end-side chips! You can find technical report here.🔥🔥🔥

Highlights

MiniCPM4.1 is highlighted with following features:

✅ Strong Reasoning Capability: Surpasses similar-sized models on 15 tasks!

✅ Fast Generation: 3x decoding speedup for reasoning!

✅ Efficient Architecture: Trainable sparse attention, frequency-ranked speculative decoding!

• MiniCPM4.1-8B: The latest version of MiniCPM4, with 8B parameters, support fusion thinking. (**

Click to expand all MiniCPM4 series models

• **MiniCPM4-8B**: The flagship model with 8B parameters, trained on 8T tokens
• **MiniCPM4-0.5B**: Lightweight version with 0.5B parameters, trained on 1T tokens
• **MiniCPM4-8B-Eagle-FRSpec**: Eagle head for FRSpec, accelerating speculative inference
• **MiniCPM4-8B-Eagle-FRSpec-QAT-cpmcu**: Eagle head with QAT for FRSpec, integrating speculation and quantization for ultra acceleration
• **MiniCPM4-8B-Eagle-vLLM**: Eagle head in vLLM format for speculative inference
• **MiniCPM4-8B-marlin-Eagle-vLLM**: Quantized Eagle head for vLLM format
• **BitCPM4-0.5B**: Extreme ternary quantization of MiniCPM4-0.5B, achieving 90% bit width reduction
• **BitCPM4-1B**: Extreme ternary quantization of MiniCPM3-1B, achieving 90% bit width reduction
• **MiniCPM4-Survey**: Generates trustworthy, long-form survey papers from user queries
• **MiniCPM4-MCP**: Integrates MCP tools to autonomously satisfy user requirements

Evaluation Results

Performance Evaluation

MiniCPM4.1 launches end-side versions with 8B parameter scale, both achieving best-in-class performance in their respective categories.

!benchmark

Best Practices

1. It is advisable to use temperature=0.9, topp=0.95. And we suggest setting max_output_token to 65,536 tokens. 2. For math problems, we recommend using "Please reason step by step, and put your final answer within \boxed{}." 3. And for English multiple-choice questions, we recommend starting with "Answer the following multiple choice question. The last line of your response should be of the following format: 'ANSWER: $LETTER' (without quotes) where LETTER is one of ABCD. Think step by step before answering." And "你回答的最后一行必须是以下格式 '答案：$选项' (不带引号), 其中选项是ABCD之一。请在回答之前一步步思考" for Chinese MCQ.

Efficiency Evaluation

MiniCPM4.1 adopts sparse attention and speculative decoding to improve the inference efficiency. On RTX 4090, MiniCPM4.1 achieves 3x decoding speed improvement in reasoning.

!benchmark

Examples

Usage

MiniCPM 4.1 can be used with following frameworks: Huggingface Transformers, SGLang, vLLM, and CPM.cu. For the ultimate inference speed, we highly recommend CPM.cu.

MiniCPM4/MiniCPM4.1 supports both dense attention inference and sparse attention inference modes, where vLLM and SGLang currently only support dense inference mode. If you want to use sparse inference mode, please use Huggingface Transformers and CPM.cu.

• Dense attention inference: vLLM, SGLang, Huggingface Transformers
• Sparse attention inference: Huggingface Transformers, CPM.cu

To facilitate researches in sparse attention, we provide [InfLLM-V2 Training Kernels](https://github.com/OpenBMB/infllmv2_cuda_impl) and [InfLLM-V2 Inference Kernels](https://github.com/openbmb/cpm.cu).

Inference with Transformers

MiniCPM4.1-8B requires transformers>=4.56.

• Inference with Dense Attention

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
torch.manual_seed(0)

path = 'openbmb/MiniCPM4.1-8B'
device = "cuda"
tokenizer = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path, torch_dtype=torch.bfloat16, device_map=device, trust_remote_code=True)

# User can directly use the chat interface
# responds, history = model.chat(tokenizer, "Write an article about Artificial Intelligence.", temperature=0.7, top_p=0.7)
# print(responds)

# User can also use the generate interface
messages = [
{"role": "user", "content": "Write an article about Artificial Intelligence."},
]
prompt_text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
model_inputs = tokenizer([prompt_text], return_tensors="pt").to(device)

model_outputs = model.generate(
**model_inputs,
max_new_tokens=32768,
top_p=0.95,
temperature=0.6
)
output_token_ids = [
model_outputs[i][len(model_inputs[i]):] for i in range(len(model_inputs['input_ids']))
]

responses = tokenizer.batch_decode(output_token_ids, skip_special_tokens=True)[0]
print(responses)

• Inference with Sparse Attention

MiniCPM4.1-8B supports InfLLM v2, a sparse attention mechanism designed for efficient long-sequence inference. It requires the infllmv2_cuda_impl library.

You can install it by running the following command:

git clone -b feature_infer https://github.com/OpenBMB/infllmv2_cuda_impl.git
cd infllmv2_cuda_impl
git submodule update --init --recursive
pip install -e . # or python setup.py install

To enable InfLLM v2, you need to add the…