basetenlabs/xgrammar

basetenlabs/xgrammar

Description: Fast, Flexible and Portable Structured Generation

License: Apache-2.0

Stars: 0

Forks: 0

Open issues: 4

Created: 2026-03-23T15:48:39Z

Pushed: 2026-03-27T04:12:09Z

Default branch: main

Fork: yes

Parent repository: mlc-ai/xgrammar

Archived: no

README:

News

• [2025/12] XGrammar has been officially integrated into Mirai
• [2025/09] XGrammar has been officially integrated into OpenVINO GenAI
• [2025/02] XGrammar has been officially integrated into Modular's MAX
• [2025/01] XGrammar has been officially integrated into TensorRT-LLM.
• [2024/12] XGrammar has been officially integrated into vLLM.
• [2024/12] We presented research talks on XGrammar at CMU, UC Berkeley, MIT, THU, SJTU, Ant Group, LMSys, Qingke AI, Camel AI. The slides can be found here.
• [2024/11] XGrammar has been officially integrated into SGLang.
• [2024/11] XGrammar has been officially integrated into MLC-LLM.
• [2024/11] We officially released XGrammar v0.1.0!

Overview

XGrammar is an open-source library for efficient, flexible, and portable structured generation.

It leverages constrained decoding to ensure 100% structural correctness of the output. It supports general context-free grammar to enable a broad range of structures, including JSON, regex, custom context-free grammar, etc.

XGrammar uses careful optimizations to achieve extremely low overhead in structured generation. It has achieved near-zero overhead in JSON generation, making it one of the fastest structured generation engines available.

XGrammar features universal deployment. It supports:

• Platforms: Linux, macOS, Windows
• Hardware: CPU, NVIDIA GPU, AMD GPU, Apple Silicon, TPU, etc.
• Languages: Python, C++, and JavaScript APIs
• Models: Qwen, Llama, DeepSeek, Phi, Gemma, etc.

XGrammar is very easy to integrate with LLM inference engines. It is the default structured generation backend for most LLM inference engines, including **vLLM**, **SGLang**, **TensorRT-LLM**, and **MLC-LLM**, as well as many other companies. You can also try out their structured generation modes!

Get Started

Install XGrammar:

pip install xgrammar

For use with MPS on Apple Silicon, install with:

pip install "xgrammar[metal]"

Import XGrammar:

import xgrammar as xgr

Please visit our documentation to get started with XGrammar.

• Installation
• Quick start

Third-Party Bindings

• Rust: xgrammar-rs — Community Rust bindings for XGrammar.

Collaborators

XGrammar has been widely adopted in industry, open-source projects, and academia. Our collaborators include:

Citation

If you find XGrammar useful in your research, please consider citing our paper:

@article{dong2024xgrammar,
title={Xgrammar: Flexible and efficient structured generation engine for large language models},
author={Dong, Yixin and Ruan, Charlie F and Cai, Yaxing and Lai, Ruihang and Xu, Ziyi and Zhao, Yilong and Chen, Tianqi},
journal={Proceedings of Machine Learning and Systems 7},
year={2024}
}