togethercomputer/xorl-wheels sgl_fused_moe_trtllm_sm100_fi0617_cu130_sm100a_92b23b9

xorl-sglang experimental_sgl_trtllm MoE-LoRA JIT cache (flashinfer 0.6.17 / CUDA 13.0 / sm100a)

Repository: togethercomputer/xorl-wheels

Tag: sgl_fused_moe_trtllm_sm100_fi0617_cu130_sm100a_92b23b9

Published: 2026-08-27T06:30:12Z

Prerelease: no

Release notes: Pre-compiled sgl_fused_moe_trtllm_sm100 — xorl-sglang's vendored TRT-LLM fused-MoE + LoRA op — packaged in flashinfer's AOT layout so the ~15-minute first-run JIT compile is skipped entirely (flashinfer 0.6.17 loads FLASHINFER_AOT_DIR//.so directly, no ninja involved; verified: 0.0s load with an empty JIT cache).

Contents

sgl_fused_moe_trtllm_sm100/sgl_fused_moe_trtllm_sm100.so (+ METADATA.txt)

Built against

| | | | --- | --- | | source | xorl-sglang dev @ 978fbbd43 (csrc tree python/sglang/kernels/ops/moe/trtllm_lora_temp/data @ 92b23b9) | | flashinfer | 0.6.17 (flashinfer-python 0.6.17, flashinfer-jit-cache 0.6.17+cu130) | | torch | 2.11.0+cu130 | | CUDA toolkit | 13.0 | | arch | sm100a (B200 family) |

Install

AOT_DIR=$(python -c 'from flashinfer.jit import env; print(env.FLASHINFER_AOT_DIR)')
curl -L https://github.com/togethercomputer/xorl-wheels/releases/download/sgl_fused_moe_trtllm_sm100_fi0617_cu130_sm100a_92b23b9/sgl_fused_moe_trtllm_sm100_fi0617_cu130_sm100a_92b23b9.tar.gz | tar -xz -C "$AOT_DIR"

Validity

The .so freezes the vendored csrc at tree hash 92b23b9 — re-release whenever `python/sglang/kernels/ops/moe/trtllm_lora_temp/data` changes or the flashinfer pin moves (check with git rev-parse HEAD:python/sglang/kernels/ops/moe/trtllm_lora_temp/data). A stale AOT .so is loaded silently in preference to a fresh JIT build; delete the op's directory from $AOT_DIR to fall back to JIT.

First consumer: test/registered/lora/test_lora_qwen3_5_35b_a3b_nvfp4_password.py (xorl-sglang #47) — 1571s cold-JIT → ~91s with this artifact.