ByteDance-Seed/VeOmni v0.1.3

v0.1.3

Repository: ByteDance-Seed/VeOmni

Tag: v0.1.3

Published: 2025-11-07T07:09:28Z

Prerelease: yes

Release notes:

Highlights

• Qwen3VL (both dense and MoE) series support by @Juntian777
• DeepSeek performance restoration by @Luosuu

What's Changed

• [dist] feat: enable EP-aware optimizer for FSDP2-based MoE-VLM training. by @Juntian777 in https://github.com/ByteDance-Seed/VeOmni/pull/145
• [model] enable deepseek ulysses and fix deepseek transpose by @Luosuu in https://github.com/ByteDance-Seed/VeOmni/pull/152
• [dist] fix: add alltoall async by @heidongxianhua in https://github.com/ByteDance-Seed/VeOmni/pull/146
• [ckpt] fix: merge ckpt to hf script by @Luosuu in https://github.com/ByteDance-Seed/VeOmni/pull/156
• [model] fix: remove npu flash attention sync by @wey-code in https://github.com/ByteDance-Seed/VeOmni/pull/154
• [model] feat: support qwen3-vl dense by @Juntian777 in https://github.com/ByteDance-Seed/VeOmni/pull/164
• [logging] fix: the logging lineno of log_rank0 by @ValMystletainn in https://github.com/ByteDance-Seed/VeOmni/pull/160
• fix: repeat kv bug in flash attention forward with ulysses by @HaoyiZhu in https://github.com/ByteDance-Seed/VeOmni/pull/162
• fix: correct code formatting for PR162 by @Juntian777 in https://github.com/ByteDance-Seed/VeOmni/pull/165
• [model] perf: eliminate per-layer CPU-GPU sync in Qwen3-VL vision attention by @Juntian777 in https://github.com/ByteDance-Seed/VeOmni/pull/169
• [model] feat: add dummy forward for video input by @Juntian777 in https://github.com/ByteDance-Seed/VeOmni/pull/177
• perf: set reshard_after_forward to False for modules without MixedPrecision by @Luosuu in https://github.com/ByteDance-Seed/VeOmni/pull/153
• [doc] feat: how to enable new models in veomni by @Juntian777 in https://github.com/ByteDance-Seed/VeOmni/pull/179
• [model] feat: support qwen3 vl moe by @Juntian777 in https://github.com/ByteDance-Seed/VeOmni/pull/178
• [doc] feat: update README with new support for Qwen3-VL and Qwen3-VL-MoE by @Juntian777 in https://github.com/ByteDance-Seed/VeOmni/pull/180
• fix: workaround duplicated AllGather for EP+FSDP2 by @Luosuu in https://github.com/ByteDance-Seed/VeOmni/pull/173

New Contributors

• @wey-code made their first contribution in https://github.com/ByteDance-Seed/VeOmni/pull/154
• @ValMystletainn made their first contribution in https://github.com/ByteDance-Seed/VeOmni/pull/160
• @HaoyiZhu made their first contribution in https://github.com/ByteDance-Seed/VeOmni/pull/162

Full Changelog: https://github.com/ByteDance-Seed/VeOmni/compare/v0.1.2...v0.1.3