Lightning-AI/LitServe v0.2.17

Release v0.2.17

Repository: Lightning-AI/LitServe

Tag: v0.2.17

Published: 2025-12-23T19:39:23Z

Prerelease: no

Release notes: Lightning AI ⚡ is excited to announce the release of LitServe v0.2.17

Highlights

Automatic Worker Restart

LitServe now supports automatic restarting of inference workers when they die, ensuring high availability and resilience in production environments. This prevents server shutdown due to isolated worker failures and maintains service continuity.

import litserve as ls

server = ls.LitServer(
MyAPI(),
restart_workers=True, # Automatically restart failed workers
workers_per_device=4
)
server.run()

When a worker terminates unexpectedly, the server automatically spawns a replacement, keeping requests flowing without interruption.

Changes

Added

• Add support for restarting the inference worker when they die by @tchaton in https://github.com/Lightning-AI/LitServe/pull/624

Changed

• Update README to reflect inference engines terminology by @williamFalcon in https://github.com/Lightning-AI/LitServe/pull/625
• chore: drop support for Python 3.9 by @bhimrazy in https://github.com/Lightning-AI/LitServe/pull/641

Fixed

• Add warning for dict/set outputs in batched predict to catch edge cases by @Copilot in https://github.com/Lightning-AI/LitServe/pull/612
• fix(sdk): Reduce the quantity of warning emitted by @tchaton in https://github.com/Lightning-AI/LitServe/pull/631
• fix(litServe): Use asyncio.sleep instead of time.sleep by @tchaton in https://github.com/Lightning-AI/LitServe/pull/633
• fix(cli): lightning-sdk installation process with uv by @bhimrazy in https://github.com/Lightning-AI/LitServe/pull/640
• Fix on_request callback not triggering for API specs by @bhimrazy in https://github.com/Lightning-AI/LitServe/pull/642

Chores

• Bump the gha-updates group with 2 updates by @dependabot[bot] in https://github.com/Lightning-AI/LitServe/pull/626
• [pre-commit.ci] pre-commit suggestions by @pre-commit-ci[bot] in https://github.com/Lightning-AI/LitServe/pull/629
• feat(litServe): Bump version 0.2.17 by @tchaton in https://github.com/Lightning-AI/LitServe/pull/632
• Bump actions/checkout from 5 to 6 in the gha-updates group by @dependabot[bot] in https://github.com/Lightning-AI/LitServe/pull/636
• [pre-commit.ci] pre-commit suggestions by @pre-commit-ci[bot] in https://github.com/Lightning-AI/LitServe/pull/637
• Bump mypy from 1.18.2 to 1.19.0 by @dependabot[bot] in https://github.com/Lightning-AI/LitServe/pull/639

🧑‍💻 Contributors

Thank you ❤️ to all contributors for making LitServe better!

Full Changelog: https://github.com/Lightning-AI/LitServe/compare/v0.2.16...v0.2.17