ReleaseAI21 LabsAI21 Labspublished Aug 21, 2024seen 5d

AI21Labs/ai21-tokenizer v0.12.0

AI21Labs/ai21-tokenizer

Open original ↗

Captured source

source ↗
GH
GitHub/github.com/AI21Labs/ai21-tokenizer
AI21Labs/ai21-tokenizer v0.12.0
Source ↗
published Aug 21, 2024seen 5dcaptured 14hhttp 200method plain

v0.12.0

Repository: AI21Labs/ai21-tokenizer

Tag: v0.12.0

Published: 2024-08-21T11:41:27Z

Prerelease: no

Release notes:

v0.12.0 (2024-08-21)

Feature

  • feat: add jamba 1.5 tokenizers (mini and large) (#96)
  • feat: add jamba 1.5 tokenizers (mini and large)
  • test: fix value error on conftest.py
  • refactor: rename file name
  • chore: fix path

Notability

notability 3.0/10

Routine tokenizer release, low impact.