Julien DenizeandGitHub fef56c1855 [Mistral Grammar] Support Grammar Factory (#38150) ...

Signed-off-by: juliendenize <julien.denize@mistral.ai>

2026-04-06 10:28:51 -04:00

..

__init__.py

[Misc] Refactor tokenizer interface (#29693)

2025-11-29 04:02:21 -08:00

test_basic.py

[Bugfix] Fix Qwen-VL tokenizer implementation (#36140)

2026-03-05 08:07:19 -08:00

test_detokenize.py

[Refactor] Simplify BOS/EOS token handling (#34435)

2026-02-12 18:18:24 -08:00

test_hf.py

[Misc] Update TokenizerLike interface and move get_cached_tokenizer (#29730)

2025-11-30 14:59:47 +08:00

test_mistral.py

[Mistral Grammar] Support Grammar Factory (#38150)

2026-04-06 10:28:51 -04:00

test_registry.py

[Refactor] TokenizerRegistry only uses lazy imports (#30609)

2025-12-13 23:16:22 +08:00