|
e281ac663a
|
[Rust Frontend] Integrate MM audio support (#48554)
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-07-15 15:00:17 +08:00 |
|
|
Bugen ZhaoandGitHub
|
487dfb3418
|
[CI] Add SPDX license header to Rust/Protobuf sources (#48472)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-13 10:22:47 +01:00 |
|
|
|
8df14cfc8c
|
[EC Connector] Add EC Transfer Params (#42433)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-12 14:35:33 +03:00 |
|
|
|
83762b77b0
|
[Frontend] Add /abort_requests to the RLHF dev API router (#47173)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-12 14:21:02 +08:00 |
|
|
Bugen ZhaoandGitHub
|
074bdd0d99
|
[Rust Frontend] Integrate MM video support (#47959)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-10 08:15:33 +00:00 |
|
|
|
dcdd756d75
|
[CI] GSM8K eval integration test for KV offloading (#46893)
Signed-off-by: Tyler Michael Smith <tyler@tylermsmith.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-08 17:59:48 -04:00 |
|
|
|
e55cc59e52
|
[Rust Frontend][CI] Unblock more end-to-end test cases (#47735)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-07 08:27:20 +00:00 |
|
|
Tahsin TunanandGitHub
|
34e6dfced8
|
[Rust Frontend] Stamp arrival_time at the frontend entry (#47787)
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
|
2026-07-07 03:27:10 +00:00 |
|
|
Isotr0pyandGitHub
|
400a9c386d
|
[Rust Frontend] Bump llm-multimodal version (#47530)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-07-03 09:48:36 +00:00 |
|
|
|
6429d5f527
|
[Rust Frontend] add repetition_detection support to sampling params (#46684)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-03 14:06:23 +08:00 |
|
|
Bugen ZhaoandGitHub
|
25fcb65d51
|
[Rust Frontend] Use enum-backed domain types for engine outputs and structured outputs (#47283)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-02 10:41:46 +01:00 |
|
|
|
08a8a4af3f
|
feat(rust): expose profiler control routes in Rust frontend (#46306)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-02 08:46:07 +00:00 |
|
|
Bugen ZhaoandGitHub
|
c638f9216a
|
[Rust Frontend] Split engine core DTOs into separate modules (#47265)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-01 15:28:21 +01:00 |
|
|
Blas Rodriguez IrizarandGitHub
|
5b431b905c
|
[Rust Frontend] Coerce completion max_tokens: null to default (#47166)
Signed-off-by: Blas Rodriguez Irizar <rodrigblas@gmail.com>
|
2026-07-01 06:41:33 +00:00 |
|
|
Bugen ZhaoandGitHub
|
7a327f0b4f
|
[Rust Frontend] Simplify unit tests with shared TestTokenizer (#47125)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-30 15:34:43 +01:00 |
|
|
Bugen ZhaoandGitHub
|
3675bcff67
|
[Rust Frontend] Refactor TLS serve path with unified MaybeTlsListener (#47101)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-30 14:31:58 +01:00 |
|
|
ReidandGitHub
|
930f8dc0a1
|
[Bugfix][Rust Frontend] Reject prompt_logprobs for streaming generate (#46839)
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-06-30 05:10:07 +00:00 |
|
|
|
b8cb75b149
|
[Rust Frontend] Add static HTTPS and mTLS support for HTTP and gRPC (#45890)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-30 01:45:59 +00:00 |
|
|
|
d350fa8ddd
|
[Bugfix][Rust Frontend] Reject min_tokens above max_tokens (#46733)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-26 03:41:33 +00:00 |
|
|
Roy WangandGitHub
|
160c80a34c
|
[Rust Frontend] Raise frontend JSON body limit (#46582)
Signed-off-by: esmeetu <jasonailu87@gmail.com>
|
2026-06-24 12:15:31 +00:00 |
|
|
|
ce9f64020b
|
[Rust Frontend] Pass effective reasoning_parser_kwargs for structured output (#46360)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-24 03:13:44 +00:00 |
|
|
|
4ed8eaafb0
|
[Rust Frontend] Integrate xgrammar-structural-tag for strict and required tool calling (#46057)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-24 10:46:49 +08:00 |
|
|
|
2d721ab5d8
|
[Rust Frontend] Align Rust allowed_token_ids validation with Python (#46348)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-23 08:32:33 +00:00 |
|
|
|
accaa434f3
|
[Rust Frontend] Support echo for token-ID completion prompts (#46219)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-06-23 08:04:41 +00:00 |
|
|
|
a8481be7a9
|
[Rust Frontend][Perf] Use dedicated runtime for HTTP/request-processing/ZMQ (#46051)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-23 04:03:20 +00:00 |
|
|
|
80abe0de7d
|
[Rust Frontend] Support thinking_token_budget for chat and completions (#46137)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: RickyChen / 陳昭儒 <ricky.chen@infinirc.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-22 16:00:02 +08:00 |
|
|
AsharandGitHub
|
4cb5e746b6
|
[Rust Frontend]: Add /get_world_size route with static parallel size (#44801)
|
2026-06-18 13:10:20 +00:00 |
|
|
Tahsin TunanandGitHub
|
7299e6509e
|
[Rust Frontend] Return model metadata fields in /v1/models (#45950)
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
|
2026-06-18 10:29:21 +00:00 |
|
|
Sahil SinghandGitHub
|
295232a26a
|
[Rust Frontend] Add /abort_requests endpoint (#44382)
Signed-off-by: Sahil Singh <sahiilsiingh37@gmail.com>
|
2026-06-17 06:40:47 +00:00 |
|
|
ReidandGitHub
|
56e4345226
|
[Rust Frontend] Support prompt-only completions (#44938)
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-06-17 06:38:06 +00:00 |
|
|
Tahsin TunanandGitHub
|
cca3365b73
|
[Rust Frontend] Add CORS support (#45753)
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
|
2026-06-16 13:47:11 +00:00 |
|
|
Bugen ZhaoandGitHub
|
f99260d2aa
|
[Rust Frontend] Lower out-of-vocab validation to text layer (#45685)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-16 03:37:58 +00:00 |
|
|
Bugen ZhaoandGitHub
|
3f65e21e32
|
[Rust Frontend] Support max_logprobs validation (#45674)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-16 10:57:56 +08:00 |
|
|
ReidandGitHub
|
c17e2f7c84
|
[Bugfix][Rust Frontend] Make metrics respect --served-model-name (#45465)
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-06-15 08:05:10 +00:00 |
|
|
FAUSTandGitHub
|
40eac9a9d9
|
[Rust Frontend] Support parallel_tool_calls = false (#44760)
Signed-off-by: zhoujinyu <2319109590@qq.com>
|
2026-06-15 07:50:48 +00:00 |
|
|
|
1ae1051b4b
|
[Bugfix][Rust Frontend] Return 400 for prompt-validation submit errors (#45286)
Signed-off-by: xiaguan <751080330@qq.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-06-12 07:53:11 +00:00 |
|
|
|
3b03a2cf47
|
[Rust Frontend] Support continuous_usage_stats stream option (#43965)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: RickyChen / 陳昭儒 <ricky.chen@infinirc.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-11 17:50:59 +00:00 |
|
|
|
1f9dd7900d
|
[Bugfix][Rust Frontend] Validate out-of-vocab token ids in request params (#44680)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-11 03:14:11 -07:00 |
|
|
Bugen ZhaoandGitHub
|
5d5591d99b
|
[Rust Frontend] Populate cached_token_count in responses (#44887)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-10 20:50:05 -07:00 |
|
|
|
9ad08c4d15
|
[Bugfix][Rust Frontend] Fix missing added tokens in hf/fastokens tokenizer (#44683)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-10 03:52:41 -07:00 |
|
|
Mohammad Miadh AngkadandGitHub
|
2c9c07c85e
|
[Bugfix][CI/Build] Fix Rust frontend build after chat conversion refactor (#45085)
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
|
2026-06-09 20:04:41 -07:00 |
|
|
Bugen ZhaoandGitHub
|
dac9e9a640
|
[Rust Frontend] Extract shared options in route helper params (#44884)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-09 17:02:35 -07:00 |
|
|
|
69fdaffbcd
|
[Rust Frontend] Add /tokenize and /detokenize endpoints (#44222)
Signed-off-by: Tan Ngoc Do <darkknightkhtn2008@gmail.com>
Signed-off-by: TanNgocDo <darkknightkhtn2008@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-09 05:11:37 -07:00 |
|
|
|
d841386d27
|
[Rust Frontend] Support API key authentication (#44321)
Signed-off-by: RickyChen / 陳昭儒 <ricky.chen@infinirc.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-09 10:15:20 +00:00 |
|
|
Bugen ZhaoandGitHub
|
bc941f375d
|
[Rust Frontend] [Refactor] Refine utility call interfaces (#44856)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-08 15:13:08 -07:00 |
|
|
|
3c0b4432be
|
[Rust Frontend] Add /pause, /resume, /is_paused endpoints (#44499)
Signed-off-by: Sahil Singh <sahiilsiingh37@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-08 17:28:37 +08:00 |
|
|
Chao-Ju ChenandGitHub
|
e64237ae82
|
[Rust Frontend] Support include_reasoning=false (#44391)
Signed-off-by: RickyChen / 陳昭儒 <ricky.chen@infinirc.com>
|
2026-06-05 16:47:50 +08:00 |
|
|
Bugen ZhaoandGitHub
|
62d6f06e3d
|
[Rust Frontend] Skip loading multimodal processor if --language-model-only is specified (#44500)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-04 17:02:54 -07:00 |
|
|
|
309385a359
|
[Rust Frontend] Add /server_info to Rust frontend (#43942)
Signed-off-by: xunzhuo <xunzhuo@vllm-semantic-router.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-03 04:30:47 -07:00 |
|
|
|
6550ff12f2
|
[Rust Frontend] Add dynamic LoRA endpoints (#43778)
Signed-off-by: xunzhuo <xunzhuo@vllm-semantic-router.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-03 07:55:29 +00:00 |
|