karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
15,774 Commits 354 Branches 175 Tags
dc8df110bc8a5c80dbb8f1fef02e98deb8aa2b09
Commit Graph
5158 Commits
Author SHA1 Message Date
omerpaz95GitHubOr Ozeri
 		be0c855ebd [KV Offload] Unified memory layout for offloading workers (#37206) 
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-04-14 21:33:33 +03:00
Micah WilliamsonandGitHub 4f4f8eaa78 [ROCm][CI] Fix condition for test_per_token_group_quant_fp8_packed (#39730) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-04-14 16:14:31 +00:00
bnellnmGitHubRobert Shaw
 		e1e318af01 [MoE Refactor] Remove MoE DP chunking (#39107) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-14 09:48:05 -04:00
ThomasGitHubthomasmaindronClaude Opus 4.6
 		6f786f2c50 [Bugfix][Model] Fix Devstral Small 2 HF format weight loading (#39293) 
Signed-off-by: thomasmaindron <thomasmaindron@users.noreply.github.com>
Co-authored-by: thomasmaindron <thomasmaindron@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-14 10:11:18 +00:00
fxmarty-amdandGitHub 4eee77b877 [fix][MOE] Fix MOE experts intermediate_size dimension not being narrowed before weight loading (#39688) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-04-14 09:35:28 +00:00
Julien DebacheandGitHub 893b2affff feat: add TxtSlicesDataset to allow sampling slices from txt file for benchmarking (#30156) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-04-14 09:20:03 +00:00
Shanshan ShenGitHubRoger Wang
 		80118853f4 [MM][Perf][CG] Support ViT full CUDA graph for Qwen3-VL video inference (#38061) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-14 16:49:32 +08:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		c0ecaed950 [Frontend] Offload blocking preprocessing & postprocessing ops to thread pool for pooling entrypoints. (#39763) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-14 08:29:25 +00:00
lalit10GitHubCyrus Leung
 		0008729abf [Model] Use mm_features for Ernie-4.5 VL M-RoPE (#39753) 
Signed-off-by: Lalit Laxminarayan Bangad <lalitbangad@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-04-14 01:11:52 -07:00
Mark McLoughlinGitHubOr Ozeri
 		d3af8c1831 [Core][Metrics][BugFix] Replace num_cached_tokens/num_external_computed_tokens with PrefillStats (#37460) 
Related to `Counters can only be incremented by non-negative amounts`
error with the `vllm:prompt_tokens_by_source_total` metric.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-04-14 09:00:45 +01:00
noobHappylifeandGitHub 25b3242d8b Fix Responses API streaming for multiple auto tool calls (#39626) 
Signed-off-by: noobhappylife <aratar1991@hotmail.com>
 2026-04-14 13:28:43 +08:00
김의진GitHubChauncey
 		b075604da1 [Bugfix] Fix Gemma4 tool parser converting bare null to string "null" (#39679) 
Signed-off-by: KimuGenie <baby11686@naver.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-04-14 04:44:46 +00:00
MonishverGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nicolò Lucchesi
 		8213e8f880 Bug/test eagle dp v0 (#38938) 
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-04-13 20:50:08 +00:00
Pedram RazaviandGitHub 3693f922ff [Bugfix][Pooling] Fix silent weight corruption with buffer-reusing iterators (#39650) 
Signed-off-by: Pedram Razavi <pedram.razavi@gmail.com>
 2026-04-13 19:37:27 +00:00
mukesh-haiGitHubClaude Sonnet 4.6Mark McLoughlin
 		5c18b961d6 [Core][Metrics] expose waiting request breakdown via labeled metric (capacity/deferred) (#38435) 
Signed-off-by: Mukesh Baphna <mukesh@hippocraticai.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
 2026-04-13 15:30:55 -04:00
haosdentGitHubDarkLight1337
 		995e9a209e [Bugfix] Use is_integrated to detect UMA GPUs for memory reporting (#35356) 
Signed-off-by: haosdent <haosdent@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-04-13 11:07:40 -07:00
zhanqiuhuandGitHub 10d9872d3a [CI][Metrics] Fix local_cache_hit assertion after prompt tokens metrics updates (#39709) 
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
 2026-04-13 15:16:56 +00:00
Tihomir ElekandGitHub 8d825b87d6 [Bug] Fix TypeError when hf_config.architectures is None during model loading (#38849) 
Signed-off-by: Tihomir Elek <tiho.elek@gmail.com>
 2026-04-13 12:13:21 +01:00
Yufeng HeGitHubChauncey
 		200a727e94 [Bugfix] Fix Responses API instructions leaking through previous_response_id (#37727) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-04-13 08:46:33 +00:00
Jesus FedericoGitHubClaude Opus 4.6wang.yuqi
 		edbc1abd1c feat: add max_tokens_per_doc in rerank request. (#38827) 
Signed-off-by: Jesus Federico <jefp@amazon.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-13 01:24:09 -07:00
Flora FengandGitHub 0e39202ca9 [Bugfix] Fix GLM tool parser streaming with MTP or stream interval (#39253) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-04-13 05:10:30 +00:00
Jee Jee LiandGitHub 715681c127 [LoRA] Support dual CUDA streams-Linear Layer (#35721) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-13 10:57:07 +08:00
Yongye ZhuandGitHub 620e8924d9 [Bugfix] [Tests] Enforce out tensor device in kernel/moe/test_cutedsl_moe.py (#39644) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-04-12 17:08:08 -07:00
Animesh JainandGitHub f00c5539d7 [compile] Bug fix for _decompose_size_nodes (#38360) 
Signed-off-by: Animesh Jain <anijain@umich.edu>
 2026-04-12 20:20:24 +00:00
Le YangandGitHub 21fab0a3db fix(moe): fix RoutedExpertsCapturer assertion failure with DP>1 and MK path (#37879) 2026-04-12 10:28:17 -04:00
Nicolò LucchesiandGitHub 3244a2ebf2 [KVConnector][NIXL] Organize NIXL connector into its own directory (#39354) 
The number of features supported by the connector has grown substantially
and the `nixl_connector.py` file has accumulated a lot of code. Creates a separate
directory and isolates connector/scheduler code in the hope of improving clarity
and maintainability.

Further refactor of components aimed at improving clarity and simplifying code
will follow soon.

Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-04-12 13:10:50 +00:00
Mark McLoughlinandGitHub 72ff142c37 [Core][Metrics] Remove vllm:prompt_tokens_recomputed metric (#38709) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-04-12 12:22:01 +03:00
Martin HickeyGitHubOr Ozeri
 		cc07dad789 [HMA] [KVEvent] Enable GPU-side KV events for HMA (#37688) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-04-12 10:01:02 +03:00
EdalatiAliandGitHub df1e30e74b [Quant] add CompressedTensorsW8A8Mxfp8 for linear and MoE layers (#38815) 
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
 2026-04-11 17:21:36 -06:00
Wei ZhaoandGitHub 59b2f7b640 [Perf] Fuse Zero Initializer for FP8 DeepGemm Block Quant Kernel (#39547) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-04-11 07:16:51 -07:00
ShubyMandGitHub 92feb9991d [Gemma4][Bugfix]: Enable Gemma4ForCasualLM to load lora adapters correctly (#38844) 
Signed-off-by: ShubyM <shubymishra20@gmail.com>
 2026-04-11 09:06:49 +00:00
Yan MaGitHubKunshang Ji
 		0f3ce4c74b [XPU] Fix spec-decode UTs under tests/v1/spec_decode (#38491) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-11 01:31:00 +00:00
Harry MellorGitHubRobert Shaw
 		fc645f1acc Add structure to requirements/ directory (#39024) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-10 13:46:41 -07:00
yzong-rhandGitHub e816a8811f [Bugfix] Fix FlashInfer crash with kv_cache_dtype_skip_layers (#39002) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-04-10 18:50:47 +00:00
zhanqiuhuandGitHub e281cb721c [CI] Add MultiConnector (Nixl+Offloading) e2e edge case tests (#39343) 
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
 2026-04-10 17:35:03 +00:00
Jee Jee LiandGitHub ecd1ea1363 [Kernel] Porting the TRTLLM minimax_allreduce_rms kernels (#37045) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-11 00:20:20 +08:00
wang.yuqiandGitHub cb5f7501cb [New Model]: jinaai/jina-reranker-v3 (#38800) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-10 15:20:40 +00:00
Nicolò LucchesiandGitHub c9dddc144b [CI] Add Nixl+OffloadingConnector e2e integration tests (#39200) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-04-10 21:40:40 +08:00
jatseng-aiGitHubClaudeMatthias Gehre
 		d468322dc1 [Kernel][Hardware][AMD] Add TritonW4A16LinearKernel for ROCm (#37352) 
Signed-off-by: jatseng-ai <jatseng@amd.com>
Signed-off-by: jatseng-ai <janet.tseng@amd.com>
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-04-10 10:25:27 +00:00
PatchyTISGitHubPatchouliTaisa
 		967146e7bd [model] support FireRedLID (#39290) 
Signed-off-by: PatchouliTaisa <patchychen@tencent.com>
Co-authored-by: PatchouliTaisa <patchychen@tencent.com>
 2026-04-10 08:43:58 +00:00
Lalithnarayan CGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Luka Govedič
 		8e8a3becd1 [ZenCPU] Make PT Backport Patch Accessible to vLLM (#38205) 
Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-04-10 08:29:35 +00:00
Richard ZouandGitHub f44afef6d6 [compile] Allow strings in custom ops without regressing compilation times (#38123) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-04-10 07:26:37 +00:00
Injae RyouGitHubIsotr0py
 		447ce22212 [GGUF] Support non-standard quant types with prefix (e.g. UD-IQ1_S) (#39471) 
Signed-off-by: Injae Ryou <injaeryou@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-04-10 07:22:53 +00:00
Yan MaandGitHub ec68d53b2b Add platform manual_seed_all API (#38468) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-04-10 13:43:50 +08:00
Kyungmin LeeGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e7a1387e73 Add EXAONE-4.5 (#39388) 
Signed-off-by: lkm2835 <lkm2835@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-09 20:53:26 -07:00
ChaunceyandGitHub ecbfbb8d61 [Feature] Add auto-detection for reasoning_config when only reasoning_parser is set (#38214) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-10 01:36:26 +00:00
Ekagra RanjanandGitHub f7cad67412 [ASR] Fix spacing bw chunks in multi chunk audio transcription (#39116) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-04-09 12:46:33 -07:00
PikaPikachuandGitHub 827268e98d [Quantization] Support Quark W8A8 INT8 MoE inference (#36320) 
Signed-off-by: kangletian <Letian.Kang@amd.com>
 2026-04-09 17:24:43 +00:00
Lucas KabelaGitHubLuka Govedič
 		a8c6ee9b78 [Performance Improvement] Update batched_count_greater_than to handle batch size 1 without recompile (#38933) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-04-09 23:51:31 +08:00
Cyrus LeungandGitHub 3b1d9c3156 [CI/Build] Fix memory cleanup in MM test (#39411) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-04-09 08:50:45 -07:00