karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
15,757 Commits 354 Branches 175 Tags
893b2affff548bf0f54978ca2ba4545c5b4d5fbe
Commit Graph
15757 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Julien DebacheandGitHub 893b2affff feat: add TxtSlicesDataset to allow sampling slices from txt file for benchmarking (#30156) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-04-14 09:20:03 +00:00
Shanshan ShenGitHubRoger Wang
 		80118853f4 [MM][Perf][CG] Support ViT full CUDA graph for Qwen3-VL video inference (#38061) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-14 16:49:32 +08:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		c0ecaed950 [Frontend] Offload blocking preprocessing & postprocessing ops to thread pool for pooling entrypoints. (#39763) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-14 08:29:25 +00:00
lalit10GitHubCyrus Leung
 		0008729abf [Model] Use mm_features for Ernie-4.5 VL M-RoPE (#39753) 
Signed-off-by: Lalit Laxminarayan Bangad <lalitbangad@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-04-14 01:11:52 -07:00
Mark McLoughlinGitHubOr Ozeri
 		d3af8c1831 [Core][Metrics][BugFix] Replace num_cached_tokens/num_external_computed_tokens with PrefillStats (#37460) 
Related to `Counters can only be incremented by non-negative amounts`
error with the `vllm:prompt_tokens_by_source_total` metric.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-04-14 09:00:45 +01:00
noobHappylifeandGitHub 25b3242d8b Fix Responses API streaming for multiple auto tool calls (#39626) 
Signed-off-by: noobhappylife <aratar1991@hotmail.com>
 2026-04-14 13:28:43 +08:00
김의진GitHubChauncey
 		b075604da1 [Bugfix] Fix Gemma4 tool parser converting bare null to string "null" (#39679) 
Signed-off-by: KimuGenie <baby11686@naver.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-04-14 04:44:46 +00:00
Flora FengandGitHub db8a6d66bf [Refactor][Parser] Migrate chat completion auto-tool/reasoning/plain streaming to parse_delta (#39446) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-04-14 04:39:45 +00:00
ChaunceyandGitHub d2130a47bb [Bugfix]: Fix MinimaxM2ToolParser missing tools parameter (#39683) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-14 11:16:39 +08:00
chunxiaozhengGitHubYihua Cheng
 		c687bf226a [LMCache][MP] optimize save when mla enabled (#38810) 
Signed-off-by: idellzheng <idellzheng@tencent.com>
Co-authored-by: Yihua Cheng <yihua98@uchicago.edu>
 2026-04-13 17:56:43 -07:00
Giancarlo DelfinandGitHub ccf90ba784 [Model Runner V2] Add full cuda graph support for eagle prefill (#37588) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-04-13 16:01:24 -07:00
Netanel HaberandGitHub 6adacfcb65 ParakeetExtractor performance and UX enhancements (#39423) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-04-13 21:37:35 +00:00
Flora FengandGitHub 14cb86c187 [Refactor][Parser] Simplify parse_delta (#39728) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-04-13 21:02:13 +00:00
MonishverGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nicolò Lucchesi
 		8213e8f880 Bug/test eagle dp v0 (#38938) 
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-04-13 20:50:08 +00:00
Pedram RazaviandGitHub 3693f922ff [Bugfix][Pooling] Fix silent weight corruption with buffer-reusing iterators (#39650) 
Signed-off-by: Pedram Razavi <pedram.razavi@gmail.com>
 2026-04-13 19:37:27 +00:00
mukesh-haiGitHubClaude Sonnet 4.6Mark McLoughlin
 		5c18b961d6 [Core][Metrics] expose waiting request breakdown via labeled metric (capacity/deferred) (#38435) 
Signed-off-by: Mukesh Baphna <mukesh@hippocraticai.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
 2026-04-13 15:30:55 -04:00
Tyler Michael SmithGitHubClaude Opus 4.6
 		f72b20976c [Bugfix] Reject non-nvfp4 dtypes when using the flashinfer_nvlink_one_sided all2all backend (#39717) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-13 19:13:51 +00:00
Yuyi AoGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Harry Mellor
 		610a3efcaf [Doc] Fix Python-only build 404 fallback guidance (#38052) 
Signed-off-by: George-ao <yuyiao772@gmail.com>
Signed-off-by: Yuyi Ao <yuyiao772@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-04-13 12:09:31 -07:00
JartXandGitHub f414f90601 [Bugfix][Kernel][ROCm] Fix triton_w4a16 scales mismatch when BLOCK_K > group_size (#39705) 
Signed-off-by: JartX <sagformas@epdcenter.es>
 2026-04-13 14:29:45 -04:00
Nicolò LucchesiandGitHub 8625ec267b [Misc] Multi-turn benchmark output performance json (#39572) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-04-13 18:15:23 +00:00
haosdentGitHubDarkLight1337
 		995e9a209e [Bugfix] Use is_integrated to detect UMA GPUs for memory reporting (#35356) 
Signed-off-by: haosdent <haosdent@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-04-13 11:07:40 -07:00
Yongye ZhuandGitHub 739e5945dc [Quantization] [Refactor] Create special "GptOssMxfp4MoeMethod" (#39604) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-04-13 12:53:58 -04:00
Santino RamosandGitHub 4d042ed85f [Bugfix] Fix tensor shape mismatch in sparse attention with speculative decoding (#39542) 
Signed-off-by: Santino Ramos <santinor@inferact.ai>
 2026-04-13 08:57:38 -07:00
zhanqiuhuandGitHub 10d9872d3a [CI][Metrics] Fix local_cache_hit assertion after prompt tokens metrics updates (#39709) 
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
 2026-04-13 15:16:56 +00:00
Wentao YeGitHubTJian
 		ccd0d1d906 [Bug] Fix rocm sparse attn indexer issue (#39225) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-04-13 07:53:45 -07:00
Yi LiuandGitHub d8ddb31644 [Bugfix][CT] Fix KV cache scale handling (#39418) 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
 2026-04-13 10:50:16 -04:00
Ekagra RanjanandGitHub 1ce0318c68 [Bugfix] stream failure when model name not in audio endpoints (#36679) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-04-13 14:20:07 +00:00
Tihomir ElekandGitHub 8d825b87d6 [Bug] Fix TypeError when hf_config.architectures is None during model loading (#38849) 
Signed-off-by: Tihomir Elek <tiho.elek@gmail.com>
 2026-04-13 12:13:21 +01:00
zofiaandGitHub 1b19bd7589 [MXFP8] [XPU] add a new compressed tensor schema and add a xpu mxfp8 gemm kernel (#38707) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
 2026-04-13 16:59:20 +08:00
Yufeng HeGitHubChauncey
 		200a727e94 [Bugfix] Fix Responses API instructions leaking through previous_response_id (#37727) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-04-13 08:46:33 +00:00
Jesus FedericoGitHubClaude Opus 4.6wang.yuqi
 		edbc1abd1c feat: add max_tokens_per_doc in rerank request. (#38827) 
Signed-off-by: Jesus Federico <jefp@amazon.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-13 01:24:09 -07:00
Flora FengandGitHub 0e39202ca9 [Bugfix] Fix GLM tool parser streaming with MTP or stream interval (#39253) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-04-13 05:10:30 +00:00
sihao_liGitHubKunshang Ji
 		9dd5ee0117 [XPU]Enhance environment collection for Intel XPU and optimize layout (#35698) 
Signed-off-by: sihao.li <sihao.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-13 12:51:46 +08:00
Jesus FedericoGitHubClaude Opus 4.6wang.yuqi
 		fa6ae31177 feat: rename logit_bias/logit_scale to logit_mean/logit_sigma for affine score calibration (#39530) 
Signed-off-by: Jesus Federico <jefp@amazon.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-13 04:43:44 +00:00
maobaolongandGitHub 2a3c32ce67 fix(lmcache): correct store for cached requests and num_scheduled_tokens in lmcache_mp_connector.py (#39655) 
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
 2026-04-13 03:29:19 +00:00
Kaicheng YangGitHubClaude Sonnet 4.6
 		4beeb0689c fused qknorm+rope kernel optimization for SM9.0 (#37376) 
Signed-off-by: EricccYang <yangyang4991@gmail.com>
Signed-off-by: Kaicheng Yang <53411596+EricccYang@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-12 19:58:37 -07:00
Zhengxu ChenGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		cae984060f [compile] Enable AOT compile with batch invariance mode. (#39201) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-12 19:58:33 -07:00
Jee Jee LiandGitHub 715681c127 [LoRA] Support dual CUDA streams-Linear Layer (#35721) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-13 10:57:07 +08:00
Kunshang JiandGitHub dc02271d76 [XPU] revert torch-xpu to 2.10 (#39656) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-13 10:50:29 +08:00
Andreas KaratzasandGitHub 4e4ad41d11 [ROCm][CI] Removed stale tests and extended acceptance test (#39651) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-04-13 10:40:26 +08:00
Yongye ZhuandGitHub 620e8924d9 [Bugfix] [Tests] Enforce out tensor device in kernel/moe/test_cutedsl_moe.py (#39644) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-04-12 17:08:08 -07:00
Animesh JainandGitHub f00c5539d7 [compile] Bug fix for _decompose_size_nodes (#38360) 
Signed-off-by: Animesh Jain <anijain@umich.edu>
 2026-04-12 20:20:24 +00:00
Le YangandGitHub 21fab0a3db fix(moe): fix RoutedExpertsCapturer assertion failure with DP>1 and MK path (#37879) 2026-04-12 10:28:17 -04:00
Nicolò LucchesiandGitHub 3244a2ebf2 [KVConnector][NIXL] Organize NIXL connector into its own directory (#39354) 
The number of features supported by the connector has grown substantially
and the `nixl_connector.py` file has accumulated a lot of code. Creates a separate
directory and isolates connector/scheduler code in the hope of improving clarity
and maintainability.

Further refactor of components aimed at improving clarity and simplifying code
will follow soon.

Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-04-12 13:10:50 +00:00
Mark McLoughlinandGitHub 72ff142c37 [Core][Metrics] Remove vllm:prompt_tokens_recomputed metric (#38709) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-04-12 12:22:01 +03:00
Nick HillandGitHub ee3c0c83db [Pooling] Disable async scheduling by default for pooling models (#39592) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-04-12 07:23:42 +00:00
Martin HickeyGitHubOr Ozeri
 		cc07dad789 [HMA] [KVEvent] Enable GPU-side KV events for HMA (#37688) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-04-12 10:01:02 +03:00
r266-techGitHubRoger Wang
 		17e787a779 fix(kimi_k25): resolve media_placeholder_token_id from tokenizer (#39344) 
Signed-off-by: r266-tech <r266.tech@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-11 21:10:24 -07:00
Xinyu ChenGitHubKunshang Ji
 		639402f5a2 Support FP8 KVCache on XPU (#37731) 
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-12 03:53:32 +00:00
Andreas KaratzasandGitHub 0f7be0f2f7 [ROCm][CI/Build] Fix memory cleanup in MM test (#39555) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-04-12 11:13:34 +08:00