karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
16,694 Commits 354 Branches 175 Tags
599e75f432e5fd7c77e65dc95587f3441201bdbc
Commit Graph
16694 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
TJianandGitHub 599e75f432 [ROCm] [Bugfix] Fix DeepSeek V4 Functionality and Accuracy (#42810) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-05-17 12:18:50 -04:00
Taneem IbrahimandGitHub 1c8e9c0399 Refactor: Pass num_labels explicitly to PoolerClassify instead of reading from global config (#42851) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-05-17 14:40:21 +00:00
zofiaGitHubKunshang Ji
 		0fa888465e [XPU] fix weight scale shape (#42725) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-05-17 16:55:10 +08:00
liuzhenweiandGitHub ff712f6447 [MRV2][XPU] add Model Runner V2 log (#42710) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-05-17 04:15:50 +00:00
Qi ZhouandGitHub 504a26ce2b Support bf16 for mamba ssm cache (#41680) 
Signed-off-by: Qi Zhou <qizzzh@google.com>
 2026-05-16 17:54:58 -07:00
weizhoublueandGitHub a94189295b Fix Weight loading for Qwen3.5-MTP and Qwen3-VL using runai_streamer (#42716) 
Signed-off-by: weizhoublue <weizhou.lan@daocloud.io>
 2026-05-16 17:54:27 -07:00
Artem PerevedentsevGitHubVadim Gimpelson
 		0867497368 [CI/Build] Bump flashinfer to v0.6.11.post2 (#41711) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-05-16 14:55:12 -07:00
Zhewen LiGitHubZhewen LiClaude Opus 4.7
 		36e74c9ea4 [KV Connector] Support disk offloading in MooncakeStoreConnector (#42689) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-16 13:34:15 -07:00
Taneem IbrahimandGitHub 787bc0d031 Add unit tests for pooler activation functions (#42824) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-05-16 14:58:16 -04:00
weizhoublueandGitHub d1586e1a12 Fix: Propagate pinned model revisions into Ultravox secondary weight loading (#42830) 2026-05-16 17:02:54 +00:00
Jiangyun ZhuandGitHub 8a56da3845 [Experimental] Breakable CUDA graph (#42304) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-05-16 22:04:12 +08:00
Andreas KaratzasandGitHub 4db300e95f [ROCm][CI] Removed problematic command override mechanism (#42807) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-16 17:35:05 +08:00
Zhewen LiGitHubZhewen Likhluu
 		657b42b592 [Docker][KVConnector] Build mooncake-transfer-engine from source (#42114) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: khluu <khluu000@gmail.com>
 2026-05-16 00:26:25 -07:00
Jee Jee LiandGitHub 32b7177909 [LoRA][Bugfix] Dedup LoRA wrapping for modules referenced from multiple attribute paths (MoE gate) (#42757) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-05-16 11:22:35 +08:00
DustHunterGitHubQwen-Coderwang.yuqi
 		39c67d714e fix: add API key authorization to /v2 endpoints (#42594) 
Signed-off-by: DustHunter <dusthunter@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-16 01:29:27 +00:00
Viktor PusGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		87a2adcb43 [Misc] Add common random prefix option to structured-output serving benchmark (#41632) 
Signed-off-by: Viktor Pus <viktorpus@tenstorrent.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-16 00:44:48 +00:00
Michael GoinandGitHub 852f567444 [Bugfix] Respect explicit --kv-cache-dtype over checkpoint kv_cache_scheme (#42782) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-05-15 17:15:52 -07:00
Michael GoinandGitHub b2a27b82d9 [Kernel][UX] Add --linear-backend arg for linear kernel selection (#39538) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-05-15 17:07:39 -07:00
Keyi LiandGitHub d0921bafef [Bugfix] Unwrap VLM wrappers for EPLB on Model Runner V2 (#42706) 2026-05-16 07:20:33 +08:00
rasdaniGitHubOpenAI CodexRoger Wang
 		1ccdf87507 [Bugfix] Fix layerwise reload alias-buffer corruption (#42481) 
Signed-off-by: rasdani <73563550+rasdani@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-15 15:20:53 -07:00
Rita BrugarolasandGitHub bd9dbe6060 [ROCm][Bugfix] Fix fused_mla_dual_rms_norm for AITER API rename _fused_qk_rmsnorm (#42606) 
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com>
 2026-05-15 14:50:03 -06:00
Michael GoinGitHubClaude
 		de2d76f352 [Build] Switch CUDA 12.9 wheel builds to PyTorch manylinux_2_28 base (#41668) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-15 13:46:16 -07:00
Sergei SkvortsovGitHubMichael Goin
 		9a7a273dfe Add HumanEval and GSM8K benchmarks to datasets (#42648) 
Signed-off-by: southfreebird <yvorott@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-05-15 13:01:21 -07:00
Lanze LiuGitHubMatthew Bonanni
 		b2c58ee942 [FlashAttn] Fix supports_kv_cache_dtype() accepting unhandled fp8 kv-cache dtype variants (#42685) 
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-15 15:34:59 -04:00
frida-anderssonandGitHub 4d67d3bde2 [ROCm] Restore fast top_k_per_row kernels for sparse MLA when topk_tokens=2048 (#42072) 
Signed-off-by: Frida Andersson <fanderss@amd.com>
 2026-05-15 19:02:57 +00:00
Blake LeddenGitHubClaude Opus 4.6Pavani Majety
 		06d020bb6e [Bugfix] Fix SM121 (DGX Spark) exclusion from Marlin/CUTLASS FP8 paths (#35568) 
Signed-off-by: Blake Ledden <blake@secondnaturecomputing.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2026-05-15 10:59:00 -07:00
chunxiaozhengandGitHub f45c210885 [LMCacheMPConnector] Prioritize importing the lmcache_mp_connector from lmcache (#42596) 
Signed-off-by: idellzheng <idellzheng@tencent.com>
 2026-05-15 17:46:31 +00:00
akii96andGitHub be7a03ea65 [ROCm] Widen AITER fused AR RMSNorm 1-stage gate (#42409) 
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
 2026-05-15 17:44:38 +00:00
Wentao YeGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		6147c70224 [Model Runner v2] Support reload weights (sleep mode) (#42673) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-15 16:41:23 +00:00
PatchyTISGitHubPatchouliTaisa
 		0162596603 [Model Runner V2] FP32 gumbel sampling. (#41775) 
Signed-off-by: PatchouliTaisa <patchychen@tencent.com>
Co-authored-by: PatchouliTaisa <patchychen@tencent.com>
 2026-05-15 09:20:08 -07:00
Markus HartikainenGitHubclintg6frida-anderssonCursor
 		46a95815d3 [ROCm][MLA] FP8 ASM prefill for AITER dense MLA backend on gfx950 (#42509) 
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: clintg6 <clint.greene@amd.com>
Co-authored-by: frida-andersson <frida.andersson@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-15 23:56:58 +08:00
BadrBasowidandGitHub fb5bd03f51 [Perf] Set IR Op Priority Once at Worker Init (#42631) 
Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
 2026-05-15 15:56:13 +00:00
Mohammad Miadh AngkadandGitHub ee58665aac [Bugfix] Fix DeepGEMM context lens contiguity in MLA indexer (#42135) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-05-15 23:29:58 +08:00
Wentao YeandGitHub 491e8d8539 [Perf] Optimize MLA attention _v_up_proj bmm by removing additional copy (#42561) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-15 08:14:26 -07:00
Wentao YeandGitHub af9616d845 [Model Runner V2] Fix kv_connector pre_forward order (#42676) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-15 08:13:59 -07:00
L.B.R.GitHubL.B.R.
 		d792d993c1 [ROCm] Widen OAI Triton MoE capability range to include gfx12 (RDNA4) (#37826) 
Signed-off-by: L.B.R. <lbr@mmonad.com>
Co-authored-by: L.B.R. <lbr@mmonad.com>
 2026-05-15 07:59:57 -07:00
Aaron HaoandGitHub e0a45f1455 [Feat][RL] IPC weight sync optimizations: multigpu support and chunked packed tensors (#37476) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: hao-aaron <ahao@anyscale.com>
 2026-05-15 22:53:06 +08:00
Benjamin ChislettandGitHub 0fe7550254 [Bugfix] DFlash FP8 KV-Cache (#42692) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-05-15 08:29:45 -06:00
Li, JiangandGitHub 95cfe102a5 [Bugfix] Ensure embeding model compilation on CPU (#42709) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-05-15 18:58:19 +08:00
pmaybankGitHubTJian
 		1dc3fe08ea gemma3 multi-gpu bug-fix (#42630) 
Signed-off-by: Philip Maybank <pmaybank@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-05-15 02:32:05 -07:00
Juan Pérez de AlgabaGitHubCyrus Leung
 		d26a28ab03 fix: propagate revision/code_revision pins to all artifact boundaries (#42616) 
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-05-15 02:31:54 -07:00
Andreas KaratzasandGitHub d735968f6d [ROCm][CI] Stage B gating (#42025) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
v0.21.1rc0 		2026-05-15 01:49:27 -07:00
bobofang_amdGitHubTJian
 		ccde9540be DeepSeekV4-Pro enable cuda graph full and piecewise mode (#42604) 
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-05-15 01:45:30 -07:00
wang.yuqiandGitHub 75fd68c7a5 [Entrypoints] Split the pooling offline API into PoolingOfflineMixin. (#42267) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-15 08:05:57 +00:00
Yifan QiaoandGitHub 4b364f810e [Core][DSV4] Skip caching SWA blocks that can never serve a prefix-cache hit (#42258) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-05-15 15:59:18 +08:00
yupengGitHubJee Jee Li
 		31fa757cf9 [Misc] Make it simpler to replace out-of-tree layer classes with related LoRA layers. (#42306) 
Signed-off-by: paulyu12 <507435917@qq.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-05-15 15:20:42 +08:00
Cyrus LeungandGitHub 2676ab1e0b [Deprecation] Remove old locations of get_tokenizer and resolve_hf_chat_template (#35024) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-05-15 00:13:32 -07:00
Daniel WGitHubLi, Jiang <jiang1.li@intel.com>
 		27b85d2084 [Bugfix] Clarify CPU backend memory error messages reference shared flag (#42479) 
Signed-off-by: daniel-devlab <282598346+daniel-devlab@users.noreply.github.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-05-15 06:35:05 +00:00
Louie TsaiandGitHub e30f39c4f1 Update Intel Xeon model list and vLLM Benchmark Suite BKMs (#42607) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-05-15 05:14:03 +00:00
Keyi LiGitHubKeyi Limergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		bf610c2f56 [Bugfix] Fix inverted condition causing thinking_token_budget to be silently ignored (#41674) 
Signed-off-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-15 12:48:49 +08:00