karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
16,190 Commits 354 Branches 175 Tags
bdb3b45e02c2c50fa5924210ebdd3ec4247eff38
Commit Graph
16190 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Luka GovedičandGitHub bdb3b45e02 Apply suggestion from @ProExpertProg 
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-04-29 16:03:22 -04:00
Luka GovedičandGitHub dc444a16b4 input not output 
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-04-29 16:02:56 -04:00
copilot-swe-agent[bot]GitHubGitHub CopilotProExpertProg
 		8521175db9 fix: migrate gpt_oss_20b moe_backend from env var to model kwargs 
- Add `flashinfer_trtllm_afp8` and `flashinfer_cutlass_afp8` to MoEBackend
  Literal in vllm/config/kernel.py so they are valid kernel_config values
- Attach `moe_backend="flashinfer_trtllm_afp8"` to the gpt_oss_20b model
  object in models.py (conditionally on Blackwell) instead of setting the
  env var in the test runner
- Remove the TODO env var block from conftest.py

Co-authored-by: GitHub Copilot

Agent-Logs-Url: https://github.com/vllm-project/vllm/sessions/8abf6884-3fc2-45fb-b5c8-1ba82b14c841

Co-authored-by: ProExpertProg <11367180+ProExpertProg@users.noreply.github.com>
 2026-04-29 18:15:03 +00:00
daniserebandGitHub d1a75e303d Fix timeout when using LoRA adapters with Nemotron Super (#40916) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-04-30 01:39:49 +08:00
Cyrus LeungandGitHub 4a42aba380 [CI/Build] Enable FP8 on NVIDIA Thor (#39712) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-04-29 09:48:52 -07:00
Avshalom ManevichandGitHub a80d6f150c better logging for large uncachable items (#41145) 
Signed-off-by: h-avsha <avshalom.manevich@hcompany.ai>
 2026-04-29 09:48:47 -07:00
Terrence ZhaoandGitHub 91a2d39014 [Models] Cohere MoE (#40817) 
Signed-off-by: Terrencezzj <terrence@cohere.ai>
 2026-04-29 15:54:54 +00:00
Frederik GossenandGitHub a05848e255 [Bugfix] Report compile time for in-memory cache hit path (#41023) 
Signed-off-by: Frederik Gossen <frgossen@meta.com>
 2026-04-29 15:32:03 +00:00
Wentao YeGitHubNick Hill
 		51fda1ba44 [Model Runner v2] Fix block table IMA issue (#40648) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-04-29 08:30:33 -07:00
Wentao YeandGitHub 39a7f4f4e2 [Perf] Optimize AllPool.forward by slicing first, 51% faster in the method level benchmark (#41163) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-29 08:11:04 -07:00
Artem PerevedentsevandGitHub b92ef9ec5a [Perf] Enable FlashInfer top-k/top-p sampler by default (#40376) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-04-29 19:10:34 +04:00
Lalithnarayan CGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		5560cac7e2 [Bugfix][CPU] Backport PT cpp codegen indirect_assert scalar-mask fix (#40973) 
Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-29 10:21:55 -04:00
pmaybankGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		5b39b268f5 hf_name argument for vllm bench throughput CLI (#41012) 
Signed-off-by: Philip Maybank <pmaybank@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-29 12:57:58 +00:00
Tianmu LiGitHubClaudeLi, Jiang <jiang1.li@intel.com>
 		22524f7a92 [Feat] CPU fp8 attn for AMX/AVX-512 (#39445) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-04-29 20:43:21 +08:00
Jee Jee LiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		9d8ad5b408 [Bugfix] Fix repeated DSv4 RoPE cache initialization (#41148) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-29 20:29:55 +08:00
Jared WenGitHubsfeng33Chauncey
 		11b69129e2 [Frontend] Add defer_loading and tool_reference support for Anthropic and OpenAI APIs (#40190) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-04-29 04:35:50 -07:00
Bugen ZhaoandGitHub 33f36d4260 [DSV4] Support max reasoning effort (#40982) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-04-29 11:03:47 +00:00
Ronen SchafferandGitHub 37e288214b [KV Offload] Tighten keys type from Iterable to Sequence in OffloadingManager (#41200) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-04-29 13:50:42 +03:00
Rohit Kumar SinghGitHubCyrus Leung
 		5371d6fb40 Fix PP in Gemma4 (#40786) 
Signed-off-by: Rohit kumar Singh <rksingh@habana.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-04-29 03:17:51 -07:00
Jiangyun ZhuandGitHub 6d7d4da99e [Bugfix] BailingMoeV2.5: rotate full qk_rope_head_dim in MLA RoPE (#41185) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-04-29 18:08:55 +08:00
AlecGitHubOpenAI Codex
 		3f1a4bb639 build: embed image provenance metadata in vLLM containers (#40653) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-04-29 03:07:41 -07:00
ChaunceyandGitHub 762022cafb [Bugfix] DSV32/V4 add missing type conversion for non-streaming tool calls (#41198) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-29 09:55:07 +00:00
ChaunceyandGitHub 3885d340a4 [Frontend]Responses API supports Tool/Function calling with streaming with named tool/function (#41110) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-29 09:11:27 +00:00
haosdentandGitHub ef70057ca7 [CI][CPU] Split CPU-Distributed Tests into per-scenario labels (#41203) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-04-29 01:28:45 -07:00
Shengqi ChenGitHubClaude
 		e48cb85185 [CI/Build] Auto-detect manylinux ABI tag for nightly wheels (#41149) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-29 00:37:14 -07:00
ChaunceyandGitHub 92879e12ba [CI] fix test_rotary_embedding_opcheck format error (#41202) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-29 00:32:37 -07:00
rishitdholakia13GitHubNick HillCyrus Leung
 		68dd7db810 [Reasoning] Support for speculative decoding with thinking budget (#34668) 
Signed-off-by: rishitdholakia13 <rishit+github@cohere.com>
Signed-off-by: rishitdholakia13 <123388671+rishitdholakia13@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-04-29 06:14:52 +00:00
Itay EtelisGitHubItay EtelisOr OzeriOr Ozeri
 		8a8c9b564e [KV Offload] Per-job store completion for CPU offloading connector (#39186) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-04-29 08:52:55 +03:00
Jee Jee LiandGitHub a269744e9f [Bugfix] Fix rope (#41113) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-28 22:42:35 -07:00
Wei ZhaoGitHubWei Zhao
 		8b49cf3a37 [Bugfix] Fix max_num_batched_token not captured in cuda graph (#40734) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Wei Zhao (Engrg-Hardware 1) <weizha@login-bia02.bia.clusters.nvidia.com>
 2026-04-28 21:33:06 -07:00
Jiangyun ZhuandGitHub 2ae73c758c [Bugfix] fix inductor error for dpsk v4 (#41135) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-04-28 21:18:46 -07:00
Fadi ArafehandGitHub d95d03c719 [BugFix][CPU] fix error on CPU runner shutdown (#41034) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-04-28 21:08:35 -07:00
Wei ZhaoandGitHub 803b9d7881 [Bugfix] Fix Deepseek V4 import error due to AOT compile cache loading (#41090) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-04-28 21:08:16 -07:00
Walter Beller-MoralesandGitHub 1312f07531 [Feature] add cohere reasoning and tool parsers (#40422) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
 2026-04-28 21:07:53 -07:00
Lucas KabelaGitHubWentao Ye
 		fa1b9840f6 [BE][Torch 2.12] Remove workaround code for fixed cublas issue (#40845) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: Lucas Kabela <lucasakabela@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-04-28 21:07:24 -07:00
Kyle SayersGitHubFlora Feng
 		916e56c05c [QeRL] Add warnings for extra memory buffering (#40309) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-04-28 21:06:54 -07:00
Kyle SayersGitHubFlora Feng
 		a085b5257d [Docs] [QeRL] Layerwise Reloading Documentation (#40317) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-04-28 21:06:38 -07:00
liangel-02andGitHub 7fd05e05ae uncomment flex backend for batch invariant mode (#40842) 
Signed-off-by: Angel Li <liangel@meta.com>
 2026-04-28 21:05:14 -07:00
Isotr0pyGitHubCopilot
 		99255f3cb5 [UX] Allow enable/disable model weights loading tracking by config (#41086) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Copilot <copilot@github.com>
 2026-04-28 21:04:49 -07:00
haosdentandGitHub 75a7cf2c10 [CI] De-flake test_chat_completion_n_parameter_non_streaming (#41147) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-04-29 03:23:59 +00:00
haosdentandGitHub 4b95e9cec4 [CI] Return HTTP 400 for unsupported chat content part type (#41121) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-04-29 10:23:26 +08:00
rasmithandGitHub 856b15c62c [CI][AMD][BugFix] Patch has_flashinfer decorator for test_select_rocm_aiter_backend (#41072) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-04-29 02:12:17 +00:00
qizixiandGitHub 6fb3f7b46b [DSV4] Align aux stream API with DeepseekV4DecoderLayer (#41171) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
 2026-04-28 17:22:03 -07:00
chelnnexyandGitHub d109eacd05 [Bugfix][ROCm] Fix gemm_a4w4 call to use updated AITER API signature (#40754) 
Signed-off-by: cheiluno <cheiluno@amd.com>
 2026-04-29 09:04:53 +09:00
Nick HillandGitHub e68fa1b90a [Core] Account for num_gpu_blocks_override in max_model_len checks (#41069) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-04-28 15:44:09 -07:00
Russell BryantandGitHub f05f3664c3 [Doc] Add missing API endpoints to security documentation (#40532) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-04-28 21:53:19 +00:00
Julien DenizeandGitHub e9f8f31e9a [FEATURE] Add EagleMistralForCausalLM (#41024) 
Signed-off-by: juliendenize <julien.denize@mistral.ai>
 2026-04-28 12:22:20 -07:00
yangrzGitHubRoger Wang
 		de3fe8dc62 [Bugfix] release KV blocks for skipped P-ranks to prevent invalid KV errors and timeouts when P_tp > D_tp and MLA (#40449) 
Signed-off-by: yangruize <yangruize7@163.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-28 11:38:43 -07:00
Joe RowellGitHubRobert Shaw
 		0899f436aa [New Model] Laguna XS.2 implementation (#41129) 
Signed-off-by: Joe Rowell <joerowell4@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-04-28 14:23:00 -04:00
rasmithandGitHub 358a755e43 [CI][AMD][BugFix] Update request URL in test_moriio_connector to match vllm-router compatibility changes (#41076) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-04-28 13:14:59 -05:00