karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 2 Packages Activity
18,263 Commits 354 Branches 175 Tags
e0bbe8d667af413ee4f5f65eaa930b329f98cb4e
Commit Graph
18263 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Woosuk Kwon e0bbe8d667 Skip compile SP pass under enforce_eager (V2 runner) 
sp_threshold force-enabled compilation_config.pass_config.enable_sp, which the
V2 model runner rejects ("does not yet support: sequence parallelism"). The
eager in-forward SP path reads sp_threshold directly and needs no compile pass;
under enforce_eager there is no compilation to attach it to. Guard the mapping.

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 15:27:09 +00:00
Woosuk Kwon 44c1a8eb78 Simplify SP token padding condition and comments 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 15:26:34 +00:00
Woosuk Kwon d58e45771c Require sp_threshold > max cudagraph size instead of aligning sizes 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 15:26:34 +00:00
Woosuk Kwon 6bcd9b3d03 revert mrv1 & do tp check 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 15:26:34 +00:00
Woosuk Kwon 2e8704317e minor 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 15:26:34 +00:00
Woosuk Kwon ab64990834 minor 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 15:26:34 +00:00
Woosuk Kwon 1513ab9940 fix 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 15:26:34 +00:00
Woosuk Kwon 5072ad2f65 MTP runs without mHC SP (full-token draft) 
Draft is a single decoder block; sharding its per-token mHC saves negligible compute and would require tp-aligning the whole spec-decode dispatch. Draft consumes the target's full pre-hc_head residual.

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 15:26:34 +00:00
Woosuk Kwon 1f42f683c4 sp with no ep 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 15:26:34 +00:00
Woosuk Kwon 44c5396c74 wip 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 15:26:33 +00:00
Igor MargulisGitHubCursormgoin
 		d8f483dc30 [Spec Decode] Fix hidden-state extraction block size for hybrid verifiers (#46301) 
Signed-off-by: Igor Margulis <igor.margulis@intel.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-06-30 08:19:51 -07:00
Nicolò LucchesiandGitHub dc148dc4d7 [CI][Bugfix] Fix Hybrid SSM NixlConnector PD prefix cache test (2 GPUs) (#47157) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-06-30 23:14:13 +08:00
tc-mbandGitHub 7cf7cbcd95 [Bugfix] MiniCPM-V 4.6: fix grid rows/cols swap in placeholder generation (#45918) 
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
 2026-06-30 08:12:44 -07:00
Juan Pérez de AlgabaGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c231d1f290 fix(security): bound tokenizer work when explicit truncation_side is set (#47007) 
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-30 23:08:51 +08:00
Giancarlo DelfinandGitHub db808b3961 [Model Runner V2][Spec Decode] Implement block verification for rejection sampling (#46781) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-06-30 08:07:24 -07:00
Arsalan ShakilandGitHub 00ebf19cca [Bugfix][Quant] Raise actionable error instead of bare assert for group-size/TP mismatch (#46230) (#46236) 
Signed-off-by: Arsalan Shakil <shakil.arsalan@yahoo.com>
 2026-06-30 14:57:14 +00:00
Seiji EicherGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ded6676458 [Bugfix] Seed RayExecutorV2 TCPStore port by DP rank to avoid collisions (#45960) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-30 07:37:34 -07:00
Bugen ZhaoandGitHub 7a327f0b4f [Rust Frontend] Simplify unit tests with shared TestTokenizer (#47125) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-30 15:34:43 +01:00
Harry MellorandGitHub 1ab9522935 Remove more unnecessary load_weights methods (#47058) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 15:22:16 +01:00
Ronen SchafferGitHubOr Ozeri
 		0fc2512094 [KV Offload] Pass ScheduleEndContext to on_schedule_end hook (#46450) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-06-30 17:07:12 +03:00
Harry MellorandGitHub 62c7d8009f Forward fix nightly errors from #44589 (#47151) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 14:02:34 +00:00
Isotr0pyandGitHub ab80b3dff4 [CI/Build] Bump PyNvVideoCodec version (#47139) 2026-06-30 06:38:46 -07:00
Qiming ZhangandGitHub 91055efd36 [XPU] C++ implementation for get_memory_info (#47134) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2026-06-30 21:34:47 +08:00
Bugen ZhaoandGitHub 3675bcff67 [Rust Frontend] Refactor TLS serve path with unified MaybeTlsListener (#47101) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-30 14:31:58 +01:00
Bugen ZhaoandGitHub bdbd7278b6 [Rust Frontend] Extend renderer/parser roundtrip tests to support token ids (#47110) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-30 14:27:45 +01:00
Harry MellorandGitHub 5dc36a4fa5 [Model] Remove Tarsier, Tarsier2 (#47143) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 13:20:33 +00:00
peizhang56GitHubTJian
 		aab7af0bcb [Bugfix][ROCm][MLA] Pass q/kv dtypes to get_mla_metadata_v1 in FP8 decode (#46997) 
Signed-off-by: pei.zhang <pei.zhang@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-06-30 05:31:16 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 		536047755e Bump actions/checkout from 6.0.1 to 7.0.0 (#33057) 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-30 13:16:20 +01:00
JianweiZhengGitHubClaudeWentao YeHarry Mellor
 		1907d3854a [Bugfix] Reject negative values for max_logprobs and long_prefill_token_threshold (#44002) 
Signed-off-by: jwzheng96 <jianweizheng@pku.edu.cn>
Signed-off-by: JianweiZheng <32029023+jwzheng96@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 13:01:03 +01:00
Chaojun ZhangandGitHub ea9ddf59fc [XPU][CI] Enable shared loader test (#45977) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
 2026-06-30 11:20:33 +00:00
Cheng JiangGitHubchengvjiangAndreas Karatzas
 		8cf7c4d8ad [Attention Backend] add HPC-Ops Attention backend (#46020) 
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-30 18:17:43 +08:00
Agata DobrzyniewiczGitHubKunshang Ji
 		8e9d70fdd5 [Kernel][XPU] Adjust kernel unit tests for XPU (#45140) 
Signed-off-by: Dobrzyniewicz, Agata <agata.dobrzyniewicz@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-06-30 09:57:27 +00:00
Juan Pérez de AlgabaandGitHub 364ee36af1 fix(security): prevent image decompression bomb OOM denial of service (#47010) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-06-30 09:39:22 +00:00
Nicolò LucchesiandGitHub 06fae69114 [Misc] Mistral label alert (#47132) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-06-30 09:02:07 +00:00
Li, JiangGitHubHarry Mellor
 		14f8660a18 [CI/Build] Add CPU test dependency pre-commit hooks (#47032) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 07:59:13 +00:00
Aman PaswanGitHubamanambakChauncey
 		aed541def4 [Bugfix][Responses] Set completed status for Harmony function calls (#46945) 
Signed-off-by: amanambak <aman.paswan@ambak.com>
Co-authored-by: amanambak <aman.paswan@ambak.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-06-30 07:55:14 +00:00
ChaunceyGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2bc20e8aba [Frontend] Add Streaming Parser Engine and new Kimi k2.5/k2.6/k2.7 Parser (#46610) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-30 07:53:17 +00:00
Chaojun ZhangandGitHub 8cc242335d [XPU] Optimize XPU worker shutdown logic to prevent resource leak (#46433) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
 2026-06-30 15:27:21 +08:00
Andreas KaratzasandGitHub ba22cb6765 [ROCm][Ray][CI] Keep assigned GPU visible for weight transfer (#47000) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-30 14:59:18 +08:00
Uros MarkovicandGitHub 81bcced482 [Bugfix][ROCm] Preserve MoE weight padding for unquantized Triton path (#46381) 
Signed-off-by: Uros Markovic <umarkovi@amd.com>
 2026-06-30 14:47:57 +08:00
Kunshang JiandGitHub fb42e5219e [Platform] Replace torch.cuda.mem_get_info with torch.accelerator.get_memory_info (#44825) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-06-30 14:39:52 +08:00
Dakai AnandGitHub 0feca7ffa8 PD disagg with Mooncake Connector: GDN support (Qwen3.5) and MLA support (Deepseek-V4-Flash) (#46807) 2026-06-29 23:29:04 -07:00
Muhammad FawazGitHubMahad Durrani
 		97b5ce5c39 [Bugfix] Raise VLLMValidationError for non-integer logit_bias keys (#46612) 
Signed-off-by: muhammadfawaz1 <135441198+muhammadfawaz1@users.noreply.github.com>
Co-authored-by: Mahad Durrani <114791389+mahadrehmann@users.noreply.github.com>
 2026-06-30 06:18:59 +00:00
Andreas KaratzasandGitHub 4236514098 [ROCm][CI][Multimodal] Use ROCm-aware FA availability check for Unlimited-OCR (#47004) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-30 14:03:13 +08:00
Blas Rodriguez IrizarandGitHub e45c8a9f4b [Rust Frontend] Start current wave for a stale DP FirstRequest (#46833) 
Signed-off-by: Blas Rodriguez Irizar <rodrigblas@gmail.com>
 2026-06-30 05:13:09 +00:00
Wei ZhaoandGitHub b153dd3f28 [Bugfix] Use larger workspace size for Flashinfer MLA LSE (#47074) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-06-29 22:11:03 -07:00
ReidandGitHub 930f8dc0a1 [Bugfix][Rust Frontend] Reject prompt_logprobs for streaming generate (#46839) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-06-30 05:10:07 +00:00
ReidandGitHub a16dbd5b85 [Rust Frontend] Avoid LoRA registry scans without active LoRA requests (#47040) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-06-30 04:58:19 +00:00
liranschourGitHubOr OzeriOr Ozeri
 		bec232a914 Secondary tier implementation for PD disaggregation (#42285) 
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: liranschour <liranschour@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-06-30 07:51:44 +03:00
linitra24GitHubJee Jee Li
 		b5c9e1ac33 [LoRA] Add language-backbone LoRA support for MiniCPM-V 4.6 (#46740) 
Signed-off-by: linitra24 <Joy25810@foxmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-06-30 04:19:31 +00:00