karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
18,671 Commits 354 Branches 175 Tags
codex/layer-parallel-plan
Commit Graph
119 Commits
Author SHA1 Message Date
Tyler Michael SmithGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Claude Opus 4.6
 		93e2ab7111 Disable dynamic speculative decoding when DP is enabled (#45963) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-07 12:52:41 +00:00
yue.yuGitHubBenjamin Chislett
 		e7c9df9449 [Bugfix][Structured Output][Spec Decode] Constrain bitmask and trim grammar advance at the reasoning boundary (#44297) 
Signed-off-by: Allen.Yu <yuyue0225sc@163.com>
Signed-off-by: yue.yu <yuyue0225sc@163.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-07-04 09:08:45 +00:00
Ekagra RanjanGitHubBenjamin ChislettNick Hill
 		07516fda67 [MRV2][SD] Make Dynamic SD comatible with Full Cuda Graphs (#45953) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-03 23:58:27 -04:00
WonderfulGitHubWan_DFBenjamin ChislettBenjamin Chislett
 		3af8789559 [Feature] Universal speculative decoding for heterogeneous vocabularies (TLI) (#38174) 
Signed-off-by: wan-danfeng <wandanfeng0802@gmail.com>
Signed-off-by: Wonderful <wandanfeng0802@gmail.com>
Co-authored-by: Wan_DF <wonderful199082@126.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-07-02 01:34:20 -07:00
Giancarlo DelfinandGitHub db808b3961 [Model Runner V2][Spec Decode] Implement block verification for rejection sampling (#46781) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-06-30 08:07:24 -07:00
Benjamin ChislettandGitHub 4c5bc41ba6 [Bugfix][Spec Decode] Fix probabilistic sampling for parallel drafting (#45956) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-06-24 05:36:23 +00:00
Nick HillandGitHub 7ee4d22009 [Spec Decode] Reject placeholder (-1) draft tokens in rejection sampler (#46533) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-24 03:32:32 +00:00
Gabriel WuGitHubClaude Opus 4.8Yongye Zhu
 		44d95069e9 Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-06-22 11:54:14 -07:00
Benjamin ChislettandGitHub 89bd2c14d3 [Spec Decode] Add Qwen3 architecture support for EAGLE3 (#43132) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-06-21 13:55:26 -07:00
stefankoncarevicandGitHub e2352c2974 [ROCm][Spec Decode] Fix probabilistic draft probs test attention backend (#45706) 
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
 2026-06-18 11:59:37 -05:00
Ekagra RanjanGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Harry MellorBenjamin Chislett
 		4ef4492e9b [V1][Spec Decode] Add Dynamic SD (#32374) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-06-14 00:14:27 -07:00
Fynn Schmitt-UlmsGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ebc6ef971a Hidden states extraction improvements (#43805) 
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-11 09:44:45 -04:00
Tianyu ZhangGitHubNick HillOpenAI Codex
 		7fe7800fa4 [BUG] Fix FP64 Gumbel precision coverage (#43150) 
Signed-off-by: tianyu-z <zhangtianyupro@gmail.com>
Signed-off-by: Tianyu Zhang <53099276+tianyu-z@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-06-05 19:04:14 +08:00
Shreyas KulkarniandGitHub 0eeba5eec1 Fix DFlash prefix cache corruption due to missing lookahead block (#42971) 
Signed-off-by: Shreyas Kulkarni <shreyas.gp269@gmail.com>
 2026-06-02 12:06:33 +00:00
Andreas KaratzasGitHubMicah Williamson
 		fd9e91d7e4 [ROCm][CI] Fix and stabilize EAGLE3 acceptance tests (#41294) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
 2026-06-01 12:40:01 -05:00
Benjamin ChislettGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		4e2eba28be [Perf] Optimize hidden state extraction logic (#37374) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-22 18:23:08 -04:00
Siddharth BedekarandGitHub f51f6844f9 [Bugfix][Spec Decode] Wire draft_probs into probabilistic draft_model rejection (#40269) 2026-05-13 21:04:03 -04:00
Giancarlo DelfinandGitHub fe5b4e0fe7 [Model Runner V2] Apply synthetic mode to probabilistic rejection sampler (#41035) 2026-05-12 13:37:03 -07:00
shanjiazandGitHub 6ccb10d794 Added peagle speculators support (#41826) 
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com>
 2026-05-12 07:55:57 -07:00
Lucas WilkinsonandGitHub b1728c1e66 [Attention][Cleanup] Remove tree attention (#42121) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-05-08 18:36:19 -07:00
Lanze LiuandGitHub 79246b5ea6 [Spec Decode] Fix max_model_len logging in speculative config for draft model (#41571) 
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
 2026-05-05 21:56:06 +00:00
wang.yuqiandGitHub a8208e6a81 [Examples] Resettle features examples. (#40995) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-28 00:33:41 -07:00
Andreas KaratzasandGitHub 428b988c98 [ROCm][CI] Fix trust_remote_code AttributeError in EAGLE3 acceptance length test (#40306) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-04-25 02:59:31 +00:00
Benjamin ChislettGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		626daa2076 [Feat] Unified Synthetic Acceptance Rate for V1 and V2 (#40662) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-24 00:48:08 +00:00
Nick HillandGitHub fe85a92e86 [Core] Avoid seq_lens_cpu GPU->CPU sync (#40654) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-04-24 00:35:55 +00:00
Matthew BonanniandGitHub cde8d24710 [Spec Decode] Move SpecDecodeBaseProposer out of eagle.py (#40732) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-23 22:28:27 +00:00
zhanqiuhuandGitHub 0b790a2501 [Speculative Decoding] Add DFlash speculators config parsing (#38300) 
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
 2026-04-15 16:22:15 -04:00
Yan MaGitHubKunshang Ji
 		0f3ce4c74b [XPU] Fix spec-decode UTs under tests/v1/spec_decode (#38491) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-11 01:31:00 +00:00
Yan MaandGitHub ec68d53b2b Add platform manual_seed_all API (#38468) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-04-10 13:43:50 +08:00
Giancarlo DelfinandGitHub 5daf62271d [Model Runner V2] Fuse probabilistic rejection sample kernels (#38496) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-04-07 17:37:37 -07:00
Aaron BatiloandGitHub 9a528260ef [Bugfix][Spec Decode] Fix extract_hidden_states for VLM models (#38987) 
Signed-off-by: Aaron Batilo <abatilo@coreweave.com>
 2026-04-05 02:41:54 -07:00
wliao2andGitHub 32e0c0bfa2 refactor hard coded device string in test files under tests/v1 and tests/lora (#37566) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
 2026-04-03 11:21:47 +08:00
Matthew BonanniGitHubSandishKumarHN
 		757068dc65 [Bugfix][Async] Fix async spec decoding with hybrid models (#38556) 
Signed-off-by: SandishKumarHN <sandishkumarhn@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: SandishKumarHN <sandishkumarhn@gmail.com>
 2026-03-31 11:08:54 -04:00
Benjamin ChislettandGitHub 494636b29d [Feat][Spec Decode] DFlash (#36847) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-03-30 15:03:15 -04:00
Giancarlo DelfinandGitHub c32e97602d [Model Runner V2] Enable forcing a specific acceptance rate during rejection sampling (#38045) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-03-26 13:38:12 -07:00
Matthew BonanniGitHubzhuhaoranzhrrrLucas WilkinsonBenjamin Chislett
 		fafe76b4af [Async][Spec Decoding] Zero-bubble async scheduling + spec decoding (#32951) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: zhrrr <43847754+izhuhaoran@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-03-23 15:37:22 -04:00
Fynn Schmitt-UlmsandGitHub 04bf5a35fa [Spec Decode] Update extract_hidden_states to use deferred kv_connector clear (#37013) 2026-03-16 14:53:45 +01:00
Kunshang JiandGitHub 53ec16a705 [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-12 07:57:47 -07:00
SladynandGitHub 4aaaf8c8ce feat(spec_decode): fuse EAGLE step slot mapping and metadata updates (#33503) 
Signed-off-by: sladynnunes <snunes@usc.edu>
 2026-03-11 04:35:33 +00:00
Matthew BonanniandGitHub 77a73458e3 Reapply [Attention] Refactor check_and_update_config (#35122) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 07:17:14 -07:00
Benjamin ChislettandGitHub 57c629e9c1 [Bugfix] Fix block_size for hybrid model MTP (#36036) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-03-05 06:10:54 +00:00
Lucas WilkinsonGitHubMatthew Bonanni
 		28ef9ba399 [BugFix] Add support for MTP num_speculative_tokens > 1 with sparse MLA (#34552) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-03 07:21:57 -08:00
Fynn Schmitt-UlmsandGitHub 9433acb8df [Spec Decode] Add hidden states extraction system (#33736) 
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
 2026-03-02 14:29:09 -05:00
Andreas KaratzasandGitHub 54254f7a61 [ROCm][CI] Fix spec decode logprobs flakiness and parametrize tree attention backends (#34599) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-20 20:25:23 -08:00
Lucas WilkinsonandGitHub aaefc58ee0 [CI] Revert PRs 34818 and 33600 (#34979) 2026-02-20 13:25:50 -08:00
Matthew BonanniGitHubRobert Shaw
 		662205d34e [Bugfix] Fix Basic Models Test (#34818) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-19 14:49:07 -08:00
Benjamin ChislettandGitHub af3162d3aa [Spec Decode] Unified Parallel Drafting (#32887) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-02-05 12:37:18 -05:00
shanjiazandGitHub d95b4be47a move spec decode slow test to test_areas.yaml (#33365) 
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com>
 2026-02-02 06:28:36 -08:00
shanjiazandGitHub 5eeba80c74 Adding optional speculator tests for larger models (#32943) 
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com>
 2026-01-29 16:54:02 +08:00
ElizaWszolaGitHubmgoinVarun Sundar RabindranathMatthew BonanniLuka GovedičLuka GovedičLucas WilkinsonLuka Govedič
 		a28b94e6ef [Performance] Split FlashAttn attention and cache update (#25954) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Luka Govedič <luka.govedic@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <luka.govedic@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2026-01-23 17:28:06 -08:00