karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
16,247 Commits 354 Branches 175 Tags
2fa1f8ec00cf85b15422cb4c0e8eb3632ee13ea8
Commit Graph
2213 Commits
Author SHA1 Message Date
sychen52andGitHub 947138b6c2 Add nvfp4 kv cache support (#40177) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-05-01 04:55:16 +00:00
Dong WandGitHub 7198940b39 [Model] Add Moondream3 model support(only query and caption skills) (#32325) 
Signed-off-by: Dong Wang <dongw2019@gmail.com>
 2026-05-01 10:06:48 +08:00
Luis 🚀GitHubAndrew Sansom
 		14043dfecd feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API (#40720) 
Signed-off-by: Luis Robaina <luis@protopia.ai>
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com>
Signed-off-by: LuisRobaina <luis@protopia.ai>
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com>
 2026-05-01 10:05:55 +08:00
SoyaazzGitHubMatthew Bonanni
 		4d5c89295b (bugfix): block_size check for flex attn (#41363) 
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-30 18:59:26 -07:00
Yongye ZhuGitHubZijing LiuClaude Opus 4.7
 		b4806c8ee1 [DSV4] Add BF16 and MXFP8 A2A support for flashinfer a2a one sided (#40960) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-30 15:33:12 -07:00
Xiaoshuang WangGitHubCyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b55b26520c [MoE] Make MoERunnerInterface a PluggableLayer for OOT support (#35178) 
Signed-off-by: wxsIcey <1790571317@qq.com>
Signed-off-by: Icey <1790571317@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-30 03:31:08 -07:00
ChaunceyGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		faab189554 [Feature]: IndexCache support for DSA models (#37735) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-29 15:15:35 -04:00
Terrence ZhaoandGitHub 91a2d39014 [Models] Cohere MoE (#40817) 
Signed-off-by: Terrencezzj <terrence@cohere.ai>
 2026-04-29 15:54:54 +00:00
Tianmu LiGitHubClaudeLi, Jiang <jiang1.li@intel.com>
 		22524f7a92 [Feat] CPU fp8 attn for AMX/AVX-512 (#39445) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-04-29 20:43:21 +08:00
Walter Beller-MoralesandGitHub 1312f07531 [Feature] add cohere reasoning and tool parsers (#40422) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
 2026-04-28 21:07:53 -07:00
Kyle SayersGitHubFlora Feng
 		a085b5257d [Docs] [QeRL] Layerwise Reloading Documentation (#40317) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-04-28 21:06:38 -07:00
Russell BryantandGitHub f05f3664c3 [Doc] Add missing API endpoints to security documentation (#40532) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-04-28 21:53:19 +00:00
wang.yuqiandGitHub a8208e6a81 [Examples] Resettle features examples. (#40995) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-28 00:33:41 -07:00
Jiangyun ZhuGitHubIsotr0pyCopilot
 		7a1eb8ac2e [Model] update for mimo v25 (#41029) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Copilot <copilot@github.com>
 2026-04-27 21:52:54 -07:00
Isotr0pyGitHubJee Jee Lizjy0516zjy0516yasongClaude Opus 4.6Copilot
 		c245d35ff4 [Model] Add MiMo-V2.5 support (#40967) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <zhujiangyun@inferact.ai>
Co-authored-by: yasong <yasong.wang@inferact.ai>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Copilot <copilot@github.com>
 2026-04-27 13:26:51 +00:00
wang.yuqiandGitHub 8d8062d0a7 [Examples] Resettle generate examples. (#36464) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-27 07:48:37 +00:00
Zhanda ZhuGitHubYubo Gao
 		5d5c776444 [Perf] FP8 FlashInfer Attn for ViT (#38065) 
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com>
Co-authored-by: Yubo Gao <ybgao-nvidia@users.noreply.github.com>
 2026-04-27 13:44:15 +08:00
+8
Yifan QiaoGitHubYongye ZhuYongye ZhuSimon MoBugen ZhaoGiancarlo DelfinJee Jee LiNick HillRoger WangRoy WangWoosuk KwonyoukaichaoZhewen LiZijing LiukhluuqizixiZhewen Li
 		4d51588e23 [Feat] DeepSeek V4 Rebased (#40860) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
 2026-04-26 18:31:08 -07:00
labAxiaomingGitHubCyrus Leung
 		f768b4473e [Docs] Add docs for context extension using the yarn method (#37430) 
Signed-off-by: xiaoming <1259730330@qq.com>
Signed-off-by: labAxiaoming <34019940+labAxiaoming@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-04-24 08:26:09 -07:00
Jiangyun ZhuandGitHub e8ee2a78db [Attention] use diff kv backend for mimo v2 flash (#40045) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-04-24 11:25:55 +00:00
Dmitry TokarevandGitHub 6dec49f27e [Build] Bump CUDA to 13.0.2 to match PyTorch 2.11.0 (#40669) 
Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
 2026-04-24 10:27:11 +00:00
Jackmin801GitHubClaude
 		079a4cf399 [MoE] Move cutlass moe to fused_moe/experts/ (#40574) 
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-24 06:05:49 +00:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
 		9744b699ba [Deprecate] Deprecate LLM.reward offline api, use LLM.encode instead. (#40688) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-04-24 05:37:50 +00:00
Sophie du CouédicGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		ff2c2bd80a [Docs]Add documentation for bench serve visualization arguments (#40539) 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-23 15:48:29 -07:00
pschlan-amdGitHubTJian
 		0098db9ec1 [ROCm] Implement GPU-to-NUMA-node detection (#40015) 
Signed-off-by: Patrick Schlangen <pschlan@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-04-23 10:08:48 -05:00
Kunshang JiandGitHub 53ecc807c0 [XPU] Upgrade torch 2.11 for xpu (#37947) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-23 10:07:35 -05:00
stevenkuangGitHubJee Jee Li
 		d0009ddb0b [Model] Support Hy3 preview (#40681) 
Signed-off-by: stevenkuang <stevenkuang@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-23 22:08:26 +08:00
Honglin CaoandGitHub 9c271f9403 [gRPC] Add standard gRPC health checking (grpc.health.v1) for Kubernetes native probes (#38016) 
Signed-off-by: Honglin Cao <Caohonglin317@hotmail.com>
 2026-04-22 21:31:00 +00:00
xiaoGitHubWangxiaoxiaoa
 		ecbe42e991 [Doc] Clarify supported keys for --speculative-config (#40455) 
Signed-off-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com>
Co-authored-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com>
 2026-04-22 04:36:17 -07:00
storyiconGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6aa057c9d7 [Multimodal] Support custom video metadata for pre-extracted frame sequences (#40133) 
Signed-off-by: storyicon <storyicon@foxmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-22 15:50:04 +08:00
philip-essentialandGitHub 123674879e [Model] Add block-local attention and YaRN for local layers to Gemma3 (#39823) 
Signed-off-by: Philip Monk <169196560+philip-essential@users.noreply.github.com>
 2026-04-21 23:34:50 -07:00
Ekagra RanjanGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
 		4eafc72928 [Audio] Bundle get_generation_prompt() params into SpeechToTextParams (#36268) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-04-22 12:24:18 +08:00
Carl YGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		4506319a28 [compile] mla + group fp8 fusion (#38877) 
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-21 23:16:58 -04:00
Shanshan ShenandGitHub 936e0b79aa [MM][CG] Optimize default max_frames_per_batch auto-infer for ViT CUDA graph video inference (#40445) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-04-21 14:47:53 +00:00
Yusuf MohammadGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ec5ef0ac73 [Doc] Add Qwen3 AWQ models to documentation (#40034) 
Signed-off-by: Yusuf <yusufmohammad@live.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-21 09:37:41 -04:00
artem-spectorGitHubartemspectorClaude Opus 4.6
 		d249a9e90e Add Granite 4.1 Vision as built-in multimodal model (#40282) 
Signed-off-by: Artem Spector <artems@il.ibm.com>
Signed-off-by: artemspector <artems@il.ibm.com>
Co-authored-by: artemspector <artems@il.ibm.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-21 05:43:39 -07:00
wang.yuqiGitHubCyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d2e2e856ad [Frontend] Remove frontend pooling multi task support. (#37861) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-21 12:27:44 +00:00
Wentao YeandGitHub 301024aa9c [Deprecation] Deprecate cprofile and cprofile_context (#39100) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-21 11:25:22 +08:00
Shanshan ShenandGitHub 8097591286 [Doc] Update ViT CUDA graph doc for mixed (image+video) inputs (#40355) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-04-21 02:31:09 +00:00
baiandGitHub 191e3fdaa1 Update flashinfer to 0.6.8 (#39959) 
Signed-off-by: bai <v@gor.io>
 2026-04-20 10:37:23 -07:00
Rita BrugarolasandGitHub fb5635d3f9 [ROCm] Add MLA dual RMS norm fusion (Q, KV) pass for DeepSeek/Kimi-K2 (#39242) 
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com>
 2026-04-20 14:56:27 +00:00
larryli2-amdGitHubTJian
 		7243e02aa1 [ROCm][Feature] Enable AITER MLA attention backend to work with Eagle3 speculative decoding on ROCm (#39616) 
Signed-off-by: larryli2-amd <larryli2@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-04-20 09:44:43 -05:00
Vasiliy KuznetsovandGitHub 38fa87caca mxfp8 online quant move to new frontend (#40152) 
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com>
 2026-04-20 06:26:12 -07:00
Ilya MarkovandGitHub 50dd4cb427 [EPLB] Add nixl-based eplb communicator (#36276) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
 2026-04-20 10:24:23 +00:00
milesialandGitHub 6d8b80802b [Docs] Fix thinking_token_budget docs (#40316) 
Signed-off-by: milesial <milesial@users.noreply.github.com>
 2026-04-20 08:09:44 +00:00
LxxandGitHub d886c26d4d [Doc] Fix typos in token_embed pooling documentation (#40266) 
Signed-off-by: YifanLi3 <lyfqlx3@gmail.com>
 2026-04-19 19:27:32 -07:00
Jee Jee LiandGitHub bfde49e287 [DOC] Add fuse_minimax_qk_norm (#39782) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-18 00:41:37 -07:00
Nick CaoGitHubClaude
 		153ba7f0f3 [Refactor] Drop direct dependency on librosa (#39079) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-18 06:55:38 +00:00
z1yingandGitHub d0697cc7b6 [Doc] Add Realtime Transcription section to supported_models.md (#39845) 
Signed-off-by: Ziying Tao <tzzying@outlook.com>
 2026-04-18 03:26:14 +00:00
Michael GoinandGitHub 1174723eba Fix TURBOQUANT backend selection in cuda.py (#40060) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-04-17 07:31:41 -07:00