karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 2 Packages Activity
6,768 Commits 354 Branches 175 Tags
6881107948c00a8564bc2fa85308f6fc2f065d64
Commit Graph
135 Commits
Author SHA1 Message Date
Nan QinGitHubAndrew Sansom
 		9609327fa4 [Core] [Bugfix]: tensor parallel with prompt embeds (#18171) 
Signed-off-by: Nan2018 <nan@protopia.ai>
Co-authored-by: Andrew Sansom <andrew@protopia.ai>
 2025-05-19 20:21:27 -07:00
Isotr0pyandGitHub 021c16c7ca [Model] Broadcast Ovis2 implementation to fit Ovis1.6 (#17861) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-11 17:56:30 -07:00
Harry MellorandGitHub d6484ef3c3 Add full API docs and improve the UX of navigating them (#17485) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-03 19:42:43 -07:00
Cyrus LeungandGitHub d7543862bd [Misc] Rename assets for testing (#17575) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-02 03:29:25 -07:00
Andrew SansomGitHub临景Bryce1010Nan2018Cyrus LeungDarkLight1337
 		cc2a77d7f1 [Core] [Bugfix] Add Input Embeddings (#15428) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: 临景 <linjing.yx@alibaba-inc.com>
Co-authored-by: Bryce1010 <bryceyx@gmail.com>
Co-authored-by: Nan2018 <nan@protopia.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-02 01:06:39 -07:00
Cyrus LeungandGitHub f89d0e11bf [Misc] Continue refactoring model tests (#17573) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-01 22:06:08 -07:00
Harry MellorandGitHub 13698db634 Improve configs - ModelConfig (#17130) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-30 10:38:22 +08:00
ponix-jandGitHub bdb2cddafc [Misc]Use a platform independent interface to obtain the device attributes (#17100) 2025-04-29 06:59:13 +00:00
Alex BrooksandGitHub fa93cd9f60 [Model] Add Granite Speech Support (#16246) 
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-04-28 10:05:00 +00:00
Cyrus LeungandGitHub 43faa0461a [Bugfix] Fix hybrid model tests (#17182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-25 15:14:37 -07:00
Harry MellorandGitHub 0a05ed57e6 Simplify TokenizerGroup (#16790) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-24 04:43:56 -07:00
Isotr0pyandGitHub cb072ce93b [Bugfix] Update Florence-2 tokenizer to make grounding tasks work (#16734) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-04-17 04:17:39 +00:00
Alex BrooksGitHubCyrus Leung
 		6b40996ae8 [Core][Bugfix] Fix Offline MM Beam Search (#16390) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-04-15 10:33:02 +08:00
wang.yuqiandGitHub fbf722c6e6 [Frontend] support matryoshka representation / support embedding API dimensions (#16331) 2025-04-11 23:23:10 -07:00
wang.yuqiandGitHub 1f5d13ab9f [New Model]: jinaai/jina-embeddings-v3 (#16120) 2025-04-08 08:39:12 -07:00
Wei ZengandGitHub 30d6a015e0 [Feature] specify model in config.yaml (#15798) 
Signed-off-by: weizeng <weizeng@roblox.com>
 2025-04-01 01:20:06 -07:00
Roger WangandGitHub c67abd614f [V1] Support interleaved modality items (#15605) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-29 06:30:09 -07:00
Russell BryantandGitHub b877031d80 Remove openvino support in favor of external plugin (#15339) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-22 14:06:39 -07:00
Cyrus LeungandGitHub baec0d4de9 Revert "[Feature] specify model in config.yaml (#14855)" (#15293) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-21 08:30:23 -07:00
Wei ZengandGitHub 0fa3970deb [Feature] specify model in config.yaml (#14855) 
Signed-off-by: weizeng <weizeng@roblox.com>
 2025-03-21 00:26:03 -07:00
Cyrus LeungandGitHub f690372b68 [Core] Update dtype detection and defaults (#14858) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-19 13:49:33 +08:00
SibiGitHubCyrus LeungAaron Pham
 		a73e183e36 [Misc] Replace os environ to monkeypatch in test suite (#14516) 
Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
 2025-03-16 20:35:57 -07:00
Cyrus LeungandGitHub 8a5a9b70d7 [CI/Build] Update defaults for test reproducibility (#14893) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-17 10:38:15 +08:00
Robert ShawGitHubrshaw@neuralmagic.com <rshaw@neuralmagic.com>Nicolò LucchesiTyler Michael SmithMichael Goin
 		d4d93db2c5 [V1] V1 Enablement Oracle (#13726) 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-03-14 22:02:20 -07:00
Ilya LavrenovandGitHub 8ca7a71df7 OpenVINO: added CPU-like conditions (#14338) 
Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
 2025-03-06 22:24:49 -08:00
Harry MellorandGitHub cf069aa8aa Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Isotr0pyandGitHub edf309ebbe [VLM] Support multimodal inputs for Florence-2 models (#13320) 2025-02-27 02:06:41 -08:00
Kevin H. LuuandGitHub 2c5e637b57 [ci] Use env var to control whether to use S3 bucket in CI (#13634) 2025-02-22 19:19:45 -08:00
Kevin H. LuuandGitHub a64a84433d [2/n][ci] S3: Use full model path (#13564) 
Signed-off-by: <>
 2025-02-20 01:20:15 -08:00
Kevin H. LuuandGitHub aa1e62d0db [ci] Fix spec decode test (#13600) 2025-02-20 16:56:00 +08:00
Kevin H. LuuGitHubEC2 Default User
 		473f51cfd9 [3/n][CI] Load Quantization test models with S3 (#13570) 
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
 2025-02-20 10:12:30 +08:00
Kevin H. LuuGitHubEC2 Default User
 		d5d214ac7f [1/n][CI] Load models in CI from S3 instead of HF (#13205) 
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
 2025-02-19 07:34:59 +00:00
Sumit VijandGitHub d88506dda4 [Model] LoRA Support for Ultravox model (#11253) 2025-02-05 19:54:13 -08:00
Russell BryantandGitHub e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Cyrus LeungandGitHub 59a0192fb9 [Core] Interface for accessing model from VllmRunner (#10353) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-20 15:00:59 +08:00
youkaichaoandGitHub 241ad7b301 [ci] Fix sampler tests (#11922) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-10 20:45:33 +08:00
Woosuk KwonandGitHub 73001445fb [V1] Implement Cascade Attention (#11635) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-01-01 21:56:46 +09:00
youkaichaoandGitHub b12e87f942 [platforms] enable platform plugins (#11602) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-30 20:24:45 +08:00
Cyrus LeungandGitHub eeec9e3390 [Frontend] Separate pooling APIs in offline inference (#11129) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-13 10:40:07 +00:00
Tyler Michael SmithandGitHub 28b3a1c7e5 [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-12-10 06:28:14 +00:00
zhou fanGitHubIsotr0py
 		ef31eabc68 [Model]: add some tests for aria model (#10770) 
Signed-off-by: xffxff <1247714429@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2024-12-02 05:36:36 +00:00
Cyrus LeungandGitHub 133707123e [Model] Replace embedding models with pooling adapter (#10769) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-01 08:02:54 +08:00
Wallas HenriqueGitHubMichael Goin
 		c27df94e1f [Bugfix] Fix chunked prefill with model dtype float32 on Turing Devices (#9850) 
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2024-11-25 12:23:32 -05:00
Maximilien de BayserGitHubFlavia Beo
 		214efc2c3c Support Cross encoder models (#10400) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-24 18:56:20 -08:00
Austin VeselkaGitHubFurtherAI
 		1b886aa104 [Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944) 
Signed-off-by: FurtherAI <austin.veselka@lighton.ai>
Co-authored-by: FurtherAI <austin.veselka@lighton.ai>
 2024-11-13 08:28:13 +00:00
Joe RundeandGitHub d58268c56a [V1] Make v1 more testable (#9888) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-11-06 11:57:35 -08:00
Aaron PhamandGitHub 21063c11c7 [CI/Build] drop support for Python 3.8 EOL (#8464) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2024-11-06 07:11:55 +00:00
Alex BrooksandGitHub cc98f1e079 [CI/Build] VLM Test Consolidation (#9372) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2024-10-30 09:32:17 -07:00
kakao-kevin-usGitHubKevin-Yang
 		6650e6a930 [Model] Add classification Task with Qwen2ForSequenceClassification (#9704) 
Signed-off-by: Kevin-Yang <ykcha9@gmail.com>
Co-authored-by: Kevin-Yang <ykcha9@gmail.com>
 2024-10-26 17:53:35 +00:00
Cyrus LeungandGitHub c866e0079d [CI/Build] Fix VLM test failures when using transformers v4.46 (#9666) 2024-10-25 01:40:40 +08:00