karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 2 Packages Activity
8,162 Commits 354 Branches 175 Tags
3e36fcbee642f41278a4881c9e2bfbbd7c28e607
Commit Graph
95 Commits
Author SHA1 Message Date
Peter PanandGitHub 533db0935d [benchmark] add max-concurrency in result table (#21095) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
 2025-07-30 01:15:43 -07:00
rongfu.lengandGitHub 18cc33dd60 [bugfix] fix profile impact benchmark results (#21507) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-07-27 22:44:24 -07:00
Ye (Charlotte) QiandGitHub e7c4f9ee86 [CI/Build][Doc] Move existing benchmark scripts in CI/document/example to vllm bench CLI (#21355) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-07-26 07:10:14 -07:00
Jialin OuyangandGitHub 10904e6d75 [benchmark] Port benchmark request sent optimization to benchmark_serving (#21209) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-07-22 05:28:00 -07:00
Li WangandGitHub 9ff2af6d2b [Benchmark] Parameterization of streaming loading of multimodal datasets (#20528) 
Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-07-09 13:35:16 +00:00
KebeandGitHub b1c1fe35a5 [Misc] remove redundant char (#20287) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-07-01 15:33:22 +08:00
d.transposedGitHubdtransposedRoger Wang
 		c635c5f744 [Misc][Benchmarking] Add variable request-rate ("ramp-up") to the benchmarking client. (#19423) 
Signed-off-by: dtransposed <damian@damian-ml-machine.europe-west3-b.c.jetbrains-grazie.internal>
Co-authored-by: dtransposed <damian@damian-ml-machine.europe-west3-b.c.jetbrains-grazie.internal>
Co-authored-by: Roger Wang <hey@rogerw.me>
 2025-06-24 18:41:49 +00:00
ReidGitHubreidliu41
 		6fa718a460 [Misc] Modularize CLI Argument Parsing in Benchmark Scripts (#19593) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-06-14 16:54:52 +08:00
Simon MoandGitHub 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Ekagra RanjanandGitHub bbfa0c61d1 [Misc][Benchmark] Add support for CustomDataset (#18511) 2025-05-31 19:07:38 +00:00
Duyi-WangandGitHub b169d5f7b6 [Misc][Tools][Benchmark] Add benchmark_serving supports for llama.cpp. (#18692) 
Signed-off-by: Duyi-Wang <duyi.wang@intel.com>
 2025-05-29 20:02:08 +08:00
Harry MellorandGitHub 009d9e7590 Convert benchmarks to ruff format (#18068) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-13 13:43:29 +00:00
d.transposedGitHubdtransposed
 		d456aea71f [Misc] Add Next Edit Prediction (NEP) datasets support in benchmark_serving.py (#16839) 
Signed-off-by: dtransposed <damian@damian-ml-machine.europe-west3-b.c.jetbrains-grazie.internal>
Signed-off-by: dtransposed <>
Co-authored-by: dtransposed <damian@damian-ml-machine.europe-west3-b.c.jetbrains-grazie.internal>
 2025-05-06 15:38:45 -04:00
Ekagra RanjanandGitHub cfe4532093 [Benchmark] Add single turn MTBench to Serving Bench (#17202) 2025-04-28 16:46:15 -07:00
Lucas WilkinsonandGitHub 881f735827 [Misc] Benchmark Serving Script Support Appending Results (#17028) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-04-24 22:53:55 -07:00
Nicolò LucchesiandGitHub 9d4ca19d50 [Misc] Benchmarks for audio models (#16505) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-04-19 02:24:14 -07:00
WWWandGitHub 268c325078 Fix range_ratio Bug in RandomDataset (#16126) 
Signed-off-by: jadewang21 <jadewangcn@outlook.com>
 2025-04-10 15:31:17 -07:00
lookandGitHub 7cd0bd7212 [Bugfix] Fix output token length check logic (#16419) 
Signed-off-by: look <eeslook@163.com>
 2025-04-10 20:16:48 +00:00
yihongandGitHub 04149cce27 [BugFix] fix some typos found by typos. (#16314) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-04-09 03:43:59 -07:00
Hyesoo YangandGitHub ba10801961 [Benchmark] Add sampling parameters to benchmark_serving. (#16022) 
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
 2025-04-06 12:30:35 +08:00
Ziji Shi (Steven)andGitHub 06f21ce7a5 [Benchmark] Add AIMO Dataset to Benchmark (#15955) 
Signed-off-by: Ziji Shi <shi.ziji.sm@gmail.com>
Signed-off-by: StevenShi-23 <shi.ziji.sm@gmail.com>
 2025-04-03 06:09:18 +00:00
Li WangGitHubRoger Wang
 		aa557e6422 [Benchmark]Fix error message (#15866) 
Signed-off-by: wangli <wangli858794774@gmail.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2025-04-02 01:32:24 -07:00
Jennifer ZhaoandGitHub effc5d24fa [Benchmark] Update Vision Arena Dataset and HuggingFaceDataset Setup (#15748) 
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
 2025-03-31 15:38:58 +08:00
Woosuk KwonandGitHub 70e132244a [Minor] Remove TGI launching script (#15646) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-28 09:30:08 -07:00
Chen XiaandGitHub e7f720ea56 [Misc]add coding benchmark for speculative decoding (#15303) 
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
 2025-03-28 10:47:05 +08:00
Simon MoandGitHub 583a9778e0 [Benchmark] Do not save detailed info to json by default (#14879) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-03-16 21:48:11 -07:00
Jennifer ZhaoGitHubJennifer ZhaoRoger Wang
 		1253b15774 [Feature] Consolidate performance benchmark datasets (#14036) 
Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-03-10 07:23:11 +00:00
Daniel LiandGitHub ad60bbb2b2 [Doc] Fix a typo (#14385) 2025-03-06 16:31:52 -08:00
VincentGitHubBrayden ZhongHarry Mellor
 		a4f1ee35d6 Deprecate best_of Sampling Parameter in anticipation for vLLM V1 (#13997) 
Signed-off-by: vincent-4 <vincentzhongy+githubvincent4@gmail.com>
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-05 20:22:43 +00:00
Harry MellorandGitHub cf069aa8aa Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Huy DoandGitHub e7ef74e26e Fix some issues with benchmark data output (#13641) 
Signed-off-by: Huy Do <huydhn@gmail.com>
 2025-02-24 10:23:18 +08:00
Roger WangandGitHub 9bebc9512f [Misc] Deprecate --dataset from benchmark_serving.py (#13708) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-02-23 13:32:20 +00:00
Huy DoandGitHub 45186834a0 Run v1 benchmark and integrate with PyTorch OSS benchmark database (#13068) 
Signed-off-by: Huy Do <huydhn@gmail.com>
 2025-02-17 08:16:32 +00:00
Keyun TongandGitHub 3ee696a63d [RFC][vllm-API] Support tokenizer registry for customized tokenizer in vLLM (#12518) 
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
 2025-02-12 12:25:58 +08:00
Woosuk KwonGitHubRoger Wang
 		58047c6f04 [Benchmark] Add BurstGPT to benchmark_serving (#13063) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2025-02-10 21:25:30 -08:00
Varun Sundar RabindranathGitHubVarun Sundar Rabindranath
 		7e1837676a [misc] Add LoRA to benchmark_serving (#12898) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-02-08 17:15:44 +08:00
Russell BryantandGitHub e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Harry MellorandGitHub 823ab79633 Update pre-commit hooks (#12475) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-27 17:23:08 -07:00
Roger WangandGitHub 3c818bdb42 [Misc] Use VisionArena Dataset for VLM Benchmarking (#12389) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-01-24 00:22:04 -08:00
Nick HillandGitHub 222a9dc350 [Benchmark] More accurate TPOT calc in benchmark_serving.py (#12288) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-01-22 13:46:14 +08:00
gujingGitHubRoger Wang
 		936db119ed benchmark_serving support --served-model-name param (#12109) 
Signed-off-by: zibai <zibai.gj@alibaba-inc.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2025-01-19 09:59:56 +00:00
Alexander MatveevandGitHub 238c0d93b4 [Misc] Add tokenizer_mode param to benchmark_serving.py (#11174) 
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
 2024-12-13 16:19:10 +00:00
Roger WangGitHubChen ZhangIsotr0py
 		c11f172187 [Misc] Adding MMMU-Pro vision dataset to serving benchmark (#10804) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2024-12-01 08:47:05 +00:00
Jaehyun AnandGitHub 8b6725b0cf [Misc] Update benchmark to support image_url file or http (#10287) 
Signed-off-by: rbbang <anjaehyun87@gmail.com>
 2024-11-16 18:15:40 +08:00
AtlasGitHubRoger Wang
 		a62bc0109c [Misc] Add Gamma-Distribution Request Generation Support for Serving Benchmark. (#10105) 
Signed-off-by: Mozhou <spli161006@gmail.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2024-11-07 11:20:30 +00:00
Tran Quang DaiandGitHub ea4adeddc1 [Bugfix] Fix E2EL mean and median stats (#9984) 
Signed-off-by: daitran2k1 <tranquangdai7a@gmail.com>
 2024-11-04 09:37:58 +00:00
Andy DaiandGitHub 855e0e6f97 [Frontend][Misc] Goodput metric support (#9338) 2024-10-20 18:39:32 +00:00
Russell BryantandGitHub 7dbe738d65 [Misc] benchmark: Add option to set max concurrency (#9390) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-10-18 11:15:28 -07:00
Kai WuGitHubIsotr0py
 		d65049daab [Bugfix] Add random_seed to sample_hf_requests in benchmark_serving script (#9013) 
Co-authored-by: Isotr0py <2037008807@qq.com>
 2024-10-17 21:11:11 +00:00
Grace HoandGitHub 5d264f4ab8 pass ignore_eos parameter to all benchmark_serving calls (#9349) 2024-10-15 13:30:44 -07:00