karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
82 Commits 354 Branches 175 Tags
e9d3f2ff7772c8efe41dc805cec71c223ec18ec8
Commit Graph
6 Commits
Author SHA1 Message Date
Woosuk KwonandGitHub e9d3f2ff77 Add memory analyzer & utomatically configure KV cache size (#6) 2023-03-11 23:23:14 -08:00
Woosuk KwonandGitHub 1a7eb7da61 Support beam search & parallel generation (#7) 2023-03-10 09:58:21 -08:00
Woosuk KwonandGitHub 04e5acc08e Fix a bug in 1D input shape (#5) 2023-03-06 10:05:27 -08:00
Woosuk KwonandGitHub 3e9f991d6a Use FlashAttention for multi_query_kv_attention (#4) 2023-03-01 21:13:08 -08:00
Woosuk Kwon fa16389a2e Clean up the server script 2023-02-24 11:56:21 +00:00
Woosuk Kwon afdbe5d373 [WIP] Add server script 2023-02-24 01:33:37 +00:00