karylab_agents/vllm

Fork 0

forked from Karylab-cklius/vllm

Files

T

History

88ed636218 [KV Connector]: Support KV push from Prefill to Decode node using Nixl KV Connector (#35264 )

Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>

2026-06-12 10:38:41 +00:00

disagg_proxy_demo.py

[Examples] Resettle Disaggregated examples. (#40759 )

2026-05-06 01:20:38 -07:00

disagg_proxy_multiturn.py

[Bench] benchmark_serving_multi_turn: make non-standard conversation_id payload opt-in (#43756 )

2026-06-09 19:41:56 -07:00

disagg_proxy_pushconnector_demo.py

[KV Connector]: Support KV push from Prefill to Decode node using Nixl KV Connector (#35264 )

2026-06-12 10:38:41 +00:00

example_mm_serve.py

[Examples] Resettle Disaggregated examples. (#40759 )

2026-05-06 01:20:38 -07:00

kv_events.sh

[Examples] Resettle Disaggregated examples. (#40759 )

2026-05-06 01:20:38 -07:00

moriio_toy_proxy_server.py

[ROCm] Profiler api support for ROCm MORI toy proxy server in PD Disaggregation (#40264 )

2026-05-07 16:58:38 +08:00

README.md

[Examples] Resettle Disaggregated examples. (#40759 )

2026-05-06 01:20:38 -07:00

README.md

Disaggregated Serving

This example contains scripts that demonstrate the disaggregated serving features of vLLM.

Files

disagg_proxy_demo.py - Demonstrates XpYd (X prefill instances, Y decode instances).
kv_events.sh - Demonstrates KV cache event publishing.
mooncake_connector - A proxy demo for MooncakeConnector.