Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads.

Mon, 01 Jan 2024 00:00:00 +0100

Cunchen Hu, Heyang Huang, Liangliang Xu, Xusheng Chen, Jiang Xu, Shuang Chen, Hao Feng, Chenxi Wang, Sa Wang, Yungang Bao, Ninghui Sun, Yizhou Shan:
Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads. CoRR abs/2401.11181 (2024)