ShuffleInfer: Disaggregate LLM Inference for Mixed Downstream Workloads.

Sun, 01 Jun 2025 01:00:00 +0200

Cunchen Hu , Heyang Huang , Liangliang Xu , Xusheng Chen , Chenxi Wang , Jiang Xu , Shuang Chen , Hao Feng , Sa Wang , Yungang Bao , Ninghui Sun , Yizhou Shan :
ShuffleInfer: Disaggregate LLM Inference for Mixed Downstream Workloads. ACM Trans. Archit. Code Optim. 22(2): 77:1-77:24 (2025)