The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: International Workshop on RESource DISaggregation in High Performance Computing (RESDIS)
Authors: Kyuli Park, Sungmin Yoon, Farid Talibli, and Sungyong Park (Sogang University, South Korea); Jae-Hyuck Kwak and Kimoon Jeong (Korea Institute of Science and Technology Information); Awais Khan (Oak Ridge National Laboratory); and Youngjae Kim (Sogang University, South Korea)
Abstract: Ceph is a widely used distributed object store, but its messenger layer imposes substantial CPU overhead on the host. To address this limitation, we propose DoCeph, a DPU-offloaded storage architecture for Ceph that disaggregates the system by offloading the communication-intensive messaging component to the DPU while retaining the storage backend on the host. The DPU efficiently manages communication, using lightweight RPC for metadata operations and DMA for data transfer. Moreover, DoCeph introduces a pipelining technique that overlaps data transmission with buffer preparation, mitigating hardware-imposed transfer size limitations. We implemented DoCeph on a Ceph cluster with NVIDIA BlueField-3 DPUs. Evaluation results indicate that DoCeph cuts host CPU usage by up to 92% while sustaining stable throughput and providing larger performance benefits for object writes over 1 MB.
Back to International Workshop on RESource DISaggregation in High Performance Computing (RESDIS) Archive Listing Back to Full Workshop Archive Listing