The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: PDSW'25: The 10th International Parallel Data Systems Workshop
Authors: Meng Tang (Illinois Institute of Technology); Zhaobin Zhu (Johannes Gutenberg University Mainz); Luanzheng Guo, James G. Bandy, and Tim Carlson (Pacific Northwest National Laboratory (PNNL)); Sarah Neuwirth (Johannes Gutenberg University Mainz); Anthony Kougkas and Xian-He Sun (Illinois Institute of Technology); and Nathan R. Tallent (Pacific Northwest National Laboratory (PNNL))
Abstract: High-performance computing facilities increasingly form hybrid environments that integrate cloud services. To avoid cumbersome network transfers when sharing data, a new class of storage gateways map a subset of facility storage to a cloud counterpart and automatically manage data mirroring. However, the performance characteristics of accessing AWS's S3 from HPC systems using different methods and patterns remains poorly understood. This paper presents a roofline-based analysis of three S3 integration approaches: NFS-mounted AWS Storage Gateway, data migration through Storage Gateway, and direct S3 API transfers. We extend I/O roofline modeling to characterize operational intensity and bandwidth ceilings across varying data sizes and access patterns. Our experimental evaluation demonstrates significant performance differences between access methods, with POSIX I/O on NFS Storage Gateway achieving up to 6.4× higher bandwidth than other approaches for large transfers. The roofline analysis reveals distinct characteristics for each method, enabling informed selection of S3 integration strategies.
Back to PDSW'25: The 10th International Parallel Data Systems Workshop Archive Listing Back to Full Workshop Archive Listing