The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Author: Qing Zheng (Los Alamos National Laboratory (LANL)), Brian Atkinson (Los Alamos National Laboratory (LANL)), Jason Lee (Los Alamos National Laboratory (LANL)), Gary Grider (Los Alamos National Laboratory (LANL))
Supervisor: ADVISOR_NAMES_AFFS
Abstract: As HPC simulations generate ever-larger datasets, reducing the volume of data that must be loaded into compute node memory for analysis has become essential for unlocking insights efficiently. In-storage analysis achieves this by processing data directly at the storage servers, allowing them to return only compact results that match regions of interest instead of raw datasets that may be orders of magnitude larger—significantly reducing data footprint.
This e-poster showcases a novel compute-near-storage architecture based on pNFS that enables secure in-storage analysis of scientific data with industry-standard software, including Arrow, Parquet, Substrait, and DuckDB. Using a real-world asteroid impact dataset, we present a live demonstration of a VTK visualization pipeline modified to offload analysis to pNFS data servers, tracing the aftermath of the impact over time and rendering the results on the client as 3D visuals. We show substantial data reduction by pushing down analysis and transmitting only insight-relevant information.
Poster: PDF
e-Poster: MP4
Poster Summary: PDF