Workshops Archive

Accelerating Advanced Light Source Science Through Multi-Facility HPC Workflows

Workshop: XLOOP 2025: The 7th Annual Workshop on Extreme-Scale Experiment-in-the-Loop Computing

Authors: David Abramov (Lawrence Berkeley National Laboratory (LBNL), ALS); Samuel Welborn (Lawrence Berkeley National Laboratory (LBNL), NERSC); Ryan Chard (Argonne National Laboratory (ANL)); Kuldeep Chawla (Lawrence Berkeley National Laboratory (LBNL), LBL IT); Xiaoya Chong and Elizabeth Clark (Lawrence Berkeley National Laboratory (LBNL), ALS); Bjoern Enders (Lawrence Berkeley National Laboratory (LBNL), NERSC); Alexander Hexemer, Jason Jed, and Wiebke Koepp (Lawrence Berkeley National Laboratory (LBNL), ALS); Harinarayan Krishnan (Lawrence Berkeley National Laboratory (LBNL), CAMERA); Seij De Leon and Dilworth Parkinson (Lawrence Berkeley National Laboratory (LBNL), ALS); David Perlmutter (Lawrence Berkeley National Laboratory (LBNL), CAMERA); Raja Vyshnavi Sriramoju (Lawrence Berkeley National Laboratory (LBNL), ALS); Thomas Uram (Argonne National Laboratory (ANL), ALCF); and Lee Lisheng Yang and Dylan McReynolds (Lawrence Berkeley National Laboratory (LBNL), ALS)

Abstract: Synchrotron light sources support a wide array of techniques to investigate materials, often producing complex, high-volume data that challenge traditional workflows. At the Advanced Light Source (ALS), we developed infrastructure to move microtomography data over ESnet to ALCF and NERSC, where CPU- and GPU-based algorithms generate 3D reconstructed volumes of experimental samples. We employ two data movement and reconstruction models: real-time processing as data streams directly to NERSC compute nodes, and automated file transfer to NERSC and ALCF file systems. The streaming pipeline provides users with feedback in under ten seconds, while the file-based workflow produces high-quality reconstructions suitable for deeper analysis in 20-30 minutes. This infrastructure allows users to leverage HPC resources without direct access to backend systems. We plan to extend this architecture to more endstations, supporting our beamline scientists and users.

Back to XLOOP 2025: The 7th Annual Workshop on Extreme-Scale Experiment-in-the-Loop Computing Archive Listing Back to Full Workshop Archive Listing