Workshops Archive

Lightning Talk: Adaptive In-Situ Sampling Framework for Accelerated Online Change Point Detection in Scientific Workflows

Workshop: ISAV25: In Situ AI, Analysis, and Visualization

Authors: Vijayalakshmi Saravanan (University of Texas at Tyler, UTD); Sai Karthik Navuluru (UTD); Seetharam Vanam Rao (UTT); Tamil Lakshman (UTD); and Khaled Z Ibrahim (Lawrence Berkeley National Laboratory (LBNL))

Abstract: High-performance computing (HPC) systems at the Exascale generate monitoring and simulation data at rates that exceed the capacity of traditional offline analysis, especially for change point detection (CPD) in large-scale scientific workflows. We present an adaptive in-situ sampling framework that combines truncated singular value decomposition (SVD)-based manifold learning with the Kernel Cumulative Sum (KCUSUM) method for statistical CPD. Implemented with MPI4py for scalable interprocess communication and ADIOS2 for high-throughput streaming I/O, the framework dynamically adjusts sampling rates in real time based on anomaly likelihood, enabling selective data retention without sacrificing scientific information. Evaluations on synthetic datasets and large-scale molecular dynamics (MD) simulations from NWChem demonstrate up to 59% memory reduction, sub-second detection latency for critical events, and near-perfect detection accuracy, all while eliminating the need for storing full trajectories. These preliminary results highlight the framework’s potential to deliver resource-efficient, real-time anomaly detection in data-intensive Exascale scientific computing environments,

Back to ISAV25: In Situ AI, Analysis, and Visualization Archive Listing Back to Full Workshop Archive Listing