Workshops Archive

LLM Agents for Interactive Workflow Provenance: Reference Architecture and Evaluation Methodology

Workshop: WORKS 2025: 20th Workshop on Workflows in Support of Large-Scale Science

Authors: Renan Souza, Timothy Poteet, Brian Etz, and Daniel Rosendo (Oak Ridge National Lab.); Amal Gueroudji (Argonne National Laboratory (ANL)); and Woong Shin, Prasanna Balaprakash, and Rafael Ferreira da Silva (Oak Ridge National Lab.)

Abstract: Modern scientific discovery increasingly relies on workflows that process data across the Edge, Cloud, and High Performance Computing (HPC) continuum. Comprehensive and in-depth analyses of these data are critical for hypothesis validation, anomaly detection, reproducibility, and impactful findings. Although workflow provenance techniques support such analyses, at large scale, the provenance data become complex and difficult to analyze. Existing systems depend on custom scripts, structured queries, or static dashboards, limiting data interaction. In this work, we introduce an evaluation methodology, reference architecture, and open-source implementation that leverages interactive Large Language Model (LLM) agents for runtime data analysis. Our approach uses a lightweight, metadata-driven design that translates natural language into structured provenance queries. Evaluations across LLaMA, GPT, Gemini, and Claude, covering diverse query classes and a real-world chemistry workflow, show that modular design, prompt tuning, and Retrieval-Augmented Generation (RAG) enable accurate and insightful LLM agent responses beyond recorded provenance.

Back to WORKS 2025: 20th Workshop on Workflows in Support of Large-Scale Science Archive Listing Back to Full Workshop Archive Listing