Workshops Archive

Do Large Language Models Speak Scientific Workflows?

Workshop: WORKS 2025: 20th Workshop on Workflows in Support of Large-Scale Science

Authors: Orcun Yildiz and Tom Peterka (Argonne National Laboratory (ANL))

Abstract: In this work, we conduct an experimental study to explore applicability of LLMs for configuring, annotating, and translating scientific workflows. We use three different workflow-specific experiments and evaluate several open- and closed-source language models using state-of-the-art workflow systems. Our studies reveal that LLMs often struggle due to a lack of training data for scientific workflows. We further observe that the performance of LLMs varies across experiments and workflow systems. We discuss the implications of our findings and draw attention to several approaches extending LLM capabilities for scientific workflows. Our findings can help workflow developers and users in understanding LLM capabilities in scientific workflows, and motivate further research applying LLMs to workflows.

Back to WORKS 2025: 20th Workshop on Workflows in Support of Large-Scale Science Archive Listing Back to Full Workshop Archive Listing