Research and ACM SRC Posters Archive

Understanding LLM Behavior on HPC Data via Mechanistic Interpretability

Poster Type: Research Posters

Author: Md Mahbubur Rahman (Iowa State University), Arjun Guha (Northeastern University), Harshitha Menon (Lawrence Livermore National Laboratory (LLNL))

Supervisor:

Abstract: Large language models (LLMs) are increasingly used in HPC for tasks like code generation and analysis, but their internal reasoning remains opaque. To address this, we study three tasks—OpenMP code completion, data race detection, and OMP code generation—using mechanistic interpretability. Sparse autoencoder ablations reveal causal features, function vector injection improves zero-shot predictions and direction vector shifts the model's output toward a desired behavior or style, even without explicitly stating it in the prompt. These methods expose and influence LLM behavior in HPC contexts.

Best Poster Finalist (BP): no
Poster: PDF
Poster Summary: PDF

Back to Poster Archive Listing