The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Poster Type: ACM Student Research Competition, Undergraduate
Author: Paul Jiang (Purdue University), Vivian Zheng (Stony Brook University)
Supervisor: Ganesh Gopalakrishnan (University of Utah)
Abstract: Modern high performance computing increasingly relies on hardware accelerators like NVIDIA Tensor Cores, which employ non-standard internal arithmetic that can evolve between hardware generations. This non-standard approach can violate the fundamental mathematical property of monotonicity, leading to incorrect outputs where adding a larger number produces a smaller result. To address this, we introduce a formal framework using satisfiability modulo theories to analyze the hardware design space by systematically varying hardware features (e.g., number of terms [𝑛], internal padding bits [𝑝]) within a custom bitvector encoding. We derive a precise condition for guaranteed monotonicity, proving that non-monotonicity can only occur when 𝑝 ≤ ⌊log2 (𝑛 − 1) − 2⌋. We also derive a formula for the maximum magnitude of error when non-monotonicity can occur. Our results provide hardware architects with provably correct design parameters to eliminate such anomalies, ensuring greater numerical stability.
Best Poster Finalist (BP): no
Poster: PDF
Poster Summary: PDF