The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Authors: Dan Stanzione (The University of Texas at Austin), Chau-Lyan Chang (National Center for High-Performance Computing (NCHC), Taiwan), Jason Haga (National Institute of Advanced Industrial Science and Technology (AIST), Japan), Frank Würthwein (San Diego Supercomputer Center (SDSC)), Venkatram Vishwanath (Argonne National Laboratory (ANL)), Thomas Hauser (National Center for Atmospheric Research (NCAR)), Weicheng Huang (NCHC)
Abstract: This BoF explores the emerging role of AI inference services in the academic HPC community. Participants will share use cases, best practices, and strategies for deploying inference across research, education, and operations, including specialized applications such as indigenous language models. Through lightning talks, live polls, and interactive panel discussions, the session aims to identify shared challenges, opportunities for service sharing, and the academic value proposition of inference in science. The ultimate goal is to foster a cross-institutional “community of practice” among global academic and governmental HPC centers.
Long Description: AI inferencing capability is increasingly important inside academia for both inference and training. While options from industry are available, resources for AI development in academia are substantially smaller than their industrial counterparts. While AI training can be relatively simply supported by the batch-scheduled GPU-enabled clusters in many HPC centers, inference workflows present new challenges, from finding cost-effective resources to scaling up workflows including surrogate models coupled to simulation, to the growing demand for persistent “ChatBot” style services to support coding, user support, data exploration, etc. Sharing use cases, best practices, software stacks, and perhaps even services across academic organizations will help promote the adoption of inference services, especially for AI research of value to the broader research ecosystem and establishing value propositions for inferencing in academia.
This BoF session, organized by a number of leading academic and government HPC centers from the Asia, Europe, and the US, will be the first step in bringing together a “community of practice” among inference service providers within the research and education ecosystem.
During the session, topics that will be discussed include:
• Use cases for inference within research, teaching, and operations for academic Community.
o What services are best provided by industry vs. academia?
o Differences and Commonalities among uses at institutions currently.
o Characterizing any broad patterns in inferencing for scientific services.
o Specialized/Unique AI model and inferencing service, for example, inference services for indigenous languages
• Best practices in provisioning and providing hardware and software stacks for inference, including support for specialized inferencing infrastructure.
• Open-weight models vs closed models for science. What’s unique for our facilities is the need to support a wide range of models for science.
• Integration of Inference simulations within scientific simulation.
o Better utilization of inference service for AI training (i.e. using inference output as input to training).
o Persistently provided or dynamically co-scheduled?
• Possibilities and Barriers for integration/sharing of inference services across organizations
o Will dynamic scaling be possible, especially across multiple organizations?
o Data and Privacy
The BoF will begin with presentations discussing how inferencing is being used at various participating centers, followed by a QA session and a wide-ranging group discussion. Each presentation will last no more than 3 minutes, thus to leave more time to interact with participants and to build the consensus which will lay the foundation of the academia community.
Expected Outcomes:
The dual goals of the BoF are to build a community and identify use cases and best practices. To this end, we will produce a report, to be published online (e.g. in Arxiv) on the discussion at the BoF, and a Slack channel will be created to foster an online community beyond the BoF towards building a Community of Practice.