The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Authors: Christopher Simmons (Cambridge Computer), Adrian Reber (Red Hat), Jonathon Anderson (CIQ), Travis Cotton (Los Alamos National Laboratory (LANL))
Abstract: The landscape of HPC cluster provisioning is rapidly evolving, with innovative open-source solutions emerging to meet modern computational demands. This BoF showcases leading open-source provisioning platforms through lightning talks from the Warewulf, Confluent, and OpenCHAMI communities, preceded by an update on the OpenHPC project. The session will highlight recent advances in container-based provisioning, cloud-native HPC management, and security-focused deployment strategies. Community members will engage in interactive discussions about best practices, interoperability challenges, and future collaboration opportunities. This forum aims to strengthen the open-source provisioning ecosystem and foster cross-project innovation for next-generation HPC infrastructure.
Long Description: High-performance computing infrastructure deployment and management remain critical challenges for the global HPC community. As computational workloads become increasingly diverse, spanning traditional scientific computing and AI/ML applications, while supporting constantly evolving hardware, including accelerators, the need for flexible, secure, and scalable provisioning solutions has never been greater. Open-source provisioning systems have emerged as essential tools that enable HPC centers to deploy and manage clusters efficiently while maintaining the flexibility to adapt to rapidly changing technology landscapes.
This BoF brings together developers, system administrators, and users of leading open-source provisioning platforms to share recent innovations, discuss current challenges, and explore future collaboration opportunities. The session will provide the HPC community with insights into the latest developments across multiple provisioning ecosystems, fostering knowledge transfer and cross-project innovation.
The session begins with a brief update on the OpenHPC project, highlighting recent releases, community growth, and integration efforts with various provisioning systems. The OpenHPC project provides a community-driven stack of common ingredients for deploying and managing Linux-based HPC clusters. Formed in November 2015 and formalized as a Linux Foundation project in June 2016, OpenHPC continues to experience rapid growth in its user community, with 1,200 participants in our Slack workspace, 1,400 subscribers to our email lists, and over 100,000 unique accesses to our repositories annually. It is used by thousands of organizations worldwide, including academic institutes, non-profit organizations, government labs, and commercial entities. This BoF proposal follows seven previously successful OpenHPC-related BoFs held at SC16, SC17, SC18, SC19, SC21, SC22, and SC24.
Following the brief OpenHPC project update, three focused lightning talks will showcase cutting-edge developments:
1. Warewulf v4.6 and Beyond: Representatives from the Warewulf project will present the latest major release series, featuring a more streamlined upgrade experience, enhanced image and kernel management, significant improvements to the configuration overlay system, expanding OS support to Debian and Ubuntu, a new REST API, and the ability to provision to local storage. They’ll also present the near-term roadmap for what’s coming in the current and next release series.
2. Confluent: Spiritual successor to xCAT with emphasis on improved ease of use, security, and interoperability. There will be an overview of automated onboarding, hardware management, and OS deployment, including scripted installation, diskless installation, and cloning.
3. OpenCHAMI: Cloud-Native HPC Provisioning: As the newest addition to the High-Performance Software Foundation (HPSF), OpenCHAMI representatives will present their microservices-based approach to HPC cluster management, highlighting security-first design principles, rapid deployment capabilities, and production experiences.
Interactive Community Engagement:
The final 30 minutes focus on community-driven discussion addressing key topics:
- Interoperability challenges and standardization opportunities
- Security best practices for modern HPC environments
- Future collaboration potential across projects
- Community feedback and feature requests
Website: https://openhpc.community/