The infrastructure powering IBM's Gen AI model development
Abstract
AI Infrastructure plays a key role in the speed and cost-competitiveness of developing and deploying advanced AI models. The current demand for powerful AI infrastructure for model training is driven by the emergence of generative AI and foundational models, where on occasion thousands of GPUs must cooperate on a single training job for the model to be trained in a reasonable time. Delivering efficient and high-performing AI training requires an end-to-end solution that combines hardware, software and holistic telemetry to cater for multiple types of AI workloads. In this report, we describe IBM's hybrid cloud infrastructure that powers our generative AI model development. This infrastructure includes (1) Vela: an AI-optimized supercomputing capability directly integrated into the IBM Cloud, delivering scalable, dynamic, multi-tenant and geographically distributed infrastructure for large-scale model training and other AI workflow steps and (2) Blue Vela: a large-scale, purpose-built, on-premises hosting environment that is optimized to support our largest and most ambitious AI model training tasks. Vela provides IBM with the dual benefit of high performance for internal use along with the flexibility to adapt to an evolving commercial landscape. Blue Vela provides us with the benefits of rapid development of our largest and most ambitious models, as well as future-proofing against the evolving model landscape in the industry. Taken together, they provide IBM with the ability to rapidly innovate in the development of both AI models and commercial offerings.
Cite
@article{arxiv.2407.05467,
title = {The infrastructure powering IBM's Gen AI model development},
author = {Talia Gershon and Seetharami Seelam and Brian Belgodere and Milton Bonilla and Lan Hoang and Danny Barnett and I-Hsin Chung and Apoorve Mohan and Ming-Hung Chen and Lixiang Luo and Robert Walkup and Constantinos Evangelinos and Shweta Salaria and Marc Dombrowa and Yoonho Park and Apo Kayi and Liran Schour and Alim Alim and Ali Sydney and Pavlos Maniotis and Laurent Schares and Bernard Metzler and Bengi Karacali-Akyamac and Sophia Wen and Tatsuhiro Chiba and Sunyanan Choochotkaew and Takeshi Yoshimura and Claudia Misale and Tonia Elengikal and Kevin O Connor and Zhuoran Liu and Richard Molina and Lars Schneidenbach and James Caden and Christopher Laibinis and Carlos Fonseca and Vasily Tarasov and Swaminathan Sundararaman and Frank Schmuck and Scott Guthridge and Jeremy Cohn and Marc Eshel and Paul Muench and Runyu Liu and William Pointer and Drew Wyskida and Bob Krull and Ray Rose and Brent Wolfe and William Cornejo and John Walter and Colm Malone and Clifford Perucci and Frank Franco and Nigel Hinds and Bob Calio and Pavel Druyan and Robert Kilduff and John Kienle and Connor McStay and Andrew Figueroa and Matthew Connolly and Edie Fost and Gina Roma and Jake Fonseca and Ido Levy and Michele Payne and Ryan Schenkel and Amir Malki and Lion Schneider and Aniruddha Narkhede and Shekeba Moshref and Alexandra Kisin and Olga Dodin and Bill Rippon and Henry Wrieth and John Ganci and Johnny Colino and Donna Habeger-Rose and Rakesh Pandey and Aditya Gidh and Aditya Gaur and Dennis Patterson and Samsuddin Salmani and Rambilas Varma and Rumana Rumana and Shubham Sharma and Aditya Gaur and Mayank Mishra and Rameswar Panda and Aditya Prasad and Matt Stallone and Gaoyuan Zhang and Yikang Shen and David Cox and Ruchir Puri and Dakshi Agrawal and Drew Thorstensen and Joel Belog and Brent Tang and Saurabh Kumar Gupta and Amitabha Biswas and Anup Maheshwari and Eran Gampel and Jason Van Patten and Matthew Runion and Sai Kaki and Yigal Bogin and Brian Reitz and Steve Pritko and Shahan Najam and Surya Nambala and Radhika Chirra and Rick Welp and Frank DiMitri and Felipe Telles and Amilcar Arvelo and King Chu and Ed Seminaro and Andrew Schram and Felix Eickhoff and William Hanson and Eric Mckeever and Michael Light and Dinakaran Joseph and Piyush Chaudhary and Piyush Shivam and Puneet Chaudhary and Wesley Jones and Robert Guthrie and Chris Bostic and Rezaul Islam and Steve Duersch and Wayne Sawdon and John Lewars and Matthew Klos and Michael Spriggs and Bill McMillan and George Gao and Ashish Kamra and Gaurav Singh and Marc Curry and Tushar Katarki and Joe Talerico and Zenghui Shi and Sai Sindhur Malleni and Erwan Gallen},
journal= {arXiv preprint arXiv:2407.05467},
year = {2025}
}
Comments
Corresponding Authors: Talia Gershon, Seetharami Seelam,Brian Belgodere, Milton Bonilla