Microsoft has announced a new supercomputer built by the company in partnership with and for OpenAI to train large-scale AI models. The Seattle-based company said at its Build conference for developers that this is one of the top five publically disclosed supercomputers.
Last year, Microsoft invested $1 billion in OpenAI with a goal to build supercomputers that leverages Azure’s cloud services. Now, that investment is showing its first results.
In terms of specifications, the new supercomputer packs some impressive numbers. It a single system with more than 285,000 CPU cores, 10,000 GPUs, and 400 gigabits per second of network connectivity for each GPU server.
Both companies said they want to build multitasking large-scale AI models to make them available for developers and the public to use. Microsoft said that the end goal is to make its large AI models, training optimization tools, and supercomputing resources available through Azure AI services and GitHub for researchers.
Earlier this year, the tech giant unveiled a 17-billion parameter natural language generation model. With this supercomputer and its AI at scale program, Mircosoft aims to create more such generic models and open-source it.
Microsoft’s Chief Technical Officer, Kevin Scott, said this is about training a model that learns to do a general set of things and making it available for developers:
By now most people intuitively understand how personal computers are a platform — you buy one and it’s not like everything the computer is ever going to do is built into the device when you pull it out of the box.
That’s exactly what we mean when we say AI is becoming a platform. This is about taking a very broad set of data and training a model that learns to do a general set of things and making that model available for millions of developers to go figure out how to do interesting and creative things with.
Since its inception in 2015, it has released some attention-grabbing projects including GPT-2 text generator and gamer bots that defeated champions in DOTA 2. The company hasn’t made it clear what it intends to do with the enormous potential of this new supercomputer.
Microsoft’s supercomputer on cloud aim is not unheard of. Amazon offers High-Performance Computing (HPC) through Amazon Web Services (AWS) already. There is no sign that Microsoft is going to make these solutions available for developers though.
Get the TNW newsletter
Get the most important tech news in your inbox each week.