Cerebras unveils world’s largest AI coaching supercomputer with 54M cores

Cerebras Techniques, the AI accelerator pioneer, and UAE-based know-how holding group G42, have unveiled the world’s largest supercomputer for AI coaching, named Condor Galaxy.

The community of 9 interconnected supercomputers guarantees to scale back AI mannequin coaching time considerably, with a complete capability of 36 exaFLOPs, due to the primary AI supercomputer on the community, Condor Galaxy 1 (CG-1), which has 4 exaFLOPs and 54 million cores, mentioned Andrew Feldman, CEO of Cerebras, in an interview with VentureBeat.

Moderately than make particular person chips for its centralized processing models (CPUs), Cerebras takes total silicon wafers and prints its cores on the wafers, that are the scale of pizza. These wafers have the equal of tons of of chips on a single wafer, with many cores on every wafer. And that’s how they get to 54 million cores in a single supercomputer.

In our interview, Feldman mentioned, “AI is not only consuming the U.S. AI is consuming the world. There’s an insatiable demand for compute. Fashions are proliferating. And knowledge is the brand new gold. That is the muse.”

With this supercomputer, you get outcomes twice as quick, utilizing half the vitality, mentioned Feldman.

“We’re the most important on the planet. We’ve offered it to an organization referred to as G42, primarily based in Abu Dhabi. We deployed it in Santa Clara, California and are at present working AI work,” Feldman mentioned. “We handle and function it by way of our cloud. It’s utilized by G42 for inside work and any extra capability is resold by them or by us. That is the primary of three U.S.-based supercomputers we intend to construct for them within the subsequent 12 months. And the primary 9, we intend to construct for them within the subsequent 18 months. And when these 9 are related, that shall be a 36 exaflop constellation of supercomputers.”

Condor Galaxy is the identify of the supercomputer, which scales from one to 32 CS-2 computer systems made attainable by the corporate’s Reminiscence X and Swarm X know-how. The machine was stood up in Santa Clara in 10 days and it’s already one of many largest supercomputers on the planet, Feldman mentioned.

The second machine shall be in Austin Texas and the third one shall be in Asheville, North Carolina. Part two’s deal worth is in extra of $100 million.

“It’s fairly loopy. After we’re accomplished, we may have 9 supercomputers, every of 4 exaFLOPs interconnected to create a distributed 36 exaFLOP AI constellation. That’s almost 500 million cores throughout 576 CS-2s with 3,490 terabytes of inside bandwidth. And we’ll want greater than half a billion AMD Epyc cores simply to feed us knowledge.”

Cerebras and G42 will deploy two extra such supercomputers, CG-2 and CG-3, within the U.S. in early 2024. With this unprecedented supercomputing community, they plan to revolutionize AI development globally.

Positioned in Santa Clara, California, CG-1 hyperlinks 64 Cerebras CS-2 methods collectively into an easy-to-use AI supercomputer with a coaching capability of 4 exaFLOPs, which is obtainable as a cloud service. CG-1 is designed to allow G42 and its cloud prospects to coach massive, ground-breaking fashions rapidly and simply, thereby accelerating innovation.

The Cerebras-G42 strategic partnership has already superior state-of-the-art AI fashions in Arabic bilingual chat, healthcare, and local weather research. CG-1 provides native help for coaching with lengthy sequence lengths, as much as 50,000 tokens out of the field, with none particular software program libraries. Feldman mentioned that programming CG-1 is finished fully with out complicated distributed programming languages, and even the most important fashions might be run with out weeks or months spent distributing work over 1000’s of GPUs.

The partnership between G42 and Cerebras delivers on all three parts required for coaching massive fashions: big quantities of compute, huge datasets, and specialised AI experience. They’re democratizing AI, enabling easy and quick access to the trade’s main AI compute, and G42’s work with numerous datasets throughout healthcare, vitality, and local weather research will allow customers of the methods to coach new cutting-edge foundational fashions.

Cerebras and G42 convey collectively a group of {hardware} engineers, knowledge engineers, AI scientists, and trade specialists to ship a full-service AI providing to resolve prospects’ issues. This mix will produce groundbreaking outcomes and turbocharge tons of of AI initiatives globally.

G42 is a conglomerate in Abu Dhabi with 22,000 staff throughout 9 firms in 25 nations.

“Now, if you wish to run a similar mannequin with 40 billion parameters on 1,000, GPUs, you need to write an extra 27,215 strains of code. Clearly, that’s not simple,” Feldman mentioned. “Now, Cerebras with a 1 billion parameter mannequin takes about 1200 strains of code to place it on one CS-1. However if you wish to run a 40 billion parameter mannequin, or 100 billion parameter mannequin, you employ the identical 1200 strains of code. That’s it. And so that you don’t have to write down 27,215 strains of code.”

“Now this takes our cloud to a brand new stage the place we’re working and working. We’re making them out there by way of our cloud. We’re providing AI supercomputers a service. If you need regular AI clusters, we’ve got these too. This actually takes our cloud to a brand new stage.”

The machine is called after the Condor Galaxy, which is about 5 occasions bigger than our personal Milky Approach.

Cereabras now has about 335 folks and it’s “hiring like loopy,” Feldman mentioned.