Summary of CERERBRAS’ ALL NEW WAFER SCALE ENGINE PACKS 2.6 TRILLION TRANSISTORS FOR DEEP LEARNING WORKLOADS
Cerebras launched the Wafer Scale Engine 2 (WSE-2) in 2021: a wafer-sized deep learning chip with 2.6 trillion transistors, over 800,000 cores, 40 GB of on-chip SRAM, and a 220 Pb/s interconnect. It matches the original WSE size (46,255 mm2) while doubling interconnect speed and greatly increasing compute density to simplify and accelerate training and inference for large AI models.
Parts used in the Wafer Scale Engine 2 project:
- Wafer-scale silicon die (46,255 mm2)
- 2.6 trillion transistors
- Over 800,000 processing cores
- 40 GB on-chip SRAM
- Interconnect fabric with 220 Pb/s bandwidth
- Deep learning optimization architecture (chip-level design)
In 2019, we saw one of the largest single computer chips manufactured by a California-based AI startup, Cerebras, that unveiled the Wafer-Scale Engine for deep learning applications. The 1.2 trillion transistors-packed Wafer-Scale Engine came with 18GB of on-chip SRAM and an interconnect speed of 100 Pb/s (Petabytes per Second). After two years of development, the manufacturer has launched the next generation WSE-2 at the Linley Spring Processor Conference 2021.
To meet the computational requirements of deep learning tasks, the all-new Wafer Scale Engine comes in the same size as its predecessor of 46,255 mm2 but features more capabilities than ever before. The Cerebras Wafer Scale Engine 2 packs 2.6 trillion transistors with more than 800,000 cores, making it the most powerful single computer chip ever made that is entirely optimized for deep learning workloads.
In AI compute, big chips are king, as they process information more quickly, producing answers in less time—and time is the enemy of progress in AI,
Dhiraj Malik, vice president of hardware engineering, said in a statement.
With the increase in training time for deep learning models that are distributed over thousands of GPUs makes it more complex for deployment. This one device takes care of everything, “making orders-of-magnitude faster training and lower-latency inference easy to use and simple to deploy.” The Wafer Scale Engine adds 40 GB on-chip SRAM and an interconnect speed of 220 Pb/s, which is more than double the predecessor keeping the exact size of the wafer.
Read more: CERERBRAS’ ALL NEW WAFER SCALE ENGINE PACKS 2.6 TRILLION TRANSISTORS FOR DEEP LEARNING WORKLOADS
- What is the transistor count of the Wafer Scale Engine 2?
The WSE-2 packs 2.6 trillion transistors. - How much on-chip SRAM does the WSE-2 have?
The WSE-2 includes 40 GB of on-chip SRAM. - What is the interconnect speed of the WSE-2?
The WSE-2 offers an interconnect speed of 220 Pb/s. - How many cores are on the WSE-2?
The WSE-2 contains more than 800,000 cores. - Is the WSE-2 larger than the original WSE?
No, the WSE-2 keeps the exact same wafer size of 46,255 mm2 as its predecessor. - Does the WSE-2 improve training and inference for deep learning?
Yes, the WSE-2 is optimized to make training orders of magnitude faster and lower-latency inference easier to deploy. - When was the WSE-2 unveiled?
The WSE-2 was launched at the Linley Spring Processor Conference 2021 after two years of development. - How does the WSE-2 compare to the first WSE in interconnect bandwidth?
The WSE-2 more than doubles the interconnect speed from 100 Pb/s to 220 Pb/s compared to the first WSE.
