IBM’s breakthrough in optics propels the era of generative AI to the speed of light.
CPO Co packaged optics
What is co-packaged optics?
In order to solve the issues of bandwidth density, power efficiency, and communication latency in contemporary networks, co-packaged optics (CPO) combines electronics and optics into a single package. CPO is utilized in many different applications, including as Ethernet switches, machine learning, and disaggregation, and is anticipated to be crucial in 6G radio-access networks.
Compared to conventional pluggable optics, CPO has a number of benefits, such as:
- Power savings: When compared to conventional pluggable optics, CPO can cut power usage by 30% to 50%.
- Scalability: To accommodate upcoming network demands, CPO offers a scalable route.
- Decreased signal deterioration: CPO’s integration of silicon and optics lowers power loss and signal degradation.
Each packaging method still has advantages and disadvantages, and Co packaged optics is still in the early phases of industrialization. One of the most promising approaches to high-speed optoelectronic co-packing is 3D packaging.
Electrical interconnects in data centers may be replaced with new co-packaged optics technology, providing AI and other computing applications with notable speed and energy efficiency gains.
IBM has revealed innovative optical research that might significantly enhance the way generative AI models are trained and operated in data centers. Researchers have developed a novel method for co-packaged optics (CPO), the next generation of optics technology to complement current short-range electrical connections and enable interconnection within data centers at the speed of light. IBM researchers have demonstrated how Co packaged optics will revolutionize the computing industry’s high-bandwidth data transmission between chips, circuit boards, and servers by creating and assembling the first successfully announced polymer optical waveguide (PWG) to power this technology.
Almost all of the world’s communications and commerce traffic is now handled by fibre optic technology, which uses light rather than electricity to transport data over great distances at rapid speeds. Despite using fibre optics for their external communications networks, data center racks still primarily employ copper-based electrical lines for communications. These cables link GPU accelerators that might be idle for over half of the time, awaiting data from other devices in a lengthy, dispersed training process that can be very costly and energy-intensive.
Researchers at IBM have shown how to introduce the speed and capacity of optics inside data centers. IBM presents a new Co packaged optics prototype module that can provide high-speed optical communication in a technical report. This technology has the potential to greatly boost data Centre communications bandwidth, reducing GPU downtime and dramatically speeding up AI computation. According to the description, this research innovation would allow:
- Lower costs for scaling generative AI by extending data centre interconnect cables from one to hundreds of meters and reducing energy consumption by more than five times as compared to mid-range electrical interconnects.
- With Co packaged optics, developers can train a Large Language Model (LLM) up to five times faster than they could with traditional electrical wiring because to faster AI model training. A conventional LLM might be trained in three weeks instead of three months using CPO, and performance gains would increase with the use of larger models and additional GPUs.
- Significantly improved data center energy efficiency, with each AI model taught saving the energy equivalent of 5,000 US houses’ yearly power consumption.
“The data center needs to change as generative AI requires more energy and processing power. Co-packaged optics can make these data centers future-proof,” stated Dario Gil, SVP and Director of Research at IBM. “This breakthrough will usher in a new era of faster, more sustainable communications that can handle the AI workloads of the future, with tomorrow’s chips communicating much like fiber optic cables carry data in and out of data centers.”
Faster bandwidth by 80 times compared to current chip-to-chip connectivity
Transistors are now densely packed onto a chip thanks to recent advancements in chip technology; IBM’s 2 nanometre node chip technology can accommodate over 50 billion transistors. By allowing chipmakers to establish optical channels connecting chips on an electronic module beyond the constraints of current electrical pathways, Co packaged optics technology seeks to expand the interconnection density between accelerators. In contrast to electrical connections, IBM’s research describes how these new high bandwidth density optical structures, when combined with the ability to transmit several wavelengths per optical channel, have the potential to increase bandwidth between processors by up to 80 times.
In comparison to the existing state-of-the-art CPO technology, IBM’s invention would allow chipmakers to install six times as many optical fibres to the edge of a silicon photonics chip, a feature known as “beachfront density.” Terabits of data per second might be transmitted by each fibre, which is roughly three times the width of a human hair and can extend from centimetres to hundreds of meters. The IBM team used typical assembly packaging procedures to create a high-density PWG at 50 micrometre pitch optical channels that were adiabatically connected to silicon photonics waveguides.
Furthermore, according to the report, these Co packaged optics modules with PWG at 50 micrometre pitch are the first to successfully complete all manufacturing-related stress testing. In addition to mechanical durability testing to ensure that optical interconnects can bend without breaking or losing data, components are exposed to high humidity conditions and temperatures between -40°C and 125°C. Additionally, PWG technology has been shown by researchers up to an 18-micrometer pitch. At that pitch, up to 128 channels of communication might be achieved by stacking four PWGs.
IBM’s sustained dominance in semiconductor research and development
In order to fulfil AI’s growing performance requirements, Co packaged optics technology opens up a new avenue that could eventually replace electrical off-module interactions with optical ones. The first 2 nm node chip technology, the first use of 7 nm and 5 nm process technologies, nanosheet transistors, vertical transistors (VTFET), single cell DRAM, and chemically amplified photoresists are just a few examples of IBM’s long history of leading semiconductor innovation.
In Albany, New York, which was recently chosen by the U.S. Department of Commerce to house America’s first National Semiconductor Technology Centre (NSTC), the NSTC EUV Accelerator, researchers finished design, modelling, and simulation work on CPO. At IBM’s factory in Bromont, Quebec, one of the biggest chip assembly and testing locations in North America, researchers put together prototypes and tested modules. For many years, IBM’s Bromont facility, which is a part of the Northeast Semiconductor Corridor between the US and Canada, has been at the forefront of chip packaging worldwide.
