In a new patent titled "Non-Adjacent Connection of High-Bandwidth Memory Chiplets, I/O Chiplets, And Compute Chiplets Through Embedded Logic Bridges", OpenAI shares plans for an AI chip solution that is going to house several HBM chiplets and compute chiplets, all connected using Embedded Logic Bridges.

The research proposes the idea of leveraging these embedded logic bridges for high-speed interconnects between longer distances, enabling more chiplets to connect together to support high-performance computing and AI workloads which require access to large amounts of memory for efficient operation.

Current packaging solutions have certain limitations regarding HBM integration as HBM memory communicates with other chiplets on the package using metal wires on the base layer. The current JEDEC standard requires HBM memory to be placed adjacent to the compute chiplet, and there are physical limitation with this approach since the metal wires are required to be less than 6mm in length from the PHY controller on the main chiplet itself.

just enable longer communication distances, but they can also provide functionality of a controller for the HBM stack, or provide the functionality of a high-speed PHY for communicating between the chiplets within a package. This D2D (Die-to-Die) interface complies with the UCIe (Universal Chiplet Interconnect Express) standard.

In an example, OpenAI showcases a compute chiplet with 20 HBM memory stacks using the Embedded Logic Bridges. With traditional approaches, you are limited to four, six, or eight stacks. This scales up the memory capacity significantly, leading to chips that are potent for larger AI models.

But this research also goes hand in hand with a technology that someone is already working on. That's Intel's EMIB or Embedded Multi-Interconnect Bridge solution. EMIB is an advanced packaging solution that acts like a bridge.

It is designed to tackle the 2.5D packaging tech, leveraging tiny bridges to expand the capabilities and designs of high-performance chips. EMIB and its successor, EMIB-T, have various advantages, they are simple, they are small, they expand beyond the reticle limits of current interposers, and the are cost effective solutions.

Can we see a future where Intel's EMIB technology gets used by OpenAI for the creation of its custom AI chips with many chiplets and large amounts of HBM memory? Well, this patent seems be pointing towards that.

By: DocMemory
Copyright © 2023 CST, Inc. All Rights Reserved