Tomahawk Ultra: Why UALink When There's Ethernet?

The Evolving Landscape of High-Speed Networking in AI and HPC

The world of high-performance computing (HPC) and artificial intelligence (AI) is constantly evolving, with companies like Nvidia leading the charge in developing advanced networking technologies. However, as other chip manufacturers such as AMD and Intel make strides in GPU performance, they face challenges when it comes to scaling their systems effectively.

One of the key factors that have allowed Nvidia to build powerful rack-scale systems is its proprietary technologies like NVLink and NVSwitch. These tools enable seamless communication between GPUs, allowing for the creation of systems with up to 72 GPUs. In contrast, AMD and Intel are still limited to around eight GPUs per system without similar interconnect solutions.

To address this gap, many industry players are turning to emerging alternatives like the Ultra Accelerator Link (UALink) protocol. This open standard aims to provide a scalable and flexible solution for connecting accelerators, offering an alternative to Nvidia’s proprietary offerings. However, not all stakeholders are convinced that UALink is the way forward.

Broadcom's Stance on Networking Protocols

Broadcom, once a member of the UALink consortium, has taken a different approach. According to Pete Del Vecchio, product line manager for Broadcom's Tomahawk line, Ethernet offers significant advantages when it comes to monitoring, telemetry, and debugging tools. He argues that there is no need to wait for new protocols that may take years to materialize.

Instead, Broadcom is focusing on its own technology called Scale-Up Ethernet (SUE). This solution claims to support systems with at least 1,024 accelerators using any Ethernet platform. For context, Nvidia’s NVLink switch technology can support up to 576 accelerators, though real-world deployments have yet to exceed 72 GPU sockets.

Introducing the Tomahawk Ultra

At the heart of Broadcom’s SUE initiative is the newly announced Tomahawk Ultra, a 51.2 Tbps switch ASIC designed to compete with both traditional InfiniBand and Nvidia’s NVLink. This chip is pin-compatible with Broadcom’s Tomahawk 5 (TH5), but under the hood, it represents a completely different design.

The Tomahawk Ultra features a large radix with 512 x 100 Gbps serializer deserializers (SerDes), optimized for high-performance networks. It promises latency as low as 250 nanoseconds while handling up to 77 billion 64-byte packets per second. This capability is particularly important for HPC systems, where smaller packets are common and can be challenging for conventional networking equipment.

Key Features of the Tomahawk Ultra

• Optimized Ethernet Header: Allows for larger payloads even with smaller packets.
• Congestion Control Mechanisms: Includes forward error correction and credit-based flow control to prevent packet loss.
• In-Network Collective Support: Enables operations like all-reduce to be offloaded onto the network, improving efficiency.

Compared to Nvidia’s fifth-gen NVLink switches, the Tomahawk Ultra offers nearly double the bandwidth at 51.2 Tbps versus 28.8 Tbps. This means that, with the same number of switches, Broadcom could support a scale-up fabric with 128 accelerators instead of 72 GPUs.

The Future of Networking Technologies

While Broadcom claims the Tomahawk Ultra already delivers better latency than UALink, these claims remain unverified until the first hardware ships. UALink’s consortium expects switch latency in the 100-150 ns range, which could offer an edge in certain applications if achieved.

AMD and Intel are also exploring options beyond UALink. At its Advancing AI event in June, AMD revealed its Helios rack systems, which will use both UALink and Ethernet for its scale-up fabric. This means that AMD will begin testing the UALink protocol over existing Ethernet infrastructure, allowing it to work out potential issues before UALink silicon becomes available.

Challenges and Opportunities

Tunneling UALink over Ethernet is not ideal, as it cannot achieve the protocol’s target latency of 100-150 ns. However, this approach allows AMD to move forward with its Helios rack systems without waiting for UALink hardware. As Robin Grindley, a principal product line manager at Broadcom, explained, "Other transport protocols... can be transported over Ethernet."

Ultimately, the competition between networking technologies like UALink, NVLink, and SUE is shaping the future of AI and HPC. As companies continue to innovate, the race for faster, more efficient interconnect solutions shows no signs of slowing down.