How to Build a 100G Leaf-Spine Network for GPU Servers (AI & RDMA Optimized)

Modern AI infrastructure is no longer constrained by compute power—it is constrained by the network.

In GPU-driven workloads such as distributed training and large-scale model synchronization, east-west traffic dominates the entire data center fabric. This makes traditional 10G/25G architectures insufficient, pushing enterprises toward 100G leaf-spine networks designed for RDMA performance.

However, designing such a system is not only about topology—it is about hardware selection, lossless configuration, and scalable procurement strategy.

Table of Contents

• Part 1: Why Leaf-Spine Architecture Is Essential for GPU Clusters
• Part 2: AI Workloads Require More Than Basic Leaf-Spine
• Part 3: Step-by-Step 100G GPU Network Design
• Part 4: RDMA & RoCEv2 Network Requirements
• Part 5: Hardware Selection for 100G AI Fabric
• Part 6: Physical Layer Design (Optics & Cabling)
• Part 7: Scaling Strategy and Procurement Considerations
• FAQ

Part 1: Why Leaf-Spine Architecture Is Essential for GPU Clusters

A leaf-spine topology flattens the network into two layers:

• Leaf switches connect directly to GPU servers
• Spine switches interconnect all leaf nodes
• Every leaf has equal-cost paths to all spines

This ensures predictable latency (1–2 hops maximum), high east-west bandwidth efficiency, and linear scalability for AI workloads.

Compared to legacy 3-tier designs, leaf-spine eliminates aggregation bottlenecks that typically cripple GPU clusters.

Part 2: AI Workloads Require More Than Basic Leaf-Spine

In real AI deployments, standard leaf-spine is often extended into rail-optimized architectures.

Each GPU server uses multiple NICs, and each NIC maps to a dedicated leaf switch. Same-index NICs connect to the same leaf.

This enables deterministic GPU-to-GPU communication, reduced cross-fabric congestion, and improved training stability for distributed workloads.

In RDMA-based environments, even minor packet loss can significantly impact performance and training efficiency.

Part 3: Step-by-Step 100G GPU Network Design

Step 1: Define Bandwidth Requirements

Each GPU node typically requires 1 × 100G NIC as a baseline.

Example: 32-node cluster results in approximately 3.2 Tbps edge bandwidth requirement.

Step 2: Build the Leaf Layer (Top-of-Rack)

Leaf switches must support high-density 100G ports, low-latency ASICs, and RDMA-ready features such as PFC and ECN.

A widely deployed option in AI clusters is: MSN2700-CS2FC, Mellanox Spectrum Switch, 32x100GbE QSFP28/2xAC PSU/x86 CPU

This 32-port 100G switch is commonly used in leaf layers due to its high port density and low-latency Spectrum ASIC design. It is frequently deployed in GPU clusters requiring predictable RDMA performance.

Step 3: Spine Layer Design

The spine layer interconnects all leaf switches using ECMP-based routing.

Best practice is maintaining a 1:1 oversubscription ratio for AI workloads to ensure non-blocking performance.

Part 4: RDMA & RoCEv2 Network Requirements

AI clusters rely heavily on RDMA (Remote Direct Memory Access), typically implemented via RoCEv2.

Because RoCEv2 runs over UDP, it is highly sensitive to packet loss. Even small congestion events can significantly degrade performance.

To ensure lossless Ethernet, three key mechanisms are required:

• ECN (Explicit Congestion Notification)
• PFC (Priority Flow Control)
• DCQCN adaptive rate control

Together, these mechanisms ensure stable GPU training performance in large-scale AI clusters.

Part 5: Hardware Selection for 100G AI Fabric

Switch selection must consider ASIC capability, buffer design, RDMA support, and real-world deployment reliability.

For smaller GPU clusters or edge AI environments: R0P82A, HPE StoreFabric SN2100M Switch, 16x100GbE QSFP28/ONIE support

For hybrid environments combining 25G and 100G: R0P81A, HPE StoreFabric SN2410M Switch, 48x25GbE SFP28/8x100GbE QSFP28

For spine or high-performance fabric layers:

MSB7570-E, Mellanox Switch-IB 2, 36x100Gb/s InfiniBand Spine Blade

MSB7560-E, Mellanox Switch-IB 2, 36x100Gb/s InfiniBand Leaf Blade

In real deployments, hardware consistency, firmware compatibility, and supply availability are critical factors affecting AI cluster stability.

Part 6: Physical Layer Design (Optics & Cabling)

100G infrastructure also depends on correct physical connectivity design:

• Within rack: DAC cables for lowest latency
• Between racks: AOC cables for flexibility
• Leaf to spine: QSFP28 optical modules over fiber

Improper cabling selection can lead to instability, signal degradation, or performance bottlenecks.

Part 7: Scaling Strategy and Procurement Considerations

Leaf-spine architecture enables horizontal scaling by adding leaf switches for more GPU nodes or spine switches for additional bandwidth.

However, in real-world AI infrastructure projects, scaling is often constrained by hardware availability, lifecycle status, and procurement lead times.

Router-switch supports enterprise AI infrastructure deployments by providing stable availability of mainstream networking hardware, multi-vendor sourcing capability, and lifecycle visibility for upgrade planning.

IT-Price can also be used for inventory lookup and pricing visibility to support procurement decisions in time-sensitive GPU cluster deployments.

FAQ

What is leaf-spine architecture in GPU networks?

It is a two-layer data center design where leaf switches connect servers and spine switches interconnect all leaves, enabling low-latency and scalable GPU communication.

Why is 100G required for AI workloads?

AI training generates massive east-west traffic that exceeds 10G/25G capabilities, making 100G the baseline for modern GPU clusters.

Why is RDMA important in GPU networks?

RDMA bypasses CPU and OS overhead, enabling direct memory-to-memory transfer, which is critical for high-performance distributed training.