Articles by tag "100B model training"

1. InfiniBand vs RoCE v2 for 100B LLM Training: 2026 Architecture Guide
   Selene Gong

   18
   Quick Take For 100-billion parameter LLM training, 200G InfiniBand consistently outperforms 400G RoCE v2 in Model FLOPs Utilization (MFU). While RoCE v2 provides higher raw bandwidth at a lower upfront cost, its reliance on PFC and ECN introduces tail latency jitter and pause frames under ...