Optical Interconnect Design for AI Clusters: GPU Fabrics, Spine-Leaf vs Rail-Optimized, and When CPO/LPO Makes Sense
How AI/ML compute clusters reshape optical interconnect choices: east-west traffic patterns, rail-optimized vs spine-leaf topologies, 800G module placement, AEC/DAC vs optics trade-offs, and the emerging role of LPO and co-packaged optics.
Conclusion first
AI clusters invert the classic datacenter traffic model: instead of a few north-south flows, thousands of GPUs exchange synchronized gradients east-west, in bursts, all at once. Optical interconnect stops being a commodity plumbing decision and becomes a topology, power, and thermal co-design problem. The winners in 2026 designs are: rail-optimized fabrics with 800G OSFP/QSFP-DD800 optics, AECs for intra-row, and a careful watch on LPO/CPO as they mature.
1. Why AI traffic breaks normal datacenter rules
Three properties of collective communication drive everything else:
| Property | Consequence for optics |
|---|---|
| Synchronized bursts (all-reduce every step) | Link utilization swings 0% → 100% in microseconds; buffer underrun shows up as collective timeouts, not just slow training |
| Rank-aware traffic matrix | Topology can be matched to traffic (rail optimization), not just bisection bandwidth |
| Zero tolerance for silent bit errors | PAM4 + FEC hides some errors; stragglers trigger step retries that stall the entire job — link quality is a throughput feature, not a compliance checkbox |
This is why AI operators obsess over corrected-codeword counters and "flapping" optics that a web-scale frontend network would never notice.
2. Topology: spine-leaf vs rail-optimized
Classic spine-leaf (frontend style):
GPU server ── leaf ── spine ── leaf ── GPU server- Any-to-any connectivity, 2 hops for cross-pod traffic.
- Simple, but every collective hop burns switch power and adds latency variance.
Rail-optimized (AI-native):
GPU0 of servers 0..N ── Rail-0 switch group
GPU1 of servers 0..N ── Rail-1 switch group
...- Same-index GPUs land on the same rail; all-reduce groups map onto physical rails.
- Most collective traffic is single-hop; spine layer only handles outliers and storage/I-O traffic.
Design ratio guidance (2026 practice): 1:1 (non-blocking) rail groups for dense training pods; 2:1 or 3:1 oversubscription is tolerable for inference and fine-tuning clusters. The optical consequence: rail-optimized designs consume more optics per GPU (each GPU port is an 800G DR8/2×FR4 or a pair of 400G), which is exactly why power-per-port dominates TCO discussions.
3. The media ladder: DAC → AEC → LPO → optics → CPO
| Medium | Reach | Power (800G) | Best use | Risk |
|---|---|---|---|---|
| DAC | ≤ 2 m | ~0 W (passive) | Same-rack top-of-rack to host | PAM4 reach shrinks at 800G |
| AEC | 3-5 m | 4-8 W | Rack-to-rack, in-row | Redriver firmware quality varies |
| LPO | 50-100 m | ~6-8 W | Adjacent-row, power-critical pods | Host SerDes must absorb channel loss; vendor pairing needed |
| Pluggable optics (DR8/2×FR4) | 100-500 m | 12-16 W | Row-to-row, leaf-spine | Heat density in cages; airflow design |
| CPO | On-package | Best per-bit power | Next-gen switch ASICs | Non-replaceable optics, new ops model |
Field heuristic: if the optical path is under 3 m, try DAC; under 5 m and power-capped, AEC; beyond that, pluggable optics; and if your switch vendor offers validated LPO SKUs for the specific SerDes, LPO can halve optics power on short hops.
4. Thermal and mechanical reality at 800G
- 800G OSFP modules run 13-16 W typical, up to ~18-20 W worst-case classes. A 64-port OSFP switch can carry ~1 kW of optics alone — airflow design matters as much as ASIC power.
- QSFP-DD800 packs the same 8-lane electrical into a smaller cage; thermal density per U is higher, so rear-panel exhaust designs and raised-breeze enclosures behave differently between the two form factors.
- Cage-to-cage pigtail routing (MPO trunks vs factory-terminated assemblies) changes insertion loss by up to 0.5 dB — in a 1.9 dB DR-class budget that is fatal (see our link budget article).
5. What to actually specify in 2026
- Form factor: follow your switch vendor's native choice; OSFP for maximum thermal headroom, QSFP-DD800 for port density.
- PMD: DR8 for structured cabling (500 m), 2×FR4 for leaf-spine (~2 km), VR8-class only for same-rack with LPO.
- Management: require CMIS 4.0+ so DDM, power classes, and lane mapping are scriptable.
- Quality telemetry: integrate corrected-codeword rate monitoring into your cluster scheduler; treat degrading links as maintenance events, not noise.
- Future-proofing: reserve cage space and cooling for 200G/lane (802.3dj-class) optics even if you buy 100G/lane silicon today.
FAQ
The FAQ block above covers rail optimization, DAC viability at 800G, and the LPO vs CPO decision.
Sources integrated and re-analyzed: public GPU platform networking guides, OSFP/QSFP-DD800 MSA thermal data, OIF and IEEE roadmap presentations, and operator experience reports on collective-communication networks.
FAQ
- Why do AI clusters prefer rail-optimized topologies?
- Collective-communication traffic (all-reduce, all-to-all) is GPU-rank-aware. Rail optimization places the same GPU index across servers onto the same switch, letting most traffic complete in one hop and reducing tail latency for synchronous training steps.
- Is DAC still viable at 800G inside a rack?
- Barely. 800G DAC reach is about 1-2 m due to PAM4 attenuation; AECs extend that to ~3-5 m with active redrivers. Beyond rack-row distances, optics (DR8/2×FR4) dominate. Power is the trade: optics cost 12-16 W per module vs ~4-8 W for an 800G AEC.
- What is the practical difference between LPO and CPO today?
- LPO (linear pluggable optics) removes the DSP from the module, cutting power roughly in half, but requires the switch SerDes to compensate for channel loss — so it is sensitive to host quality. CPO moves optics onto the switch ASIC package entirely: best power efficiency, but the module becomes non-replaceable and repairability suffers.