Every AI infrastructure conversation I'm in ends up at the same question: how do we get more out of the GPUs we already bought?
The usual answers are schedulers, checkpointing and smarter parallelism. The network shows up later, as a cost to keep down. I think that order is backwards, so I built a TCO model to test it.
The short version: on a 10,000-GPU training cluster, SRv6 traffic engineering recovers about $14.3 million a year in GPU value against $10 million of Year-1 spend. Payback comes in at 8.4 months.
Start with what the GPUs cost
The model uses a big cluster, though still well short of hyperscaler scale:
10,000 GPUs in 1,250 eight-GPU nodes
$250K per node including interconnect, for $312.5M of GPU capex
Three-year straight-line depreciation: $104.2M a year
$100M a year for power, cooling and operations
That's an annual cost base of $204.2M. Fully loaded, it works out to about $2.33 per GPU-hour, and you pay it whether the GPU is doing useful work or not.
That last point is the whole argument.
Where the waste hides
Large training jobs are synchronized. Every step waits on collective operations like all-reduce, and a collective only finishes when its slowest flow does. When hashing puts two elephant flows on the same uplink, or congestion builds on a spine, thousands of GPUs sit idle waiting on a few links.
The model assumes congestion and flow collisions waste 10% of GPU time. On a $204.2M cost base, that's about $20.4M a year of hardware, power and people producing nothing.
What SRv6 TE changes
SRv6 lets the source put the path directly in the IPv6 packet as a list of segments. Instead of hoping a hash spreads flows evenly, the fabric can steer them on purpose. It places flows on paths with headroom and keeps big collectives from piling onto the same links.
The model assumes SRv6 TE wins back 7 of those 10 lost points. The Year-1 incremental cost, covering enablement plus DPUs, is $10M.
The math
Recovered value: 7% × $204.2M = $14.3M a year
Year-1 investment: $10.0M
Payback: 8.4 months
Year-1 return: $1.43 back for every $1 spent, or $4.3M net
If I could put one number on a slide, though, it would be this one:
Each point of recovered utilization is worth about $2.0M a year on this cluster.
That changes the buying decision. You aren't evaluating a routing feature. You're buying GPU capacity at a discount.
Does it hold up?
A TCO model is only as good as its assumptions, so I stress-tested the two that matter most.
Utilization gain. The investment pays for itself within Year 1 at about a 4.9% gain. At 4%, the first year returns 0.82x, and payback slips to about 15 months. At 10%, the first year returns 2.04x.
Spend. With the benefit held at $14.3M, cutting the investment to $5M brings payback down to 4.2 months. Raising it to $15M stretches payback to 12.6 months, just over a year.
So the case needs SRv6 to recover about five points of utilization to clear Year 1. That makes the first job clear: measure how much GPU time your fabric is actually costing you today. If network-driven idle time is well under five points, the case gets thin. If it's near the 10% in this model, the case gets easy.
The capacity nobody counts
Recovered utilization also shows up as time. A 30-day training run drops to 27.9 days. Across 12 large runs a year, that frees about 25 days of full-cluster time, which is roughly 0.84 of an extra large training run without buying another GPU.
The model puts no dollar value on shipping a model two days sooner. For a lot of teams, that's worth more than the utilization math.
Where I kept it conservative
The fabric itself ($46.9M, or 15% of GPU capex) stays out of the cost base, so recovered value is measured against GPU and operating spend only.
The extra-run figure uses the 30-day baseline, not the shorter 27.9-day run.
There's no revenue value for faster time-to-model.
The analyst take
SRv6 TE isn't a networking line item. It's a GPU utilization play. Teams that budget it that way, against the GPU cost base instead of the switching budget, will decide faster and have an easier time defending the decision.
If you want to test this with your own cluster size, node cost and measured idle time, reach out. The model is built to swap in those inputs.
The full model on one page
— Ray Mota, ACG Research
Source: ACG Research SRv6 TE ROI Model, September 2026. Figures are model outputs based on the stated assumptions, not measured results from a specific deployment.
Originally published on Ray Mota's Substack. Subscribe to get new analysis by email.
