AI’s Invisible Bottleneck: Why the Network Will Determine the Return on AI
A GPU waiting for data represents expensive computing capacity that cannot do useful work.
That observation captures the central argument of the conversation between ACG Research’s Ray Mota and Cisco’s Bill Gartner: the value of AI infrastructure depends on how effectively its components work together. Buying powerful processors is one part of that equation. Connecting them reliably, efficiently, and economically is another.
As AI systems expand across racks, buildings, and geographic locations, networking increasingly determines how much of that computing investment becomes productive capacity. Congestion, packet loss, inconsistent latency, limited fiber availability, and power constraints can all reduce the performance of an otherwise capable system.
Mota describes this as an “invisible bottleneck.” The industry sees the GPUs, the model announcements, and the investment commitments. The optical infrastructure that holds these systems together receives less attention, even though its performance can determine the outcome.
Their discussion moves through the entire chain: the changing nature of AI traffic, the distinction between traditional data center interconnect and scale-across architectures, the physical limits of fiber, the economics of GPU utilization, advances in coherent optics, the challenges of network operations, and the growing importance of security and sovereignty.
Watch the full conversation: AI’s Network Bottleneck: Cisco’s Bill Gartner & ACG Research’s Ray Mota.
The figures discussed below reflect statements and estimates made during the interview. Cisco’s shipment counts and product savings claims are presented in that context; they should be evaluated against the requirements and configuration of a particular deployment.
AI changes the relationship between computing and communication. Traditional infrastructure planning often treated compute, storage, and networking as distinct investment categories. Organizations could expand each domain according to its own requirements, with the network connecting the resulting systems.
Large distributed AI workloads make those boundaries harder to maintain.
A collection of GPUs becomes useful as a larger computing system only when those GPUs can exchange information effectively. The processors may perform work in parallel, but portions of that work depend on coordination, synchronization, and the timely arrival of data from other processors.
The network therefore participates directly in the execution of the workload.
Mota opens the discussion by contrasting familiar north–south traffic patterns with the increasingly intensive east–west communication associated with AI. North–south traffic generally moves into or out of a computing environment. East–west traffic moves among systems within or across the infrastructure.
AI increases the importance of those internal exchanges. A system can have substantial processing capacity and still deliver disappointing results if the communication fabric cannot support the workload.
Gartner’s point is straightforward: GPUs provide the computing engine, while the network enables those engines to function collectively. As the infrastructure expands, the quality of that collective operation becomes increasingly consequential.
For infrastructure buyers, this changes the evaluation process. Processor specifications remain relevant, but they need to be considered alongside the performance of the system that connects those processors.
Scale-across introduces a more demanding form of data center connectivity. One of the interview’s most useful distinctions concerns traditional data center interconnect, commonly called DCI, and the emerging requirement to extend AI systems across multiple data centers.
Traditional DCI connects computing and storage resources between facilities. It supports activities such as replication, backup, application delivery, and the exchange of traffic supporting internet services.
Scale-across extends the communication fabric supporting a distributed AI system beyond an individual facility.
Gartner describes a progression. AI infrastructure first grows within a rack. It then expands across multiple racks. Eventually, practical constraints—especially the availability of electrical power—can make further expansion within one building difficult.
At that point, operators may need to place additional infrastructure in another data center.
The challenge is that the application still needs those resources to behave as parts of a coordinated system. The interconnection must support that coordination despite the distance between facilities.
Primary purpose: Traditional DCI exchanges application data, replicates storage, and connects services. Scale-across supports communication among distributed AI computing resources.
Effect of delay: Some traditional applications tolerate additional delay relatively well. Synchronized AI workloads can be sensitive to delayed communication.
Effect of packet loss: Recovery mechanisms may conceal the impact from users in traditional applications. In AI workloads, recovery can interrupt progress and reduce useful GPU work.
Performance emphasis: Traditional DCI emphasizes capacity, availability, and service delivery. Scale-across also places particular emphasis on congestion control and consistent latency.
Economic consequence: Traditional DCI problems can degrade applications or disrupt services. Scale-across problems can also cause a loss of productive time across costly computing resources.
These are differences in workload requirements, rather than absolute boundaries. Traditional applications can also have strict performance needs, and AI workloads vary substantially in their sensitivity to distance and disruption.
Gartner uses the idea of making resources in separate buildings behave more like nearby resources to explain the architectural objective. That requires careful interpretation: networking cannot eliminate the propagation delay imposed by physical distance.
The practical goal is to minimize avoidable impairments—such as congestion, errors, and unstable performance—while choosing distances and architectures the workload can tolerate.
This is why scale-across cannot be evaluated solely as a larger bandwidth connection between buildings. Its purpose is to preserve the efficiency of a distributed computing system.
Agentic AI adds a second source of networking pressure. The interview distinguishes the communication demands of coordinated GPU infrastructure from the traffic generated by AI agents.
Gartner begins with the familiar chatbot interaction. A person submits a request, waits while the system processes it, and receives a response. The person might then clarify the question and begin another exchange.
From a networking perspective, that interaction often has a recognizable request-and-response pattern.
Agentic AI can produce a more sustained and distributed sequence of activity. An agent acts on behalf of a person or an application. Completing one task may require it to communicate with other agents, applications, and sources of information.
One user request can therefore initiate many additional interactions.
Those agents may operate in a cloud environment, within an enterprise data center, or on an individual device. Their communication can cross organizational and geographic boundaries.
The result is a traffic pattern shaped by machine activity rather than solely by the pace of human interaction. Mota emphasizes this distinction: machines can continue exchanging information while a person would otherwise be reading, thinking, or preparing the next question.
This does not mean every agentic application generates continuous high-bandwidth traffic. It means that capacity planning based only on simple user-to-chatbot exchanges can overlook the number, distribution, and persistence of the interactions behind an agent’s work.
The planning questions become more detailed. Where will agents run? Which systems will they contact? How frequently will they communicate? Which exchanges are sensitive to delay? What happens when one dependency becomes unavailable?
A network designed around isolated application transactions may need to evolve as enterprises introduce workflows involving multiple cooperating agents.
Fiber capacity is becoming a strategic constraint. Increasing demand eventually reaches the physical infrastructure carrying the traffic.
Gartner identifies the availability and capacity of installed fiber as a major bottleneck. The industry has made extraordinary progress in extracting more capacity from existing fiber, but that progress does not create unlimited headroom.
Dense wavelength division multiplexing, or DWDM, allows multiple optical signals to share a fiber by transmitting them at different wavelengths.
Capacity can increase by raising the data rate carried on each wavelength, using more wavelengths, or expanding the portion of the optical spectrum used for transmission.
Historically, much of this activity concentrated on the C-band. Gartner describes an illustrative configuration carrying approximately 32 wavelengths at 800 gigabits per second each.
That corresponds to 25.6 terabits per second of aggregate wavelength capacity in the example.
Expanding into the L-band provides additional usable spectrum. Gartner describes approximately 64 wavelengths at 800 gigabits per second when combining C-band and L-band operation.
The corresponding arithmetic is 51.2 terabits per second.
Those examples explain the scale of the capacity expansion. They are not universal limits or guaranteed capacities for every installed fiber route. Actual results depend on the optical system, channel plan, transmission distance, and conditions along the route.
The larger point is that capacity levels once considered difficult to imagine are becoming relevant to real deployments, particularly in hyperscale AI infrastructure.
Gartner also discusses probabilistic constellation shaping, one of the techniques used to improve the relationship among transmission efficiency, capacity, and reach. Such advances help optical systems use the available transmission channel more effectively.
However, improvements in modulation and signal processing operate within physical limits.
Gartner invokes Claude Shannon’s work in information theory to explain that a communication channel has a finite capacity determined by its characteristics, including bandwidth and noise. Optical transmission also presents practical engineering constraints that influence how closely systems can approach their achievable limits.
The implication is that operators cannot assume successive equipment upgrades will indefinitely satisfy demand over the same installed infrastructure.
They may need additional fiber, new routes, or a different balance between increasing capacity per fiber and deploying more fiber pairs.
That turns fiber planning into a long-term AI infrastructure decision. The availability of GPUs is only one consideration when the routes connecting those GPUs may also become constrained.
The network directly affects the economics of GPU utilization. Mota raises the financial question underlying the technical discussion: how does the network influence the return on a large AI investment?
Gartner’s answer centers on interdependence.
An individual GPU has its own performance characteristics. A large AI system, however, may depend on thousands or even hundreds of thousands of GPUs communicating as parts of a distributed workload.
If that communication becomes unreliable, expensive computing resources can spend time waiting, recovering, or repeating work.
Gartner offers an example involving 100,000 GPUs and a link that briefly loses and regains connectivity—a link flap. In a tightly coordinated workload, disruption in one part of the communication fabric can affect a much larger group of participating processors.
Depending on the application and its recovery mechanisms, work may pause or return to a checkpoint.
The example illustrates a potential failure consequence. It does not establish that every link interruption affects every GPU in every architecture. The scope depends on the workload, the communication pattern, the network design, and the system’s resilience.
Nevertheless, the economic exposure is clear. A local networking problem can have consequences that extend well beyond the cost of the affected port or optical module.
Gartner contrasts this with a conventional application environment in which one affected server may leave many other servers continuing their work independently.
In a synchronized AI workload, dependencies can spread the impact.
Mota describes the resulting loss of productivity as GPU starvation. Near the end of the interview, he connects the same idea to stranded capital: infrastructure has been purchased and powered, but cannot deliver its intended value because data is not arriving when needed.
The analytical implication is that a low-cost network can become expensive if it reduces useful computing output.
Procurement decisions therefore need to account for the effect of networking on the productivity of the broader AI investment. Component cost remains relevant, but so do the costs of waiting, interruption, and recovery.
Tail latency explains why averages can conceal expensive problems. A particularly important part of the conversation concerns the difference between average latency and tail latency.
Average latency summarizes typical delay. Tail latency describes the slower end of the distribution—the exchanges that take substantially longer than most others.
For a workload that must wait for multiple participants before proceeding, the delayed participant can determine when the next stage begins.
Mota captures this with the phrase “single slowest packet first.” Gartner connects that observation to synchronization: the system may care deeply about the last required contribution to arrive.
This helps explain why a network can look healthy in an average-latency report while still limiting application performance.
Most exchanges may complete quickly. A smaller number of slow exchanges can repeatedly delay coordinated progress.
The relevant operating questions therefore extend beyond whether the network has enough nominal bandwidth. They include whether congestion produces unpredictable delays, whether recovery events create long pauses, and whether the application experiences consistent performance under load.
Gartner also emphasizes optical reliability across different distances, from short connections within computing environments to longer connections across facilities.
Distance changes the engineering problem, but the need for dependable communication remains.
Power constraints make efficiency improvements valuable throughout the system. Power appears repeatedly in the interview because it influences both operating cost and the ability to expand infrastructure.
Mota cites estimates from his team suggesting that networking accounts for approximately 15–20% of total data center power consumption. He also refers to an estimate that roughly 60% of networking energy is associated with moving data across connections.
Those figures are presented as discussion estimates. The interview does not establish a universal measurement boundary or methodology, and the proportions will vary with architecture and workload.
Gartner’s response focuses on the decision principle.
It is reasonable to pursue efficiency improvements in the largest power-consuming parts of a facility. It is also necessary to pursue improvements elsewhere when power availability limits growth.
His formulation is simple: “every watt matters.”
A watt saved in the interconnect contributes to the same power budget as a watt saved elsewhere in the infrastructure. Repeated across many connections, equipment-level improvements can become operationally significant.
The interview identifies two distinct areas of optical innovation: coherent pluggable interfaces and the optical line systems supporting transmission over longer routes.
These address different parts of the network, so their savings should be evaluated separately.
Coherent pluggables change how optical transport functions are delivered. Gartner explains the traditional role of a transponder.
A router may produce a short-reach optical signal suitable for connecting equipment over a relatively limited distance. A transponder converts that signal into a wavelength suitable for longer-distance DWDM transmission.
The interview uses the familiar distinction between gray optics for the short-reach interface and colored optics for the wavelength carried by the transport system.
Historically, the transponder function resided in dedicated equipment within an optical chassis.
Advances in coherent technology, digital signal processing, and packaging have allowed that function to move into a pluggable module installed directly in a compatible router or switching platform.
Gartner credits Acacia, which Cisco acquired, as an important contributor to that development, while acknowledging innovation elsewhere in the industry.
Moving the function into a smaller form factor can reduce the equipment needed for an applicable connection. That can affect power consumption, capital cost, physical footprint, installation, and ongoing operations.
Gartner illustrates the power opportunity by comparing a transponder consuming approximately 250 watts with a 400G pluggable consuming approximately 25 watts.
Within that example, the reduction is 225 watts, or 90%.
The scope of the comparison matters. A reduction in power for a particular transport function does not imply a 90% reduction in the power consumption of the complete network or data center. Operators still need to account for the host platform, amplification where required, and other supporting infrastructure.
The example nevertheless shows why coherent pluggables have become economically attractive.
Gartner describes an evolution from 400G coherent pluggables toward 800G offerings. The underlying development is broader than a speed increase: optical transport capability is being delivered in a form that can simplify the surrounding architecture.
Repeater huts reveal another physical limit on expansion. Additional fiber does not eliminate every infrastructure constraint.
Gartner describes a change in the scale of fiber builds. Routes that once carried a small number of fiber pairs may now need to support hundreds.
Over long distances, optical signals require amplification. The associated equipment is often housed in repeater huts along the route.
These facilities have finite space and electrical capacity. Many were designed for substantially smaller deployments.
An operator may therefore have enough fiber capacity planned along a route while lacking the space or power needed to support the corresponding amplification equipment.
The constraint moves from the conduit to the building containing the line-system equipment.
Gartner discusses multi-rail optical line systems as a response. These systems are designed to support multiple fiber paths more efficiently, improving the use of power and space at amplification sites.
He cites Cisco’s Open Transport 3000 and describes a configuration supporting approximately 128 fiber pairs. He also cites reductions of roughly 75% in power and 80% in space.
Cisco’s published explanation expresses the savings as power and rack-space reductions per rail, an important qualification when interpreting the percentages. See Cisco’s explanation of its multi-rail optical innovations.
The wider lesson is that infrastructure expansion depends on the supporting facilities along the entire path.
Data centers receive much of the attention, but intermediate amplification sites can also determine whether a route can support growing demand.
Operational simplicity becomes more valuable as the infrastructure grows. Higher capacity does not automatically produce a network that is easier to operate.
Mota asks whether the architectural changes under discussion risk creating additional complexity. Gartner responds by emphasizing the importance of visibility, context, and automation.
He offers a broad economic observation: customers may spend four or five dollars in operating expenditure for every dollar of capital expenditure.
The interview does not specify a uniform accounting period or deployment model for that ratio. Its purpose is to emphasize the scale of the operating burden, including power, staffing, troubleshooting, and field service.
Gartner’s fiber-failure example makes the problem concrete.
A fiber carrying 64 wavelengths at 800G can support a substantial amount of downstream traffic. If that fiber fails, many connected systems may generate alarms.
The operator’s challenge is to identify the underlying failure quickly and understand which alerts are consequences of it.
Without that context, staff can spend valuable time investigating symptoms across multiple layers.
The same principle applies to complex AI infrastructure. A disruption can appear in application performance, routing behavior, optical interfaces, and service-level monitoring at the same time.
Useful automation must connect those observations to the underlying cause.
Gartner also discusses using traffic information to understand where capacity is being consumed and how the network might be better utilized.
The operational objective is to shorten the path from observation to understanding to action. A larger volume of telemetry is helpful only when operators can use it to make better decisions.
AI security has two connected responsibilities: protecting the infrastructure and constraining what agents can do. The interview broadens security beyond conventional perimeter protection.
Gartner warns that AI capabilities are available to attackers as well as defenders. In his account, that can compress the time between the disclosure of a vulnerability and attempts to exploit it.
Organizations may require substantial time to validate and deploy upgrades. Attackers using automated tools may move much faster.
His architectural response is to place security controls closer to the applications and infrastructure being protected, including integration with switching and routing.
Perimeter firewalls retain a role. As workloads and interactions become more distributed, enterprises also need enforcement within the environment.
The second security responsibility concerns the behavior of AI agents themselves.
Gartner uses a customer-service agent as an example. The agent may legitimately need information that helps resolve a customer’s problem. That assignment does not automatically justify access to the organization’s human resources system.
The distinction is between an agent’s ability to reach a system and its authority to use that system.
As agents receive greater responsibility, enterprises need explicit boundaries around the applications, information, and actions available to them.
Gartner invokes an analogy associated with Cisco product leader Jeetu Patel: giving an agent broad authority can resemble giving a teenager a credit card. The point is that delegated capability requires limits and oversight.
Mota adds another dimension through his concept of conflict of intent. Multiple agents may pursue different objectives, and their actions can interfere with one another or produce an outcome the organization did not intend.
This expands the governance problem. Enterprises need to understand the purpose assigned to each agent, the permissions accompanying that purpose, and the controls that prevent an agent from exceeding its role.
The practical implication is that AI architecture, security policy, and agent governance need to be designed together.
A full-stack approach requires coordination across technical and organizational boundaries. Mota describes a familiar industry pattern: silos of innovation and automation.
Individual teams can make progress within their own domains while the overall system remains inefficient.
Gartner acknowledges the organizational roots of that problem. Companies develop specialized teams, build expertise around particular product areas, and reinforce those structures over time.
Those structures can make it difficult to adopt architectures that cross established boundaries.
He gives the example of separate IP and optical operations teams. Converging technical functions may create friction when the organizational responsibilities remain divided.
The same challenge appears in product development. Improving silicon, optics, systems, and software independently does not guarantee that they will combine into the most efficient platform.
Gartner describes Cisco’s full-stack approach as an effort to optimize the customer’s overall outcome across those layers.
He explicitly pairs that position with a commitment to openness. His argument is that an integrated design process can improve system performance without requiring a proprietary solution at every layer.
He also acknowledges a role for best-of-breed components. The question is how well those components work together under the actual workload.
For buyers, the useful test is therefore operational and measurable: does the proposed architecture improve useful throughput, power efficiency, reliability, deployment, and management?
The interview presents Cisco’s position on those benefits. A deployment-specific evaluation would still need to establish how they appear under a customer’s requirements.
Mota reinforces the broader principle through an observation about cloud applications and videoconferencing: visible application experiences depend on an effective underlying network.
AI increases the importance of that underlying infrastructure because its inefficiencies can become more directly visible in computing performance.
Managed optical fiber networks provide another path to capacity. The discussion then turns to who builds and operates the infrastructure connecting AI facilities.
Hyperscalers sometimes construct and manage their own networks. They deploy or acquire fiber, install equipment, and operate the resulting infrastructure.
In other circumstances, they need capacity in a location where another provider already has suitable assets or operational capabilities.
Gartner describes managed optical fiber networks as an arrangement in which a provider delivers and manages dedicated optical network infrastructure for the customer.
Several factors can make that model attractive.
One is speed. An existing provider may be able to supply infrastructure sooner than the customer could establish it independently.
Another is geographic reach. A hyperscaler may need access to locations beyond its existing network footprint.
Gartner also cites local licensing and operating requirements, including examples involving India, as a reason to work through service providers. The precise requirements depend on the market and arrangement.
The result can be a network dedicated to a hyperscaler’s needs while another organization manages it.
Gartner identifies India and the Middle East as areas where these arrangements have supported capacity expansion. He also points to publicly discussed network investment by companies such as Lumen in the United States.
For service providers, the opportunity extends beyond supplying generic connectivity. They can contribute fiber assets, route access, deployment capabilities, and ongoing operations to support customers’ AI infrastructure.
The broader observation is that AI network growth can occur through several ownership and operating models.
The shipment figures in the interview indicate the scale of adoption Gartner describes. When Mota asks about Cisco’s progress, Gartner cites more than 800,000 shipped 400G coherent ports and more than 80,000 shipped 800G ports.
He also says Cisco has supplied coherent pluggables to more than 500 service providers.
Gartner qualifies the figures by acknowledging that they may already lag the latest totals. They should therefore be read as a snapshot offered during the conversation.
He describes substantial deployment into hyperscaler infrastructure while also identifying service-provider uses, including DCI and metro network upgrades.
The significance is the breadth of applications he describes: coherent pluggables are serving multiple types of customers and network environments.
Sovereign AI requires control over the routes data takes. Toward the end of the interview, Mota asks how sovereignty changes the design of optical transport infrastructure.
Gartner begins with control.
Organizations and governments may want assurance that infrastructure remains under their authority and that access cannot be determined by a third party whose interests could change.
That discussion often begins with the location of servers, GPUs, and stored data.
Networking adds another question: where does the data travel?
A system can store information within a particular jurisdiction while using communication paths that cross other jurisdictions or depend on infrastructure controlled elsewhere.
The sovereignty requirement therefore extends to the network’s physical and logical design.
Gartner identifies the need to control the underlying infrastructure and provide mechanisms that govern where data can go.
He also makes a useful distinction: sovereignty does not necessarily require prohibiting every cross-border transfer. The requirement may instead be to define permitted movement and enforce the relevant boundaries.
Those boundaries can reflect the policy of a government, an enterprise, or another organization.
The infrastructure implication is that computing location, storage location, route selection, and operational control need to be considered together.
Fiber and transport planning become part of the organization’s ability to implement its chosen sovereignty policy.
For enterprises, inference and distributed agents make the discussion immediately relevant. The final question brings the conversation beyond hyperscalers and service providers.
Gartner expects inference to be a central enterprise application of AI. He also expects agentic systems to operate across on-premises and external environments.
That means an enterprise can face significant AI networking requirements without operating a frontier-model training cluster.
Its agents may need to exchange information with applications in private infrastructure, services in the cloud, and other agents outside the organization.
The network’s ability to support those interactions will affect the performance of the overall workflow.
Gartner identifies underestimating the network as a major enterprise risk. Model and GPU choices receive substantial attention, while the infrastructure connecting applications and agents can remain an afterthought.
His recommendation is to treat networking as strategic infrastructure within the AI initiative.
The following planning priorities are an analytical extension of the interview’s themes:
Map the complete workflow. Identify where computing, data, applications, and agents reside, along with the communication paths connecting them.
Evaluate performance under realistic conditions. Examine congestion, packet loss, recovery behavior, and tail latency alongside nominal bandwidth.
Connect networking decisions to useful computing output. Assess how waiting and interruption affect the value of the broader infrastructure investment.
Plan power and optical capacity together. Consider the constraints within data centers and at supporting sites along longer routes.
Include operations and security early. Establish how staff will identify failures, correlate alerts, enforce permissions, and manage agent behavior.
Define sovereignty requirements precisely. Determine which infrastructure, locations, routes, and controls must remain within the organization’s authority.
These priorities make the discussion actionable without assuming that every enterprise requires the same architecture.
A customer-service inference workflow, a distributed training system, and a geographically dispersed agent platform have different requirements. Each still benefits from understanding how communication affects the work it is trying to perform.
The central investment question is how much productive AI capability the complete system can deliver, at what operating cost, and with what level of reliability.
Mota and Gartner’s conversation connects that question to infrastructure that can otherwise remain out of view: wavelengths, optical interfaces, fiber routes, amplification sites, routing behavior, operational systems, and security controls.
Each contributes to the ability to move information where it is needed, when it is needed.
That is why networking deserves a central place in AI planning. As computing systems become more distributed and interdependent, the connections among their components increasingly determine the value those components can produce.
A GPU that is waiting for data still consumes capital, space, and power.
The network helps determine when it can get back to work.
Watch the complete interview: Cisco’s Bill Gartner and ACG Research’s Ray Mota discuss AI’s network bottleneck.
Originally published on Ray Mota's Substack. Subscribe to get new analysis by email.
