Enterprise AI bottleneck shifts from models to legacy infrastructure
Tech leaders from LinkedIn, Walmart and Zendesk warned that outdated enterprise systems, rather than AI models, are the primary constraint on deploying agents at scale, signaling a shift in corporate IT budgets.
At VB Transform 2026, infrastructure leaders from LinkedIn, Walmart and Zendesk agreed that moving AI agents from pilots to production is blocked by legacy systems designed for human workers. The panel concluded the engineering challenge now lies in closing the speed gap between millisecond-acting agents and enterprise infrastructure built for manual processes.
Each company hit a different version of this wall. At LinkedIn, Animesh Singh, senior director of AI platform and infrastructure, found that Kubernetes takes too long to spin up containers on demand. The company solved this by pre-provisioning pools of containers to swap workloads in real time.
Walmart faced a governance crisis when an internal agent tool went viral, creating dozens of overlapping tools built by "citizen developers." Desiree Gosby, SVP of corporate technology services and technology strategy, said the fix was building governance to spot duplication rather than restricting the tool. Zendesk's bottleneck was data-driven, with Sami Ghoche, VP of applied AI, noting that a public figure of 20 billion customer conversations cannot simply be dumped into a large language model's context window.
"You can't really do that, so instead you have to really invest in the underlying data pipelines and all the data infrastructure that comes with that," Ghoche said. LinkedIn also discovered that relying on LLMs to evaluate other LLMs caused persistent hallucinations, forcing the company to shift 80% of its workflows to deterministic code.
Capex shifts to data plumbing
For investors, these findings suggest the next phase of enterprise AI spending will pivot away from purchasing frontier model access. Capital is instead flowing toward modernizing data pipelines, building governance layers, and developing custom orchestration harnesses. Walmart's experience specifically highlights the financial risk of uncoordinated AI adoption, where duplicated efforts waste engineering resources.
To protect these infrastructure investments, all three companies are building systems to avoid vendor lock-in. LinkedIn developed an internal AI gateway that standardizes outbound API calls, allowing rapid switching between cloud providers or on-premises data centers. "Every single outbound call going to an LLM, whether it's on a public cloud or on-prem in our own data centers, follows the same semantics, the same API calls," Singh said.
Walmart built a similar gateway to route workloads based on effectiveness rather than a fixed policy, keeping compliance-heavy tasks strictly deterministic. The executives advised companies to prioritize evaluation frameworks before building. "It'll force you to break the problem down, and once you have a robust set of evals, you can move a lot faster," Ghoche said.
Gosby urged putting agent-building tools directly into employees' hands early, provided monitoring infrastructure is in place. Singh stressed building for independence to protect future flexibility. "Keep that context within your enterprise so that you can reuse it when you ship the model or the harness tomorrow," he said.
