83% of Organisations Expect AI Infrastructure Overhaul as Agentic AI Reshapes Enterprise Technology

The enterprise AI race is no longer just about building smarter models. It is increasingly about whether existing IT infrastructure can support them.

According to Google’s latest State of Infrastructure report, 83% of organisations expect to make significant infrastructure upgrades before deploying agentic AI systems at production scale. The findings suggest that legacy IT environments, fragmented data ecosystems and rising operational complexity are becoming the biggest barriers to enterprise AI adoption.

Infrastructure becomes the biggest AI bottleneck

Unlike conventional AI applications that respond to individual prompts, agentic AI systems are designed to independently plan, reason and execute multi-step tasks across multiple enterprise applications. These systems interact with databases, business software, APIs and cloud platforms, making infrastructure performance far more critical than before.

The report indicates that many organisations are discovering their existing technology stacks were never designed for this level of AI-driven automation. Faster data access, high-performance computing, resilient networking and unified data platforms are becoming essential for deploying AI agents effectively.

Legacy systems remain a major challenge

Google Cloud found that 43% of IT leaders consider integrating legacy APIs and existing enterprise data sources the biggest obstacle to implementing agentic AI.

Many enterprises continue to rely on disconnected applications and fragmented databases, making it difficult for AI agents to access accurate, real-time information across departments such as finance, procurement, operations and customer service.

The report argues that enterprise data quality and accessibility have become more important than AI model capability. Without secure, unified access to business information, AI agents cannot deliver reliable outcomes at scale.

AI deployments are increasing operational costs

Infrastructure complexity is also driving up the cost of AI implementation.

According to the research, 81% of technology leaders reported that engineering overhead and operational complexity have become major unexpected expenses while scaling AI initiatives.

Instead of focusing solely on developing AI applications, engineering teams are spending significant time connecting legacy software, databases and cloud environments before AI services can be deployed.

This growing complexity is encouraging enterprises to modernise their cloud infrastructure and adopt more integrated architectures capable of supporting enterprise-scale AI workloads.

Google introduces Agentic Data Cloud

To address these infrastructure challenges, Google Cloud has introduced its Agentic Data Cloud framework.

The platform is designed to connect AI models, enterprise databases and analytics platforms within a unified architecture, allowing AI agents to move seamlessly between reasoning and execution without requiring repeated manual integrations.

Google also highlighted support for open technologies such as Apache Spark and Apache Iceberg, alongside services including BigQuery and Spanner, to improve interoperability across hybrid and multi-cloud environments.

Demand rises for AI-ready infrastructure

The report further found that 36% of respondents identified the lack of high-performance vector databases as another critical infrastructure gap.

Vector databases enable AI systems to understand relationships between enterprise data, improving the accuracy and relevance of AI-generated responses.

As organisations transition from AI experimentation to full-scale deployment, demand is expected to grow for specialised AI infrastructure, including advanced computing hardware, high-speed networking, modern storage systems and stronger data governance capabilities.

AI infrastructure investment enters the spotlight

Google Cloud’s research suggests enterprise AI is entering a new phase where infrastructure investment is becoming as important as AI software development itself.

For cloud providers, data centre operators and enterprise technology vendors, the shift is expected to create significant demand for AI-ready infrastructure as businesses prepare for the next generation of intelligent automation.