Where Will Enterprise Data Center Operators Run Their AI?

As more companies put AI capabilities into day-to-day use for employees and customers, CIOs and their data center ops teams face the question: Where should those AI workloads run? On-premises data centers? Colocation facilities? Public clouds? A few recent surveys illuminate how enterprise data center leaders are thinking about this choice.

We’re talking here largely about enterprise data centers, meaning the airlines, banks, manufacturers, universities, retailers, and other organizations that run their businesses using data centers, using AI as a part of that tech stack.

One of the most interesting data points I found on this question points to on-premises data centers falling significantly this past year as the place to run AI workloads. At the same time, on-prem is growing in popularity to hold AI data. Below are more on this and other survey results I found while researching this topic.

Also, a dash of perspective as we look at this. For many enterprise data center operators, AI isn’t yet the biggest priority. Heresy, I know. Bur research and anecdotes suggest that finding the right data center model for AI workloads is a work-in-progress issue this year, something that many are just exploring now.

As one example, I recently talked with the data center leader of a midsized retailer who has one rack of AI compute infrastructure, with no liquid cooling in the data center. AI isn’t a huge share of that data center today. But the company does have an AI leadership group that’s been meeting every week for the past two months, where business unit leads suggest AI use cases and the IT and infrastructure teams helps them assess and execute on those. So, the AI load is sure to grow.

With that, here’s some of the best current research I’ve spotted on the topic:

On-premises data centers dropped dramatically in the past year as the most suitable place to run AI or machine learning workloads

In 2025, nearly half (46%) of enterprise IT leaders cited on-premises as most suitable for AI, but that dropped to 18% this year, in a survey by Foundry, backed by the colo CoreSite, of 300 enterprise IT leaders across industries. Hybrid jumped from 12% to 33% and public cloud rose from 7% to 14%. Colo held steady around 36% this year.

GPUs are moving to public cloud, but AI data is moving away from it

There’s a super interesting and fast shift here spotted by a Flexential survey of 350 IT decision-makers. From 2025 to 2026, the share of companies running most of their GPUs in the public cloud went from 34% up to 54%. On the flip side, companies running most of their GPU through GPU-as-a-service AI specialists dropped 10 points and those doing so in on-premises data centers fell 9 points, to just 4% of orgs.

Meanwhile, AI data shifted in the other direction, away from public clouds. In 2025, 47% of organizations cited public cloud as one place where they housed their AI data, and in 2026 that fell to 26%. Meanwhile, those housing AI data on-premises jumped 10 points (to 30% housing AI data on-prem). Those using colo rose 14 points (to 34%) and hybrid increased 8 points (to 56%).

Most companies have AI in production

The Foundry/CoreSite survey found most companies were in production with GenAI apps (64% in production), chatbots (58%), and agentic AI (49%). Another quarter to a third were in pilot or proof of concept for those uses, and less than 2% had no plans to use them.

AFCOM’s State of the Data Center 2026 survey, published in January, found 74% of organizations planned to deploy AI-capable solutions in their data centers, including to support GenAI (46%), compete in a new market (44%), create a new service (38%), or support client use cases (36%).

But managing AI workloads isn’t necessarily enterprise IT’s biggest headache

Some important perspective: just 22% of respondents to the AFCOM 2026 State of the Data Center report cite managing high-performance workloads such as AI and LLMs as one of their primary challenges. Rising costs for power and cooling (40%), scaling up to meet power and cooling demands (37%), and talent shortages (36%) loom larger.

Research and anecdotes suggest that AI use is definitely into full production in many enterprise organizations, leading to swift shifts in what infrastructure those workloads run on. It also suggests this AI adoption is still in the early stages, meaning more and more dramatic such shifts are coming our way.