Over the past few days, the scale of the AI infrastructure buildout has become increasingly difficult to comprehend.
Nvidia is reportedly working with some of the largest names on Wall Street around financing that could support as much as $500 billion of AI infrastructure. At the same time, the hyperscalers are planning hundreds of billions of dollars of capital spending on data centers, GPUs, networking and power.
The numbers are staggering.
And they keep bringing me back to a very simple question:
Will the economic returns ultimately justify the amount of capital being deployed?
Or, put another way:
Build it and they will come. But what if they don't — at least not in the way today's investment assumptions anticipate?
The infrastructure case is compelling
There is certainly a powerful argument for making these investments.
AI adoption is growing rapidly. Models are becoming more capable. Training requires enormous amounts of compute. Inference demand could ultimately dwarf training demand as AI becomes embedded in everyday applications and business processes.
If AI becomes as fundamental to computing as many expect, we may look back at today's data center buildout the way we look at earlier investments in telecommunications, electricity or the internet.
The infrastructure has to exist before much of the innovation can happen.
But there is another side of this equation that I believe deserves more attention.
Someone ultimately has to generate the return
For much of my career, I sat on the enterprise side of major technology transformations.
That perspective makes me look at today's AI investment boom somewhat differently.
The infrastructure providers can build data centers. Nvidia and others can manufacture increasingly powerful chips. Model companies can build increasingly capable models.
But ultimately, enterprises have to turn all of this technology into economic value.
They need production applications that either generate revenue, reduce costs, improve productivity, reduce risk or create some other measurable business outcome.
And getting from an impressive AI demonstration to a production system inside a large enterprise is considerably harder than it sometimes appears from Silicon Valley.
The model may be the cheapest part of the solution
There is enormous attention on model costs and compute costs.
Those costs matter.
But inside a large, regulated enterprise, they may be only one component of the total cost of deploying AI.
Consider what surrounds the model:
- Data access and data quality
- Integration with existing applications
- Cybersecurity
- Identity and access management
- Model governance
- Compliance
- Auditability
- Testing and controls
- Operational resilience
- Regulatory requirements
- Monitoring
- Legacy technology integration
And eventually one of the most important questions:
Who owns the risk when the AI gets something wrong?
None of these problems are particularly glamorous. They don't generate the excitement of a new model release.
But they are exactly the kinds of issues that determine whether an AI proof of concept becomes a production application used by thousands of employees or customers.
They also cost real money.
When CEOs discuss the productivity benefits of AI, I would like to hear more discussion about this total cost of implementation.
Then there is the domain problem
There is another issue I believe the market may be underestimating: domain expertise.
Many technology companies are positioning AI as capable of transforming banking, healthcare, insurance, capital markets and virtually every other major industry.
Their technology may be extraordinary.
But knowing how to build an AI model is very different from understanding how a large financial institution actually operates.
A trading system, for example, isn't simply software that buys and sells securities. There are workflows, controls, risk systems, market data, regulatory obligations, settlement processes, exception handling, entitlements, audit requirements and decades of interconnected technology.
The same is true in healthcare, insurance and other complex industries.
Successful enterprise AI will require more than great AI engineers.
It will require combining great technology with deep domain expertise.
I suspect that combination will eventually separate many of the winners from the losers.
And what if inference itself changes?
There is another assumption behind today's infrastructure buildout that I've been thinking about.
We tend to assume that enormous growth in AI usage means enormous growth in centralized cloud inference.
Perhaps it will.
But technology rarely develops in a straight line.
Open-weight models are improving rapidly. Smaller specialized models are becoming increasingly capable. Phones and PCs are gaining powerful AI hardware. Enterprises can run models on their own infrastructure.
That creates an interesting possibility.
What if AI inference becomes increasingly hybrid?
Train the largest models centrally.
Run the most computationally demanding workloads in massive data centers.
But move appropriate inference closer to where the data and users actually reside — enterprise servers, PCs, phones and edge devices.
There are compelling reasons to do so:
- Cost
- Latency
- Privacy
- Security
- Data sovereignty
- Availability
And eventually enterprises will see the full bill for AI inference.
When they do, CFOs and CIOs will inevitably ask:
Does every one of these requests really need to go to an expensive cloud model?
Some will.
Some won't.
That doesn't necessarily mean less cloud
There is an important counterargument.
Cheaper inference — whether local or cloud-based — could dramatically expand the number of AI applications.
This has happened repeatedly in technology.
When computing becomes cheaper, we don't necessarily spend less on computing. We often find thousands of new things to do with it.
Local AI could therefore reduce the cloud resources required for an individual task while simultaneously helping create millions of new AI workloads.
Total cloud demand could continue growing rapidly.
Which brings us back to the question I started with.
Follow the economics
We are becoming very good at measuring how many billions of dollars are being invested in AI.
Perhaps the more important metric will eventually be:
How many dollars of sustainable economic value are being created for every dollar invested in AI infrastructure?
The answer won't be the same everywhere.
There will almost certainly be enormous winners.
There will probably also be infrastructure that isn't utilized as expected, AI projects that never reach production, vendors whose economics don't work, and enterprises that discover their implementation costs are substantially higher than anticipated.
None of that means the AI revolution isn't real.
I believe it is.
But a transformative technology and a successful investment are not necessarily the same thing.
The internet transformed the world. That didn't mean every investment made during the internet infrastructure boom generated an attractive return.
AI may ultimately transform the world on an even larger scale.
The investment question is who captures the value — and whether the value captured justifies the extraordinary amount of capital being deployed today.
That is the question I will be watching.
Inside Enterprise AI is a practitioner's perspective on AI, cloud and technology transformation — and the economics of making them work inside large enterprises.