Can AI CapEx Pay Off? Start With Token Cost, Then Watch the Payback Period.

The most important message from Nvidia's latest shareholder meeting was not another reminder that the world is building "AI factories." It was that Nvidia is trying to give those factories a cost curve.

Over the past year, investors have largely accepted that AI infrastructure spending will keep rising. What they have not accepted is that this spending automatically creates value. The real question is more specific: with the same megawatt of power, the same rack space, and the same dollar of depreciation, how many tokens can an AI data center produce, and how much cheaper are those tokens than they were one generation ago?

Jensen Huang cited SemiAnalysis' InferenceX benchmark and called the GB300 NVL72 the "inference king." The point was not just that Blackwell is faster than Hopper. The point was inference economics. According to Business Insider's summary of the benchmark data, a Hopper system produces roughly 90 tokens per second per GPU, while a top-end Blackwell system produces around 6,000 tokens per second. On a per-megawatt basis, Hopper produces about 54,000 tokens per second, while Blackwell reaches roughly 2.8 million tokens per second, a roughly 50x increase. On cost, the benchmark implies a drop from about $4.20 per million tokens to about $0.12 per million tokens, or roughly 35x lower cost.

That matters because AI CapEx should not be judged only by the price of the GPU system. It should be judged by how many monetizable tokens each unit of capital, power, and data center capacity can produce. If the cost of one million tokens falls from several dollars to a few cents, applications that were previously too expensive to scale begin to look different: coding agents, customer-service agents, ad-generation tools, enterprise knowledge assistants, software testing, and data analysis all have to pass through that cost gate.

Nvidia's answer is not simply a faster GPU. It is system-level co-design. Blackwell, GB300 NVL72, and the coming Vera Rubin platform are built around rack-scale architecture: GPUs, Grace or Vera CPUs, NVLink, networking, memory, and software optimized together. Inference is not just matrix math. An agent has to read context, call tools, access databases, write code, store memory, and test its own output. CPU scheduling, networking, memory bandwidth, and power efficiency all affect token output. If the GPU is waiting on the rest of the system, capital is sitting idle.

That is why Vera Rubin matters beyond the usual next-chip story. Tom's Hardware reported that Nvidia is targeting up to 10x lower cost per token with Rubin NVL72 versus Blackwell, while also reducing the number of GPUs needed to train mixture-of-experts models. Whether the final real-world gains match the target or not, Nvidia's strategy is clear: keep pushing token production costs lower until inference becomes a unit-economics business, not just a technical showcase.

But lower token cost is only the first step. AI CapEx pays off only if those cheaper tokens enter products customers actually pay for, and if the resulting revenue shows up quickly enough in margins and free cash flow.

Microsoft is the most important stress test. It clearly has AI revenue. The company has disclosed that its AI business has surpassed a $37 billion annual revenue run rate, up 123% year over year. Azure remains strong, and Microsoft Cloud generated $54.5 billion in quarterly revenue. The problem is that AI revenue growth has not erased the pressure from infrastructure spending. Microsoft's fiscal Q3 capital expenditures were $31.9 billion, and the company guided to more than $40 billion the following quarter. MarketWatch reported that Microsoft's fiscal 2026 CapEx guidance is approaching $190 billion, while free cash flow has declined by roughly 10%.

That is why the stock has been under pressure despite strong AI headlines. Investors are no longer valuing Microsoft only as a high-margin software compounder. They are also asking whether it is becoming a much more capital-intensive AI infrastructure company.

The tension shows up when AI revenue is translated into cash flow. If Microsoft's AI business is running at $37 billion in annualized revenue, and if mature free cash flow margin is roughly 30%, that implies about $11 billion of annual AI free cash flow. Against nearly $190 billion of annual CapEx, that is not enough to make the payback case obvious. Even if the infrastructure is depreciated over five years, that level of investment creates roughly $38 billion of annual depreciation pressure, before considering power, maintenance, networking, and continued expansion.

This does not mean Microsoft's AI investment is failing. It means the market needs nearer-term proof. The key signals are whether AI revenue keeps accelerating, whether CapEx growth starts to peak, and whether free cash flow stabilizes. If those signals do not appear over the next few earnings cycles, AI revenue growth alone may not be enough to preserve Microsoft's old premium as a capital-light software business.

Amazon and Alphabet face the same issue with different disclosures. AWS generated about $37.6 billion in quarterly revenue, up 28%, and Andy Jassy has said AWS AI has an annualized run rate above $15 billion. But Amazon's quarterly capital expenditures were already above $43 billion, with full-year CapEx expected around $200 billion. Alphabet's evidence leans more toward usage: Google Cloud revenue reached about $20 billion, Gemini API usage hit 16 billion tokens per minute, and token usage was growing rapidly. At the same time, Alphabet also raised its 2026 CapEx guidance to roughly $180 billion to $190 billion.

These companies can show that AI demand is real. They can show that usage is rising. What they have not fully shown yet is that AI infrastructure investment is producing fast enough profit and cash recovery at the company level.

That is the current state of the AI buildout. Nvidia has produced evidence that the unit cost of AI production is falling. Cloud providers have produced evidence that AI revenue and usage are growing. But the payback period is still being tested in public markets.

The framework should be simple: Are tokens getting cheaper? Are those tokens being used in products customers pay for? Are AI revenue and free cash flow growing faster than CapEx, depreciation, and power costs?

Nvidia has answered the first question more clearly than anyone else. Microsoft, Amazon, and Google are still answering the second and third. Cheaper tokens are necessary. They are not sufficient.