SemiAnalysis Outlines AI Chip and Power Bottlenecks

Hyperscalers like Google, Amazon, Meta, and Microsoft are aggressively increasing capital expenditures for 2026, but Taiwan Semiconductor Manufacturing Company (TSMC) remains the primary industry bottleneck. According to SemiAnalysis researcher Jordan Nanos, TSMC is taking a measured approach to capital spending, meaning wafer production cannot keep pace with demand. This constraint is forcing a massive reallocation of TSMC's 3-nanometer capacity away from smartphones and toward AI accelerators. NVIDIA is poised to capture the lion's share of this capacity for its upcoming Rubin GPUs, which will utilize High Bandwidth Memory 4 (HBM4), while competitors like AMD command only a tiny fraction of the supply. Additionally, AI's share of global DRAM wafer capacity is projected to jump from 12 percent in 2023 to over 60 percent next year.

Despite these supply constraints, hardware advancements are yielding massive efficiency gains for software practitioners. Testing by SemiAnalysis reveals that NVIDIA's Blackwell GB200 NVL72 system delivers up to a 50-fold improvement in inference performance over the Hopper H100 generation. While raw floating-point operations increased by less than two times, the leap is driven by networking improvements that resolve bottlenecks. This allows techniques like prefill-decode disaggregation to transfer key-value caches rapidly. Consequently, the cost to serve fast tokens drops from over $2 per million tokens on the H100 to under 10 cents on the GB200.

To support these chips, data centers are scaling at an unprecedented rate, with rack power density skyrocketing from a historical average of 9 kilowatts to 190 kilowatts today, and an expected 600 kilowatts by the end of next year. Companies like Meta and Amazon have already constructed 1-gigawatt facilities, and projects like TeraWulf's 750-megawatt site in New York highlight the massive infrastructure shift. SemiAnalysis models track over 6,000 data centers globally, forecasting that by the end of 2028, there will actually be a surplus of empty data center capacity waiting for chips. For developers, this means that while hardware supply remains tight, the rapid deployment of high-performance systems will ultimately guarantee a future of cheaper, faster, and higher-quality tokens.