Microsoft makes Maia 300 the centrepiece of its AI inference push

The story

Microsoft has unveiled Maia 300, the third generation of its custom artificial-intelligence accelerator, and assigned it a much larger role than an experimental alternative to graphics processors. The company says production deployments will begin in 2027, when it plans to expand the Maia family to multi-gigawatt scale and make it the primary inference system behind Microsoft products and services. Reuters reported that the deployment could eventually involve more than one million chips.

The announcement puts inference—the work of running trained models to answer prompts, write code and carry out multi-step tasks—at the centre of Microsoft's silicon strategy. Maia 200 is already serving Microsoft Copilot and internal model development, with GitHub Copilot and Microsoft Foundry scheduled to follow. Maia 300 is designed for workloads that place sustained pressure on memory and data movement: long-context reasoning, low-latency conversations and agentic processes that repeatedly call tools and generate tokens.

Microsoft's architecture separates compute, memory-controller and input-output functions into specialised chiplets. The compute chiplets use TSMC's N2P process, while other dies can use technologies better suited to bandwidth, connectivity or cost. Each accelerator includes eight HBM4 stacks with 288 gigabytes of capacity and a claimed 18 terabytes per second of peak bandwidth inside a 950-watt thermal envelope. Microsoft says moving from a uniform memory design to a locality-aware arrangement lets software keep data closer to the compute engines that use it.

At rack scale, the company combines 96 accelerators in 24 liquid-cooled trays within a 167-kilowatt envelope. The configuration provides 27.6 terabytes of high-bandwidth memory and 1.7 petabytes per second of aggregate memory bandwidth. A two-tier, Ethernet-based network can join as many as 128 racks into a 12,288-accelerator superpod, while one programming model spans communication inside a rack and across the larger system.

Those specifications support ambitious performance claims. Microsoft expects Maia 300 to deliver 40% more tokens per megawatt than what it calls leading GPU-based systems, up to seven times the rack-scale token throughput of Maia 200 and more than 2.5 times its predecessor's performance per watt. The company has not identified the external systems behind the first comparison or published a reproducible benchmark suite. Reuters also noted that one Maia 300 rack draws about 167 kilowatts, compared with roughly 142 kilowatts for an Nvidia GB300 rack, although the configurations contain different numbers and types of accelerators.

That distinction is important. Tokens per megawatt can be a useful datacentre measure, but it depends on model choice, numerical precision, batch size, latency target, software maturity and how much of the system's power is counted. Maia 300 adds support for enhanced MXFP4 and sub-three-bit storage formats, which can reduce memory traffic and raise throughput, yet narrower representations can affect model quality if not carefully calibrated. Independent results will be needed before Microsoft's headline advantage can be compared across equivalent workloads.

The strategic logic is clearer than the benchmark picture. Microsoft expects more than $175 billion in capital expenditure this calendar year, Reuters reported, and inference costs recur every time a customer uses an AI service. Owning more of the stack gives the company room to co-design models, kernels, networking, cooling and power delivery. It also reduces exposure to the prices and product schedules of Nvidia and AMD, even though Azure will continue to offer a heterogeneous fleet rather than abandoning outside accelerators.

The move intensifies a broader shift among cloud providers. Google has spent years developing Tensor Processing Units, while Amazon operates Trainium and Inferentia families. Custom silicon does not automatically displace general-purpose GPUs: customers value mature software, portability and the ability to run many model types. Microsoft's strongest initial advantage is that it controls large internal workloads and can optimise Maia for them before asking outside developers to adopt the platform.

INNOVOX analysis: Maia 300 is best understood as an infrastructure-economics project. The technical centre of gravity is not peak arithmetic alone but limiting data movement, fitting more memory bandwidth inside a power budget and connecting thousands of accelerators without treating networking as a separate afterthought. If Microsoft can turn those choices into lower cost per useful token at scale, it will gain negotiating leverage across the accelerator market and more control over the margins of its AI services.

What to watch now is execution. Microsoft must translate prototype specifications into reliable volume production, secure N2P and HBM4 supply, mature its software tools and disclose measurements that customers can reproduce. Azure availability, pricing and support for models outside Microsoft's own portfolio will show whether Maia becomes a broadly useful cloud platform or remains primarily an internal optimisation. Microsoft says it will share more at Ignite in November; the decisive evidence will arrive when 2027 production workloads reveal utilisation, latency, energy and cost per token.

INNOVOX analysis

Maia 300 matters less as a single faster chip than as Microsoft's attempt to control the economics of AI inference from silicon through software and datacentre power. If the 2027 rollout reaches multi-gigawatt scale, Microsoft can tune models and infrastructure together while gaining leverage over external accelerator suppliers. The main uncertainty is whether the claimed efficiency survives real workloads, software migration and mass production.

What to watch

Watch for production volumes, customer access through Azure, benchmark methodology, yield and packaging constraints at TSMC, HBM4 supply, software compatibility beyond Microsoft's internal models, and measured cost per token under long-context and agentic workloads. Microsoft Ignite in November is the next announced venue for additional strategy and adoption details.