US Equities insigtX

Inference Drives Two-Thirds of AI Workloads, Reshaping Chip Deployment Landscape

AI spending is shifting from giant training clusters to smaller, rapidly deployable sites, with modular data centers emerging as the natural home for inference workloads. Today, power, cooling,…

Published
Market
US Equities
Source
insigtX

AI spending is shifting from giant training clusters to smaller, rapidly deployable sites, with modular data centers emerging as the natural home for inference workloads. Today, power, cooling, and memory supply impact token generation as much as the raw performance of the chips themselves. Inference-first chipmakers are looking to capitalize on this shift. Positron AI Inc. raised $875 million at a $5 billion valuation in September, building inference devices around general-purpose memory rather than scarce high-bandwidth memory.

Inference has surpassed training as the driving force behind infrastructure procurement decisions, according to Cameron McCaskill, Positron's head of sales and business development. "We're at a point now where about two-thirds of the workloads running on these AI infrastructures are inference, and one-third is training. And if you go back to 2023, that ratio was roughly 85% training and 15% inference," McCaskill said. "So you can see now why inference is driving the economic decisions of how they're building out their infrastructure."

McCaskill spoke with Rebecca Knight and John Furrier of theCUBE Research during an exclusive broadcast on theCUBE, SiliconANGLE Media's livestreaming studio, at the SC26 conference. They discussed inference overtaking training, Positron's general-purpose memory chip design, and why low-power modular data centers suit inference workloads.

Why Modular Data Centers Fit Low-Power Inference Positron's first-generation Atlas servers run on programmable chips, and the company plans to tape out its own chips for its Titan servers early next year. McCaskill noted that Oracle is bringing roughly 340 Atlas servers online in its cloud this quarter. A new cloud provider has also signed on to use Atlas, with a commitment to adopt Titan. Positron's core design choice is skipping HBM.

"Almost everyone else in the industry is using what's called HBM, or high-bandwidth memory. It's very supply-constrained, and it's very expensive," McCaskill said. "We went with general-purpose memory." This choice also keeps power consumption low enough for air-cooled data halls. McCaskill explained that a single Positron server draws about 4,500 watts, and a full rack of Titan servers stays within 40 kilowatts. This power footprint suits sites where operators can find a few megawatts of cheap electricity without facing heavy regulatory hurdles.

Deployment speed is a key part of the appeal for inference providers looking to monetize new capacity, McCaskill said. "We're seeing a class of companies emerging now that are coming to us, and that's modular data centers—they have 0.5-megawatt, 1-megawatt, or 2.5-megawatt modular data centers that can be deployed very quickly," McCaskill said. "And the goal for many inference players is to go from securing land and power to profitable token generation in 90 to 120 days. Those modular data centers, combined with our technology, can be a great solution."

The following is the full video interview, part of SiliconANGLE and theCUBE's coverage of the SC26 conference:

(* Disclosure: NYSE Wired sponsored this segment of theCUBE. NYSE Wired and other sponsors have no editorial control over theCUBE or SiliconANGLE content.)

Original: https://siliconangle.com/2026/11/21/inference-two-thirds-ai-workloads-chip-deployment-positron-sc26/

insigtX content is informational and educational, not investment advice.