CPUs are Back: The Datacenter CPU Landscape in 2026
SemiAnalysis · Gerald Wong, Dylan Patel
CPUs are back at the center of datacenter spend in 2026, with frontier AI labs triggering a shortage by buying x86 servers for RL training while AMD's chiplet stack, ARM's move into selling its own Meta-designed CPU, and Intel's stumbles with Clearwater Forest redraw the competitive map.
The CPU stagnation story of the GPU boom years is over. Reinforcement learning workloads at OpenAI and others now consume so many commodity cores that labs are bidding against clouds for supply, just as the vendor landscape fragments: AMD rides a single CCD tapeout across the entire SKU stack with yield and process-migration advantages mesh designs can't match, ARM has crossed from licensor to vendor by shipping Phoenix directly to Meta with Cloudflare and OpenAI behind it, and Intel's Clearwater Forest looks more like a Foveros Direct learning vehicle than a volume product given its 17% gain over Sierra Forest and weak hybrid bonding yields. The stakes are which architecture and which vendor captures the next cycle of datacenter buildout.
Frontier AI labs are running out of CPUs for reinforcement learning training and are competing directly with cloud providers for commodity x86 servers, igniting an unexpected shortage as we move through 2026.
A single CCD design, varied in count around a shared I/O die, spans the entire core-count range with better yields and faster process migration. Mesh designs by contrast need a separate reticle-sized tapeout per core-count tier.
In 2026 ARM ships Phoenix, a full datacenter CPU designed end-to-end for Meta, with Cloudflare and OpenAI also lined up — directly competing with the hyperscalers it licenses Neoverse to.
After years of stagnation while GPUs and networking dominated datacenter spend, CPU demand has inflected upward in late 2025, marking a real shift in the CPU's role in the AI datacenter.
Despite a two-year gap, new cores, new node, and new packaging, Clearwater Forest is only 17% faster than Sierra Forest at equal core counts. Combined with low hybrid bonding yields, this suggests Intel intends it more as a Foveros Direct learning exercise than a high-volume product.
Open
- · Can Intel recover volume competitiveness after using Clearwater Forest as a learning vehicle?
- · How will hyperscalers respond to ARM directly competing with their Neoverse-based custom silicon?
- · How long will the RL-driven CPU shortage persist, and does it ease as RL infrastructure matures?
Pipeline
- source kind
- url
- generated by
- anthropic+voyage
- candidates
- 92 (selected 5)
- embeddings
- voyage-3.5
Sections
Candidate pool grouped by section. Selected candidates are bolded.
Considered candidates (87)
Redundant with selected · 16
- mechanismReinforcement learning is the biggest new CPU demand driverc 0.90 · sim 0.87
RL training loops need huge CPU clusters to run code compilation, verification, tool use, and physics simulation in the reward environment, with growing RL complexity demanding ever larger high-performance CPU clusters next to the GPUs.
overlapped with: RL training is driving a 2026 CPU shortage
- claimDatacenter CPUs split into two rolesc 0.85 · sim 0.89
As GPUs took the spotlight, CPUs bifurcated into head-node CPUs that feed GPUs and cloud-native CPUs optimized for throughput-per-watt to serve the rest of the internet.
overlapped with: CPUs are relevant again after being sidelined by the GPU boom
- implicationCPU-to-GPU power ratios will keep risingc 0.85 · sim 0.82
Because AI accelerators are improving perf-per-watt much faster than CPUs, future generations like Rubin may need an even higher CPU-to-GPU power ratio than the 1:6 seen at Fairwater, making CPUs a structural bottleneck in AI training.
overlapped with: RL training is driving a 2026 CPU shortage
- mechanismAgentic and RAG inference is driving a step-change in CPU demandc 0.80 · sim 0.83
RAG and agentic models hammer the internet through API calls and database queries far more intensively than humans, pushing AWS and Azure into massive Graviton, Cobalt, and x86 buildouts to service the traffic.
overlapped with: RL training is driving a 2026 CPU shortage
- claimRome's central I/O die rethink leapfrogged Intel to 64 coresc 0.80 · sim 0.82
In 2019, AMD surrounded a central I/O die with eight 8-core CCDs on TSMC N7 while keeping the I/O die on GlobalFoundries 12nm. Routing all inter-CCX traffic through the I/O die collapsed the topology to two NUMA domains and doubled core count past Intel's 28.
overlapped with: Chiplets let AMD cover the whole SKU stack from one CCD tapeout
- evidenceIntel's Q4 results show the CPU inflectionc 0.70 · sim 0.86
Intel reported an unexpected uptick in datacenter CPU demand in late 2025 and is raising 2026 capex on foundry tools while reallocating wafers from PC to server to meet the new demand.
overlapped with: CPUs are relevant again after being sidelined by the GPU boom
- contextReticle limits force the move to chipletsc 0.70 · sim 0.84
By Ice Lake, Intel hit the 26x33mm reticle limit at 40 cores. Adding AMX engines in Sapphire Rapids meant a monolithic die would have regressed to 34 cores, leaving chiplets as the only path forward.
overlapped with: Chiplets let AMD cover the whole SKU stack from one CCD tapeout
- implicationNVIDIA's Bluefield-4 signals that CPU deployments are evolvingc 0.70 · sim 0.83
The inclusion of NVIDIA's Bluefield-4 in the roadmap reflects a shift in how CPUs are deployed, with DPUs increasingly absorbing roles traditionally held by host CPUs.
overlapped with: CPUs are relevant again after being sidelined by the GPU boom
- contextARM's Venom and Qualcomm's SD2 reshape the merchant CPU fieldc 0.60 · sim 0.83
ARM is preparing its Venom core specifications while Qualcomm re-enters the datacenter CPU market with SD2, broadening the set of credible non-x86 options.
overlapped with: ARM is becoming a chip vendor and competing with its own licensees
- caveatThe DRAM shortage cuts across every datacenter CPU segmentc 0.60 · sim 0.84
The ongoing DRAM shortage is expected to materially affect each datacenter CPU segment, shaping deployment economics regardless of vendor or architecture.
overlapped with: CPUs are relevant again after being sidelined by the GPU boom
- contextIntel missed the AI buildout wavec 0.55 · sim 0.82
As hyperscalers poured capex into GPUs, Intel's server CPU revenue stayed flat, and it lacked a competitive AI accelerator to participate in the boom.
overlapped with: CPUs are relevant again after being sidelined by the GPU boom
- examplePhoenix gives Meta a hyperscaler-grade ARM CPUc 0.55 · sim 0.83
Phoenix uses 128 Neoverse V3 cores across two 3nm reticle-sized dies with 12-channel DDR5-8400 and 96 lanes of PCIe6, plus coherent shared memory to XPUs for use as an AI head node.
overlapped with: ARM is becoming a chip vendor and competing with its own licensees
- contextHyperscalers built their own ARM CPUs, eroding Intel's marketc 0.50 · sim 0.83
AWS, Microsoft, and Google rolled out in-house ARM datacenter CPUs, closing off a major addressable market, while AMD chipped away at Intel's remaining x86 share.
overlapped with: ARM is becoming a chip vendor and competing with its own licensees
- caveatHybrid bonding bandwidth is surprisingly lowc 0.40 · sim 0.85
Clearwater Forest's vertical interconnect delivers only 35GB/s per 4-core cluster between the compute die and the base die's L3 and mesh, a constraint likely tied to Intel's struggles bringing Foveros Direct into production.
overlapped with: Clearwater Forest looks like a yield learning vehicle, not a volume product
- contextCloud computing was the prior decade's CPU growth enginec 0.35 · sim 0.83
The 2010s saw CPU demand driven by the consolidation of compute into public clouds like AWS, with virtualization letting a single CPU host many secure VMs.
overlapped with: CPUs are relevant again after being sidelined by the GPU boom
- contextThe section also previews longer-term CPU trends beyond individual roadmapsc 0.30 · sim 0.84
Beyond per-vendor roadmaps, the discussion extends to future CPU trends that will shape the datacenter landscape past the 2028 horizon.
overlapped with: CPUs are relevant again after being sidelined by the GPU boom
Below top-k · 71
- claimDropping SMT cripples Diamond Rapids in the datacenterc 0.85
Intel's post-Spectre P-core design omits SMT, which was tolerable on client chips backed by E-cores but devastating where throughput is the whole point. Diamond Rapids' 192 cores deliver 192 threads against Granite Rapids' 256, and we expect only ~40% more performance.
- claimThe AMD-Intel performance gap will widen through 2028c 0.85
AMD already enjoys higher per-core performance—96-core Turin matches 128-core Granite Rapids—so Venice versus Diamond Rapids should stretch the lead further. Venice's new die-to-die fabric and larger core domains also improve core-to-core latency over Turin.
- mechanismMesh interconnects became the scalable answerc 0.80
Intel's 2017 Skylake-X adopted a 2D mesh borrowed from Xeon Phi, arranging cores in a grid with half-rings along rows and columns. This mesh architecture became the foundation that has scaled core counts for the next decade.
- claimIntel's Diamond Rapids finally copies AMD's central-I/O chiplet layoutc 0.80
Unable to grow a single mesh past Granite Rapids' 10x19, Intel splits Diamond Rapids into four Core Building Block dies flanking two I/O and Memory Hub dies. The move concedes multiple NUMA nodes and L3 domains for the first time.
- evidenceMicrosoft Fairwater dedicates 48MW of CPU to a 295MW GPU clusterc 0.75
OpenAI's Fairwater datacenter pairs a 48MW CPU and storage building with the main 295MW GPU cluster, showing that tens of thousands of CPUs are now needed to manage the petabytes of data GPUs generate.
- implicationCancelling 8-channel Diamond Rapids-SP cedes the mainstream socketc 0.75
Intel's late cancellation leaves its highest-volume market without a new generation until at least 2028. That hurts precisely where AI tool-use and context-storage workloads sit, which favor mainstream CPUs with good connectivity over peak-per-socket monsters.
- implicationAMD's new 8-channel SP8 platform will eat Intel's enterprise sharec 0.75
While Intel kills its 8-channel SKU, AMD introduces an 8-channel Venice SP8 platform with up to 128 Zen 6c cores as a Siena successor. AMD is positioned for large share gains in enterprise, traditionally an Intel stronghold.
- claimAWS was first hyperscaler to ship its own cloud CPUc 0.75
By combining the Annapurna Labs acquisition with ARM Neoverse reference designs, AWS bypassed Intel and went directly to TSMC, capturing better margins on EC2.
- mechanismMerchant ARM lost because enterprise software moves slowlyc 0.75
Hyperscalers could port internal workloads quickly to their own ARM silicon, but the enterprise market that Ampere depended on was too slow to adopt ARM, and AMD eventually matched its core counts with much higher per-core performance.
- mechanismGPUs displaced CPUs because AI is mostly matrix multiplicationc 0.70
AI training and inference are dominated by easily parallelized matrix math, which runs 100-1000x worse on CPUs than on GPUs with thousands of vector units and dedicated tensor cores.
- mechanismCloud-native CPUs trade features for throughput-per-wattc 0.70
Cloud-native CPUs use higher counts of medium-sized, area- and power-efficient cores with less cache and IO, enabling 10:1 socket consolidation and freeing power for GPUs.
- evidenceIntel raising Xeon prices as inventory depletesc 0.70
Facing unexpected depletion of CPU inventory, Intel is raising prices across its Xeon line while ramping additional tooling to expand production.
- contextOn-die data fabrics become the central design problemc 0.70
As CPUs moved from dual-core in 2005 to hundreds of cores today, the way cores are interconnected on die has become the dominant constraint in datacenter CPU design.
- evidenceSierra Forest's cloud-native bet largely failedc 0.70
Despite being built at hyperscaler request for lower TCO per core, Sierra Forest saw limited adoption: hyperscalers had already moved to AMD or their own ARM designs, and enterprise customers weren't interested. The 288-core Sierra Forest-AP never made general availability.
- claimVenice adopts advanced packaging just as Intel walks away from itc 0.70
AMD's 2026 Venice finally moves CCD-to-I/O links onto an EMIB-equivalent advanced package, the same generation Intel retreats to substrate traces. The extra shoreline forces the I/O hub to split into two dies, adding a new NUMA hop Intel avoids.
- claimVera brings Nvidia back to custom ARM cores with SMTc 0.70
Stung by Neoverse bottlenecks, Nvidia revives its custom core team with Olympus, an ARMv9.2 design that reinstates SMT for 88 cores and 176 threads. Wider FP ports, SVE2 FP8 support, and doubled L2 underpin a claimed 2x performance jump over Grace.
- claimAmpere collapsed and was absorbed by SoftBankc 0.70
SoftBank acquired Ampere for $6.5B in 2025 partly to feed its Stargate CPU effort and partly because Oracle wanted out of a failing business that never reached volume.
- claimHuawei's Kunpeng 960 extends the roadmap into 2028 with a two-variant splitc 0.70
Huawei plans to continue its datacenter CPU roadmap through 2028 with the Kunpeng 960 series, following an established pattern of splitting the design into two distinct variants tuned for different workloads.
- implicationHuawei is positioned to take meaningful share of Chinese hyperscaler CPU deploymentsc 0.70
By 2028, Huawei is expected to capture significant share of CPU deployments at Chinese hyperscalers, reflecting both domestic substitution and its strengthening roadmap.
- implicationGravitons now design Gravitonsc 0.65
AWS runs EDA and CI/CD flows for Graviton, Trainium and Nitro silicon on thousands of internal Graviton CPUs, creating a self-reinforcing dogfooding cycle.
- evidenceOracle's Ampere purchases collapsed from $48M to under $4Mc 0.65
Oracle's Ampere CPU spend fell from $48M in fiscal 2023 to $3M in 2024 and $3.7M in 2025, with Oracle never exhausting its pre-payment.
- mechanismHead-node CPUs prioritize per-core speed and bandwidth to GPUsc 0.60
Head-node CPUs need high per-core performance, large caches, and high memory and IO bandwidth to keep GPUs fed, with designs like NVIDIA's Grace offering coherent memory access for KV-cache expansion.
- evidenceAMD expects strong double-digit server CPU TAM growthc 0.60
AMD is expanding supply capability to gain share in a server CPU market it believes will grow in the strong double digits in 2026.
- mechanismRing buses extended scaling but introduced non-uniform latencyc 0.60
Intel's Nehalem-EX in 2010 adopted a ring bus where data hops one stop per clock around a loop of L3 cache slices. This let core counts grow to 8-10, but core-to-core latency became non-uniform depending on position around the ring.
- mechanismEMIB stitches a mesh across multiple diesc 0.60
Sapphire Rapids used Intel's EMIB packaging and a Modular Die Fabric to extend the mesh across four 15-core dies, creating an 8x12 mesh spanning nearly 1600mm² that still appeared logically monolithic to software.
- mechanismXeon 6 disaggregates I/O onto an older nodec 0.60
Xeon 6 splits I/O onto Intel 7 dies while compute moves to Intel 3, letting Intel reuse Sapphire Rapids I/O IP and save cost since I/O does not benefit much from advanced nodes. The same I/O dies pair with both P-core Granite Rapids and E-core Sierra Forest compute.
- mechanismClearwater Forest stacks compute on a base die via hybrid bondingc 0.60
Clearwater Forest uses Foveros Direct hybrid bonding to stack 18A compute dies on base dies containing the mesh, L3 and memory interface, reaching 288 cores. This vertical disaggregation keeps non-scaling logic on Intel 3 while compute moves to 18A.
- contextAMD's 2017 EPYC Naples return forced chiplet design out of necessityc 0.60
AMD re-entered the datacenter CPU market in 2017 with a small team that could only afford to tape out a single die, reused across desktop, server, and embedded. Intel mocked it as 'four glued-together desktop die,' but the constraint birthed the chiplet approach.
- caveatNaples paid for its core count with four-NUMA-domain latency chaosc 0.60
A dual-socket Naples system had four NUMA tiers (intra-CCX, inter-CCX, die-to-die, inter-socket), and most software was not NUMA-aware. Embarrassingly parallel workloads thrived while memory- and latency-sensitive ones suffered, giving Intel's 'inconsistent performance' jab real teeth.
- contextNvidia Grace is built around feeding GPUs, not raw CPU throughputc 0.60
Grace pairs ARM Neoverse V2 cores with NVLink-C2C at 900 GB/s, LPDDR5X memory at 500 GB/s, and up to 480 GB capacity to serve as extended GPU memory. The 3.2 TB/s mesh bisection bandwidth reflects priorities tilted toward data flow.
- mechanismGraviton2 won customers by undercutting x86 on pricec 0.60
AWS used heavy discounting during the COVID boom to migrate workloads from x86, trading lower per-core performance for far better performance per dollar at 64 Neoverse N1 cores.
- exampleGraviton5 doubles cores and leans on cache to hide memory gapc 0.60
Graviton5 hits 192 Neoverse V3 cores on TSMC 3nm with 172B transistors, growing shared L3 from 36MB to 192MB to compensate for memory bandwidth only rising 57% as core count doubled.
- implicationGraviton becomes the head node for Trainium clustersc 0.60
Trainium3 systems use Graviton CPUs as head nodes at a 1:4 CPU-to-XPU ratio, tightly binding AWS's custom CPU and AI accelerator roadmaps.
- caveatMicrosoft keeps Intel for AI head nodesc 0.60
Unlike AWS, Microsoft confines Cobalt 200 to general purpose compute and uses Intel Granite Rapids as the head node CPUs in its Maia 200 rackscale system.
- evidenceOver 1 billion Neoverse cores deployedc 0.60
ARM has shipped over a billion Neoverse cores across datacenter CPUs and DPUs with 21 CSS licenses across 12 companies, more than doubling datacenter royalty revenue year-over-year.
- evidenceSanctions stretched Huawei's CPU cadence to five yearsc 0.60
After the 2019 Kunpeng 920, Huawei's next part — the Kunpeng 920B with 80 SMT-enabled cores — only arrived in 2024, a gap attributed to US sanctions forcing redesigns.
- mechanismA high-performance variant targets AI head nodes and databasesc 0.60
The performance-oriented Kunpeng 960 will offer 96 cores and 192 threads with a promised 50%+ improvement in per-core performance, aimed at AI head nodes and database workloads.
- mechanismA high-density variant pushes core counts to 256 or beyondc 0.60
The density-oriented Kunpeng 960 variant targets virtualization and cloud compute, scaling to 256 cores and possibly higher to maximize throughput per socket.
- mechanismVenice CCDs move to an internal mesh with 32 Zen6c cores eachc 0.55
Eight TSMC N2 CCDs in a 4x8 mesh push core counts to 256, a third more than Turin-Dense. Zen6c gets the full 4MB L3 per core that was halved on Zen5c, creating 128MB cache pools per CCD.
- evidenceVenice claims 1.7x perf/W over Turin with new AI instructionsc 0.55
AMD says the 256-core Venice is over 1.7x better in SPECrate2017_int_base perf/W than 192-core Turin, implying higher per-core throughput from Zen 6 IPC gains. Zen 6 also adds AVX512_FP16, AVX_VVNI_INT8, and AVX512_BMM.
- evidenceGrace's Branch Target Buffer chokes when hot code spans too many regionsc 0.55
Neoverse V2 organizes instructions into 32 2MB virtual regions; performance collapses past 24 as the BTB thrashes, and a 32-region overflow flushes the entire 64MB buffer. Hand-optimizing code locality can yield 50% speedups.
- exampleGraviton3 beat AMD and Intel to DDR5 and PCIe5c 0.55
With a chiplet design using TSMC N5 compute, EMIB packaging, and Neoverse V1 cores, Graviton3 shipped DDR5 and PCIe5 a full year ahead of Sapphire Rapids and AMD's competing parts.
- exampleGraviton5 cut PCIe lanes to save TCOc 0.55
PCIe lane counts dropped from 96 to 64 on Graviton5 because AWS observed customers rarely used all lanes — a cost optimization with no performance impact.
- implicationGoogle migrating its own services to ARMc 0.55
Gmail, YouTube and Google Play are being moved onto Axion alongside x86, and Google plans future Axion CPUs as head nodes for TPU clusters running Gemini.
- exampleKunpeng 950 brings Huawei back to the leading edgec 0.55
For 2026, Huawei's Kunpeng 950 doubles to 192 cores on a new LinxiCore and ships in TaiShan 950 SuperPoD racks, claiming 2.9x OLTP gains using its GaussDB Multi-Write architecture.
- mechanismCrossbars scale quadratically and hit a wall at four coresc 0.50
All-to-all crossbar interconnects require N(N-1)/2 links, growing from 1 connection at 2 cores to 28 at 8 cores. This complexity limited practical crossbar designs to roughly four cores.
- mechanismStitching multiple rings created NUMA penaltiesc 0.50
To push past single-ring limits, Haswell and Broadwell connected two counter-rotating rings via buffered switches, scaling to 24 cores. But crossing between rings doubled access latency from under 50ns to over 100ns, hurting latency-sensitive workloads.
- mechanismNaples used a 4-die MCM stitched together by Infinity Fabricc 0.50
Each Zeppelin die held two 4-core CCXs connected by a crossbar, with Infinity Fabric on Package linking the four dies and Infinity Fabric Inter Socket enabling dual-socket coherence. This let AMD reach 32 cores against Intel's 28.
- mechanismDiamond Rapids stacks compute on a base cache die and pairs cores in DCMsc 0.50
Each CBB hybrid-bonds 32 Dual Core Modules on Intel 18A-P onto a base Intel 3-PT die holding L3 and the local mesh, with two cores sharing an L2 inside each DCM. The total reaches 256 cores, but mainline SKUs cap at 192 due to yield.
- mechanismIntel drops EMIB and routes CBB-to-IMH over the substratec 0.50
Long substrate traces give each CBB direct access to both IMH dies, capping inter-CBB hops at two. The downside is that cross-CBB latency will be appreciably worse than staying on-die.
- evidenceVenice's 16-channel MRDIMM memory delivers 2.67x Turin bandwidthc 0.50
The split I/O dies host 16 memory channels, up from 12 in Genoa, and AMD finally supports multiplexed memory. MRDIMM-12800 yields 1.64 TB/s of bandwidth per socket.
- implicationGrace's branch predictor bottleneck is slowing AI workloads on GB200/GB300c 0.50
The buffer-flush behavior is the reason Grace currently drags down AI workloads in the GB200 and GB300 systems.
- mechanismVera disaggregates memory and I/O into a six-die CoWoS-R packagec 0.50
Vera in 2026 doubles C2C bandwidth to 1.8 TB/s, widens memory to 1.5 TB across eight 128-bit SOCAMM modules at 1.2 TB/s, and packages one 3nm compute die, four LPDDR5 dies, and a PCIe6/CXL3 I/O die together. L3 grows to 162 MB across a 7x13 mesh.
- exampleMicrosoft Cobalt 200 jumps a core generation rather than core countc 0.50
Cobalt 200 grew only modestly from 128 to 132 cores but moved from Neoverse N2 to V3 across two TSMC 3nm dies, yielding a 50% speedup over Cobalt 100.
- exampleGoogle's Axion enters at two tiersc 0.50
Google's Axion line spans high-performance C4A instances with up to 96 Neoverse V2 cores and cheaper N4A instances with 64 Neoverse N3 cores on a smaller 3nm die for scale-out services.
- mechanismAmpereOne optimized for density and noisy-neighbor isolationc 0.50
AmpereOne reached 192 custom cores on 5nm using an interposer-less chiplet design and oversized 2MB L2 per core, prioritizing density and isolating tenants from shared-mesh contention.
- contextHuawei is China's serious datacenter CPU effortc 0.50
While Loongson and Alibaba's Yitian exist, Huawei's Kunpeng line — backed by HiSilicon's custom TaiShan cores and fabrics — is the most credible Chinese datacenter CPU program.
- contextAMD's roadmap extends through Verano and Florencec 0.50
AMD's datacenter CPU roadmap beyond 2026 includes the Verano and Florence generations, continuing its post-Zen5 trajectory into 2028.
- contextIntel pushes ahead with Coral Rapids while cancelling other CPU linesc 0.50
Intel's forward roadmap centers on Coral Rapids, alongside notable cancellations of other CPU lines that reshape its datacenter portfolio.
- evidenceCascade Lake servers being retired at less than a fifth of the powerc 0.45
Millions of Intel Cascade Lake servers from the COVID-era cloud spend are being decommissioned and replaced by modern AMD and Intel parts that match performance at under 20% of the power draw.
- mechanismMilan fixed Rome's 4-core VM ceiling with a CCX ring busc 0.45
Rome VMs had to stay within 4 cores to avoid cross-die penalties. Milan in 2021 doubled CCX size to 8 cores via a ring bus while reusing Rome's I/O die.
- exampleGraviton4 scaled cores, memory and I/O by 50%c 0.45
Graviton4 moved to Neoverse V2 with 96 cores, 12 memory channels and tripled PCIe5 lanes, delivering 30-45% gains and adding dual-socket support.
- contextAmpere was the original merchant ARM server vendorc 0.45
Backed by Oracle, Ampere's 80-core Altra and 128-core Altra Max used Neoverse N1 cores on TSMC 7nm to challenge the x86 duopoly with cost-effective ARM parts.
- exampleTypical head-node CPU-to-GPU ratiosc 0.40
Current designs pair 1 Vera CPU with 2 Rubin GPUs, 1 Venice with 4 MI455X, 1 Graviton5 with 4 Trainium3, and 2 x86 CPUs with 8 TPUv7s per node.
- caveatSub-NUMA clustering trades capacity for latencyc 0.40
Large meshes introduce uneven latency to far-away memory controllers, so Intel offers Sub-NUMA Clustering that splits the mesh into quadrants. This reduces average latency but forces the OS to treat each quadrant as a smaller socket with its own L3 and memory pool.
- evidenceSapphire Rapids latency regressed despite the new architecturec 0.40
Crossing EMIB links pushed core-to-core latency from Skylake's 47ns to 59ns. Intel responded by enlarging private L2 cache to 2MB per core, resulting in more L2 than L3 on die.
- exampleSapphire Rapids' multi-year delay revealed execution problemsc 0.40
Originally slated for 2021, Sapphire Rapids slipped to early 2023 and went through stepping E5, likely due to difficulties getting the mesh to function correctly across EMIB.
- exampleZen 4c/5c and Siena show the modularity payoffc 0.40
Swapping in compact Zen 4c or 5c CCDs gave AMD Bergamo and 192-core Turin-Dense without changing the I/O die or socket, and Siena's 4-CCD 6-channel platform reused the same parts for smaller systems.
- exampleSpectre and Meltdown forced disabling SMTc 0.30
In 2018, the Spectre/Meltdown vulnerabilities exploited branch prediction across SMT threads, forcing cloud providers to disable SMT at a cost of up to 30% performance — a hit that would shape Intel's later design decisions.
- contextPretraining still needs CPUs for data plumbingc 0.30
In pretraining and fine-tuning, CPUs store, shard, and index data fed to GPU clusters and handle image and video decode for multimodal models, though decode is increasingly migrating into GPUs.
- caveatBit Matrix Multiplication is unlikely to find broad usec 0.30
Zen 6's new BMM instruction makes binary matrix ops cheap and could help Verilog simulation, but binary precision is too coarse for LLMs. Adoption will likely stay narrow.
Janitor
Non-content spans (acknowledgements, references, footnotes, headers, boilerplate) are dropped before the decomposition runs.
- total spans
- 125
- kept
- 124
- dropped
- 1
- content · 124
- boilerplate · 1