2026-08-12 00:52:16
Hello you fine Internet folks,
I did an audio interview with Ravi Subramanian from Synopsys where we talked about the physics that goes into chip design and EDA tools.
Hope y’all enjoy!
The transcript below has been edited for conciseness and readability.
George Cozma: At AMD’s Advancing AI was last week, you guys made some announcements there about 3DIC, and I would just like to learn a little bit more about that and hear a bit more about how you guys are thinking about thermals, but not just thermals, things like current such how to carry current from package to the compute dies?
Ravi Subramanian: So maybe a good starting point is the two main trends going on in computing from a pure silicon perspective, chips are getting bigger and bigger to the point where we have some reticle-limited chips. So we see that companies are using 2.5D and 3D types of integration techniques to have multi-die solutions to handle the ever-increasing compute complexity.
Thermals have always been a problem in chip design, so it’s not new. The first wave of big thermal innovation really had to do with the early eras of CPUs. As CPUs were getting faster and faster, how did they need to allow the device to operate in a package that’s sitting inside a laptop or sitting inside a desktop. Then came the mobile communications world brought some severe limits, because it was the first time the whole chip operation was determined by a battery-powered device.
George Cozma: Holds up phone.
Ravi Subramanian: Bingo. Exactly. All of a sudden, what was needed in doing low-power design, in terms of the EDA tools to do low-power design, and what was needed in terms of managing thermal effects not just from the compute die, but also from the power amplifier, which is sitting not so far away from the compute die in that chip, and then the battery, which is also creating heat. So all of a sudden, we now have a system where it’s not so simple to put a fan in a chassis and control things. The mobile industry really created the first step-function in how you need to think about thermal from a system perspective.
Then fast-forward, we go to automotive chips, and then we see regulations or rules really determining how the chip should operate because of the environment that it’s in.
George Cozma: So for automotive, is it less the actual thermals of the chip and more the environment of where the chip is being placed?
Ravi Subramanian: So the environment is a bigger factor, but if you look at a typical mobile SoC, it’s about 2 to 2.5 billion gates. A typical automotive ECU chip now is about 7 billion gates. So it is bigger, 3x bigger, and there are big power challenges. As a simple example, the way that’s really measured now is, “Oh, it translates into a much smaller range for the car,” because of how much energy is consumed by the compute die. So Mercedes, BMW, the US car makers, and others have talked about what’s the range restriction because of how much [power] is consumed by the chip.
Then we get to the data center, which is now another type of system with another set of constraints around the chip, the package, the rack, and then the whole building that everything is sitting in. So that’s the scale of how the thermal problem has changed, right? With different system complexity and the importance of not only the chip power dissipation and the way packaging is handled, but also the environment and what the environment brings in.
So now, if we look at 3D-IC and chip-in-package, and maybe more specifically about AMD…
George Cozma: If I remember correctly, AMD announced at their Advancing AI event which was last week and while there Synopsys announced the release of 3DIC which came with AMD as the co-partner for it.
Ravi Subramanian: Yes, yes. So AMD has been a leader since their presentation at ISSCC, I believe it was in 2019, when they disaggregated the die, and then they showed what the advantage is. That was the first seminal paper in this field that really set the stage for what’s the thinking behind going multi-die, and obviously evolving that to 2.5D and 3D.
Now, the minute you go multi-die, you get a number of advantages in terms of the total cost of the solution because of the fact that you’ve really thought about which logic node should which processing be in. But then what you’ve also done is you created a problem in that you have to move so much data around, and the energy consumed by moving the data.
So to get higher performance, Moore’s Law only gave you so much. You pick an advanced node, you can get the highest performance there, but then within a package now, you have multiple dies, each one having power dissipation during operation, which is very much related to the workload running on it.
So the first step is understanding, for a given architecture with a workload running on it, exactly how much power is consumed and where in the workload the power is actually consumed. So that’s one capability needed early on pre-silicon, that’s a verification type of problem.
And then the second piece is, once you actually have a structure or an architecture, what happens is, because of the package and because of the die, there can be warpage, and there are mechanical effects that used to be second and third-order effects and those effects now [are first-order effects], because the chips are getting bigger, you actually have mechanical effects creating a strain in the silicon, and that changes the electrical properties.
George Cozma: Okay. I know the whole joke is: if you want the best thermal dissipation, make it the size of a wafer; if you want the cheapest cost, make it the size of your fingernail.
Ravi Subramanian: Exactly! So what we have then is, in order to be able to analyze all the effects, sign-off used to be, “I’m going to make sure the functional performance of the chip is okay, I’m going to make sure the electrical performance is okay.” But now, you need to look at the mechanical-electrical pairing. And then similarly, you need to look at the thermal, and that is, as the chip is running, you have IR drop, you have electromigration, and you need to be able to analyze the performance of the chip with those effects incorporated.
And then finally, the last one is electromagnetic effects. Electromigration has to do with electrons; electromagnetics with Maxwell’s equations. So a good way to think about it is there’s Moore’s Law coming one way, and then there are all the notions of material stiff equations, partial differential equations for looking at materials and then Maxwell’s equations for electromagnetics, right? So, you know, even though people say, “Well, Moore’s Law was doubling every 18 months and you had to deal with that complexity,” 10 years ago, many of the effects we’re talking about were second and third-order effects, and didn’t really affect the performance.
George Cozma: And now it’s first-order.
Ravi Subramanian: Now it’s first-order, exactly. So the definition of what you do to sign off a chip before you give it to TSMC means you not only do the classical electrical sign-off, but you now have to co-sign-off on mechanical-electrical, thermal-electrical, electromigration, and electromagnetic. Because as you get to smaller and smaller wires, these actually behave as antennas as the signal speeds increase.
George Cozma: Yeah, effectively everything starts to become RF.
Ravi Subramanian: Exactly. So even within a package, if you have very high-speed connections, essentially you have radiating wires, right? And they could create interference because one wire is next to another where it could create a crosstalk effect, etc.
So that’s the scope of the nature of the problems AMD had to solve in successfully getting out some of the chips they announced at the Advancing AI event. We worked very closely together with them—R&D teams working with chip designers to understand what do they want to analyze, and how do we make that happen pre-silicon. Because as you may read, some of these chips now, they’re very complex multi-die or 3D-IC, and they have complex packaging, and sometimes they find a problem only after it comes out.
George Cozma: I mean, I think Jensen famously talked about the issues that they had with Blackwell.
Ravi Subramanian: Exactly. And so now people want to very much say, “How can I bring the view of those problems upfront? How can I look at those earlier and be more confident about handling them?”
George Cozma: So, I guess, going to that second part, the whole way of powering a chip via basically through TSVs and through all that, how has that changed the way that EDA tools work?
Ravi Subramanian: Sure. So a good way to think of that is maybe I’m going to start with chip and then system, right? On the chip side, today, if you look at a typical blade that’s going into many data centers, on the board you have the chips, but you also have a lot of power supplies, LDOs and other types of chips, that are managing how current is distributed across the chip.
For a very long time, the standard architecture has been 54V. As you try to push more and more signal across, the current in these wires starts getting higher and higher. So there’s a trend, an initiative that’s happening that I’ll talk about briefly, but right now, what that means is you have to carefully manage the currents on the board and have these LDOs managing how these currents are fed into these packages and then ultimately fed into the chips.
If you have I/O chips that are primarily SERDES or driving signals from chip package off the package, they have a driver strength that’s a certain amount. Whereas what you have in terms of a specific digital processing chip, you have power domains even within that chip where you’re powering up and powering down, and you want to carefully manage how current is fed to that chip and regulated to that chip. So you literally have a world of regulators regulating how currents are fed to chips.
Now, a big key trend that’s happened recently is a view to change the data center architecture and the rack standard to 800V. That’s going to dramatically reduce the magnitude of currents, steering towards 800V as a standard. Nvidia and others are driving towards that, and companies such as Bosch and others are supporting standardized components for that.
So the architecture of a board, you can expect, is going to start looking very different. Because now you have 800V, much lower current, and what you have with respect to trace lengths and how much energy is lost, because you have a much lower current, so the $I^2R$ combination of how much energy is being lost, it completely changes that equation in terms of how effective we can be in the management of the delivery of power to every packaged device.
George Cozma: Okay. And so on these 54V systems, was there a lot more thought having to be put into where certain like I/O wires. For example, where you have these big NVLink racks, you have these switches, and you need to connect the GPUs to the switches was there a lot more consideration in 54V systems and racks to how you would lay out those wires [compared to 800V systems]?
Ravi Subramanian: Absolutely. So the two main things there would be: first, you want to have high signal fidelity, so signal integrity, analysis of signal integrity with various routing topologies for those wires. That’s one piece, and that’s really artists at work using tools to see that.
And then the second one is the rate at which data rates are increasing. With each new generation of chip, you have to feed the beast, which means feeding the data from memory to the chip, and then from the chip into a switch to somewhere else, right? Those speeds are growing dramatically.
What those interfaces look like, and the introduction of new types of links between chip-to-chip, Nvidia introduced NVLink to be able to have heterogeneous systems connecting; there’s the UCIe standard, the UAL standard, all of these about chip-to-chip. But with UCIe, it’s not one standard, every company has their own view of “We want 8 lanes, we want 16 lanes, we want 12 lanes,” right?
George Cozma: If I had a nickel for every time I’ve heard that this morning, I would have two nickels—which isn’t a lot, but it’s odd that it’s happened twice! [Laughs]
Ravi Subramanian: [Laughs] Yeah! So that’s where understanding how you’re going to deal with much higher speeds comes in. Ultimately, you have to do signal integrity, but the architecture of how you’re going to connect, what standard are you going to use chip-to-chip, package-to-memory, package-to-package, package-to-switch, all come into play there.
George Cozma: Moving sort of more from just the EDA side to the IP side, right? We’ve been talking a lot about 200G networking or 200G SerDes. Moving forward, 400G is going to become the standard, although Nvidia has said that they have 400G. What is the difference between the industry standard 400G and what Nvidia is using, which I believe is called simultaneous bidirectional 200G? What exactly are the differences?
Ravi Subramanian: Sure. So today, if we look at 224G SerDes or 224G Ethernet, that’s a standard that’s well-established. That’s where a lot of IP is getting sold and put into systems and chips.
400G is not yet standardized. That standards committee is still going, and we’ve got many companies making contributions, trying to steer the standard one way or another. And there, you have what’s called PAM signaling, right? And there are a lot of different proposals with variations of that.
George Cozma: I believe the current proposals are for PAM6 or PAM8?
Ravi Subramanian: Exactly. So that’s going on there, and there’s a lot of jostling to get to a 400G standard, but there’s also a lot of jostling to get something out in 400G, though that has to essentially get adopted and become a de facto standard. In the meantime, what we’re seeing with 224G is 224G plus optical is going to extend the life of 224G. So that’s at least the standards landscape.
Now, with Nvidia and bidirectional 224, that’s really trying to pull forward a higher speed capability, not quite 400 as a standard, but very high performance, greater than 220. And by having bidirectional capability, you are able to have much greater throughput into and out of systems. So that’s the current standards landscape.
In terms of when will 400G be a standard? The way that’s going, it’s probably going to be at least a year and a half to two years towards that, just looking at how the history of standards has been. What we’re also seeing is hyperscalers are trying to create a pseudo-custom version of the standard. The standard is relatively mature, but there are specific changes that they want to make to really support the systems that they are building for their workloads and their data center architectures.
George Cozma: Well, thank you for sitting down with me.
Ravi Subramanian: Thank you!
2026-08-06 03:19:22
Hello you fine Internet folks,
NVIDIA has published a 45-page whitepaper explaining Vera, its first server CPU built around the company’s own Olympus core. On paper, Vera is a fascinating chip with an 88-core monolithic compute die, Olympus being a 10-wide Arm v9.2 core that has value prediction, a graph prefetcher, 2 MB of private L2 per core, 164 MB of shared last-level cache, and eight LPDDR5X memory interfaces promising 1.2 TB/s.
Unfortunately, NVIDIA also spends a good part of the paper trying to turn those interesting design choices into a morality play about x86. Traditional simultaneous multithreading is drawn as time-slicing, a configurable NUMA topology is presented as an unavoidable 32-node maze, four SPEC components become “agentic benchmarks,” undefined performance-counter ratios are promoted as causal proof, and an unlabeled pictogram becomes a 1.8x reinforcement-learning result.
The frustrating part is that Vera does not need this help, with early independent testing suggesting Olympus is genuinely formidable. The whitepaper’s strongest case is the hardware; its weakest case is the story wrapped around it, so let’s pull that story apart.
Before getting out the cheese grater, let’s talk about the good stuff. Olympus is a very wide out-of-order Arm core.
Its front end can decode ten instructions per cycle and handle up to two taken branches per cycle. NVIDIA describes a neural branch predictor, value prediction, memory renaming, a large instruction window, six 128-bit SVE pipes, four load pipes, two store pipes, a 96 KB L1 data cache, and roughly 10-cycle access to a 2 MB private L2. Eighty-eight of those cores sit behind a 3.4 TB/s coherency fabric and a distributed 164 MB system-level cache.
Looking closer at the core, the value prediction is one of the more unique additions that Olympus has. This has been a research area for a long time and what value prediction allows Olympus to do is if the core correctly predicts a result, dependent instructions can keep moving instead of piling up behind a long-latency operation. Researchers have discovered that Apple uses value prediction in their cores and AMD talked about how in Family 17h (Zen 1 and 2) they could predict the value of some floating point instructions. However, AMD’s Family 17h implementation was quite limited, while Olympus appears to have a broader value-prediction implementation closer to Apple’s.
However, the graph prefetcher is not unique to NVIDIA. Intel has a similar mechanism called Data-Dependent Prefetcher that has been in shipping silicon since at least 2022. Intel’s newest datacenter CPU, Granite Rapids, also has an Array of Pointers prefetcher which “treats the data prefetched for a constant stride load as a pointer and may issue prefetch requests to the memory addresses corresponding to the pointer’s value.” This is fundamentally the same producer-consumer idea that NVIDIA describes for its graph prefetcher. Intel’s implementation is fairly constrained, so NVIDIA’s implementation may be able to deal with more complex chains than Intel’s implementation. So while Vera’s Graph Prefetcher may be an implementation that can deal with more workloads, producer-consumer prefetching is not a new idea.
Nor is a “neural branch predictor” a new idea. Back in 2012, AMD implemented a perceptron branch predictor in the Piledriver microarchitecture and continued to use a perceptron-based branch predictor in Zen 1. However, starting with Zen 2, AMD used a perceptron BPU only for its initial direction prediction, with a TAGE predictor overriding it because it delivered a 30% reduction in mispredictions. With Zen 5, AMD has likely fully committed to TAGE predictors, if it had not already done so with Zen 3 or Zen 4.
Moving to the SoC side, with how beefy the Olympus core is, NVIDIA has given Vera an equally beefy memory subsystem. Vera pairs eight SOCAMM2 LPDDR5X modules with up to 1.5 TB of capacity and 1.2 TB/s of bandwidth. NVIDIA claims the populated memory subsystem only consumes approximately 50 watts. A conventional EPYC or Xeon platform can offer higher-capacity DIMMs which are easier to replace, but it pays for that flexibility in board area and power.
Most importantly, we have more than NVIDIA’s results to look at. In May, Michael Larabel at Phoronix ran an early Vera system against current Arm and x86 servers. Across the NVIDIA-permitted test set, Vera’s geomean was 10% above a 5 GHz EPYC 9575F, 1.55x a Xeon 6980P, and 1.63x Grace which makes Vera the most performant Arm server CPU we have seen in public testing. There are major caveats with the testing, such as NVIDIA choosing the permitted workload scope and not allowing frequency or power monitoring. The system that Phoronix tested was pre-production and the test window was one day which puts a fairly hard limit on what they could test irrespective of the limits NVIDIA placed. This means that broader coverage will have to wait until Vera can be found in the wild rather than just in NVIDIA’s labs.
Still, the result is strong enough that we can reject the explanation that the charts in NVIDIA’s whitepaper are all fantasy. Olympus appears to be a fast CPU core, so now we can ask whether the whitepaper proves what NVIDIA says it proves.
Here is the first major technical error in the document.
Figure 5 contrasts “Traditional SMT (x86)” with NVIDIA’s Spatial Multithreading. The x86 side depicts the branch predictor, decode, execution, load/store, and memory stages alternating between two threads. The caption says Vera avoids “opportunistic time-sharing” by partitioning resources across its two hardware threads.
NVIDIA’s diagram gives a misleading impression of how SMT is usually implemented, both on x86-64 and other ISAs. SMT implementations share various stages in the execution pipeline by either selecting a thread to service every cycle, or by behaving in a thread-agnostic manner. Fetch, decode, and allocate typically service threads on a per-cycle basis, while the execute and memory access stages are thread agnostic and can service micro-ops from both threads in the same cycle. Stages that threads arbitrate for do not leave resources unused when both threads can be fed, as NVIDIA’s diagram suggests. Static partitioning and per-cycle selection would provide the same average throughput to both threads in the absence of per-thread stalls. If there are stalls, per-cycle selection can give otherwise unused throughput to the un-stalled thread.
The same idea applies to thread agnostic stages like execute and cache access. Each thread is permitted to utilize as many execution units or cache ports as it can feed. In contrast, statically partitioning resources as NVIDIA suggests could lead to one thread being compute bound and unable to use half of the core’s execution resources because they’re reserved for the other thread.
Text in NVIDIA’s paper emphasizes “determinism, isolation, and quality of service” as advantages for NVIDIA’s Spatial Multithreading approach. Performance is conspicuously not called out. QoS may be a more important consideration than throughput for NVIDIA’s target market, and Spatial Multithreading may not be a bad design point. But NVIDIA’s figure makes it look like vertical space represents time, and gives a misleading impression that Spatial Multithreading is meant to give larger performance gains than traditional SMT.
By reducing resource interference between threads, Spatial Multithreading improves determinism, isolation, and quality of service compared to traditional SMT approaches. The result is a CPU architecture that can run large numbers of concurrent agent tasks while maintaining more consistent latency and throughput. - NVIDIA’s Vera whitepaper
Vera’s actual SMT performance is unknown of course, and a lot of variables go into SMT gains besides partitioning strategies at fetch, decode, execute, and memory access. Out-of-order resources like the reorder buffer, register files, and memory ordering queues can be duplicated, statically partitioned, watermarked, or competitively shared. Partitioned structures were split between the two logical processors in multi-threaded mode and recombined for one thread in single-thread mode, which was documented in 2002. Various SMT implementations use different strategies for each structure, and those choices can have significant implications for SMT gains.
Also something to note is that it apparently takes 10,000 cycles for an Olympus core to transition back to the single-thread mode once the sibling thread on that core is done. This means that software will have to be very aware about launching a second thread on an Olympus core due to the penalties incurred not only from the partitioning scheme but also from the delay of swapping back to a single thread.
It’ll be interesting to see what strategy Vera uses to partition its out-of-order resources, and how its SMT performance compares to that of other modern cores. NVIDIA’s whitepaper gives no information on that. What it does do is present a misleading diagram that suggests traditional SMT is prone to leaving resources unused, when it may actually be better at keeping the core fed than NVIDIA’s Spatial Multithreading.
NVIDIA next tells us that a large two-socket x86 system can expose “as many as 32 NUMA domains,” while Vera presents one per socket. The number is not invented. On a many-chiplet EPYC system, an administrator can expose cache-local regions as separate NUMA nodes. If you turn every locality knob toward maximum granularity, the node count gets large.
What NVIDIA leaves out is that this is configurable with AMD’s tuning guide listing NPS4, NPS2, NPS1, and even NPS0 modes. The optional “LLC as NUMA” setting can expose each last-level-cache domain separately. So “32 NUMA nodes” is not the inevitable user experience of a chiplet CPU, it is one end of a locality-control spectrum. NVIDIA presents an optional high-granularity configuration as though it were an unavoidable reality of x86 systems.
Vera’s one domain per socket simplifies scheduling and memory placement, while multiple domains let tuned software exploit physical locality. Vera chooses the simpler presentation, and NVIDIA is free to argue that this better matches its intended software stack. But an OS-visible NUMA node is an abstraction, not a wormhole. Vera still has 88 cores, distributed cache and home nodes, memory controllers around a large die, and a packet-switched coherency fabric. A flat software topology can make those distances around the large monolithic compute die more consistent, but it cannot make them nonexistent.
The paper’s core-to-core heatmaps would be a good place to quantify that advantage. Instead, NVIDIA provides colored squares with no core identities, no minimum/median/maximum table, no distribution, and no measurement procedure. “Up to 50% lower” captures NVIDIA’s best result, not Vera’s typical behavior.
One NUMA node per socket is genuinely simpler, but the whitepaper compares it against an optional 32-domain x86 configuration and presents that edge case as the baseline. The counterpoint here is that Intel has a Mesh NoC just like Vera has. The difference really between these two setups is that the clustered setup of EPYC has high latency between clusters but within a cluster the latency is low, whereas Vera and Xeon Mesh setup has uniformly average latency; the different configurations are just engineering tradeoffs.
The benchmark section is where the whitepaper, ostensibly about a CPU, starts wearing an AI conference badge it found on the floor.
NVIDIA selects four SPEC CPU 2026 integer workloads, CPython, GCC, LLVM, and Cppcheck, and calls them “agentic benchmarks.” SPEC itself describes them as a Python interpreter, two optimizing compilers, and a C/C++ static analyzer. Those are legitimate CPU programs. They stress large instruction footprints, branch-heavy code, allocation, and dependency chains. Agents can absolutely invoke programs like them.
But they are not agents: no model is serving tokens, no agent runtime is choosing tools. No sandbox is starting, blocking on I/O, retrieving context, evaluating an answer, or feeding observations back into a policy. These workloads may be useful proxies for the code-heavy portions of an agentic pipeline. Calling them “agentic benchmarks,” however, turns that partial overlap into a claim that they represent the complete end-to-end workload.
The paper does correctly label the SPEC results as estimates, because the Vera reference hardware was not generally available at the time of the run. Figure 15 shows a 1.7x to 1.8x advantage for the four selected components, normalized per physical core under a fully loaded two-socket system. Flip to the configuration pages and the full estimated SPECrate 2026 Integer Base totals are 925 for two Vera sockets and 898 for two EPYC 9755 sockets which is a 3.0% system-throughput advantage.
Both numbers can be true. Vera uses 176 physical cores across two sockets, while the EPYC system uses 256. Divide each score by physical-core count and Vera is about 50% faster per core across the full integer-rate suite with the selected tests reaching 70 to 80%.
There is another terminology collision. NVIDIA calls Figure 19 “single thread IPC” while describing a fully loaded system.
The published configuration runs 352 copies on 176 Vera cores and 512 copies on 256 EPYC cores, two copies per physical core. Maybe NVIDIA sampled one logical thread while its sibling was active, maybe it aggregated counters and divided, the paper does not say. The SPEC results are useful, and Vera’s per-core performance is genuinely strong. However, framing those tests as agentic workloads and emphasizing normalized figures makes the advantage appear broader than the disclosed results justify.
NVIDIA attributes Olympus’s reported IPC lead to four counter groups. Depending on the selected workload, Vera supposedly achieves up to 2.3x more branch predictions per cycle, 3.5x more taken branches per cycle, 2.4x more instruction-fetch operations per cycle, and 4.3x more backend operations per cycle.
That sounds technically specific, but it is impossible to audit without the PMU event names and definitions, raw counts, sampling intervals, clock frequencies, etc. Not to mention that an Arm instruction is not the same unit of work as an x86 instruction. An internal backend operation is even less portable: one microarchitecture may split an instruction into several micro-operations while another keeps it fused.
Cross-ISA IPC can still be informative when paired with retired-work counts, clock frequencies, and code analysis. It cannot stand alone as a performance metric. Two binaries can complete the same task in the same amount of time while reporting very different IPC. For one may simply retire more instructions that are doing less work per instruction, then you have to factor in clock frequency which could be wildly different. IPC describes the behavior of the core running a piece of code, not a universal measure of useful work.
Looking at the branch predictor results, more branch predictions per cycle could indicate a capable predictor or it could also mean the Arm binary contains more branches, the benchmark moves through code faster, or NVIDIA’s event counts speculative predictions that the EPYC event does not. Higher backend operations per cycle may correlate with performance while telling us little about which feature caused it. To isolate value prediction, graph prefetching, or the neural predictor, we need on/off experiments or at least event definitions and miss rate deltas. While the IPC advantage of Vera over Turin may be real, the charts in NVIDIA’s whitepaper don’t provide enough granularity of the results to show it.
The memory section contains NVIDIA’s strongest result and one of its weakest conclusions.
Against the dual-socket EPYC 9755 system in the paper, Vera reaches roughly 1.1 TB/s in NVIDIA’s loaded-latency plot while Turin levels off near 400 GB/s. Vera also shows 12.7 GB/s per core versus 3.1 GB/s per core for Turin. Those results do not line up with our testing of Turin CPUs.
In our testing of Turin, we were able to get approximately 570 GB/s out of Turin with the 12 channel DDR5-6400 memory subsystem. This is in direct contradiction to NVIDIA’s results which top out at ~400 GB/s of memory bandwidth. This does also throw the per-core memory bandwidth numbers into dispute with the per-core bandwidth increasing to ~4.5GB/s for the EPYC 9755.
Vera still comes out ahead, but our Turin result substantially changes the size of that advantage. Comparing Vera’s roughly 1.1 TB/s against the 570 GB/s we measured gives NVIDIA a 1.9× bandwidth lead rather than the nearly 3× lead shown in the whitepaper. Revising the EPYC 9755’s per-core result from 3.1 GB/s to approximately 4.5 GB/s similarly reduces Vera’s advantage from 4.1× to roughly 2.8×. And if we look at the SKU that AMD actually puts forward as the SKU for AI head nodes, the EPYC 9575F, then the per-core result becomes 12.7 GB/s vs the 9575F’s ~9 GB/s which is about 40% improvement for Vera. Those are still good numbers for Vera, but they tell a considerably less dramatic story.
Looking at the theoretical figures, AMD lists Turin’s limit at 614 GB/s from its 12 DDR5-6400 channels. Our 570 GB/s result reaches approximately 93% of that theoretical limit. Vera’s eight LPDDR5X-9600 interfaces provide 1.2 TB/s, while NVIDIA’s measured 1.1 TB/s reaches roughly 92% of that figure. In other words, both processors convert a remarkably similar percentage of their theoretical memory bandwidth into sustained bandwidth. Vera wins because it has approximately twice the peak bandwidth of one Turin socket and fewer cores competing for it, not because Turin is unusually poor at using its available memory bandwidth.
Despite that, the whitepaper repeatedly credits Vera’s monolithic compute die while contrasting it with “traditional chiplet-based CPUs.” A monolithic die may reduce fabric traversal and improve loaded latency, but our Turin result directly weakens that explanation for the bandwidth difference. A chiplet-based EPYC 9755 reaching approximately 93% of its theoretical limit is clearly not being held back by its chiplet topology in this case. The bulk of Vera’s bandwidth advantage comes from the memory interfaces attached to the processor.
The comparison also aged almost immediately. NVIDIA published its technical blog and whitepaper on July 21, while AMD launched 6th Gen EPYC two days later. The 96-core EPYC 9686F, which is much closer to Vera’s 88-core count, provides 16 memory channels supporting DDR5-8000 or MRDIMM-12800 for 1,024 or 1,638 GB/s per socket with the top MRDIMM speed giving Venice more theoretical bandwidth than Vera both in total memory bandwidth and per-core memory bandwidth depending on what SKU you look at.
What the new EPYC specifications and our Turin testing demonstrate is narrower than NVIDIA’s claim of “3× more memory bandwidth than the latest x86 CPU” depends on a Turin result that does not represent the bandwidth we could extract from the same processor generation. Against our result, Vera delivers approximately 1.9× the total bandwidth and 2.8× the bandwidth per core, that remains an impressive platform result but it is not evidence of a bandwidth advantage for monolithic Arm processors over chiplet-based x86 CPUs.
NVIDIA reports a 2.6x PageRank advantage over EPYC 9755 and shows Vera scaling almost linearly to 32 cores while EPYC flattens to just a 10X performance increase at 32 cores. NVIDIA attributes this improvement over Turin as down to the monolithic compute die with their high-bandwidth Scalable Coherent Fabric, the 1.2 TB/s of memory bandwidth that Vera has, and the graph prefetcher inside the Olympus core.
However, while the paper links GAP Benchmark Suite, it omits what variables NVIDIA used which are important factors on how this test runs. The scaling plot stops at 32 cores even though the machines have 88 and 128 cores per socket. While the 2.6x result is interesting, without the variables that NVIDIA used, the result is likely irreproducible.
For the ClickHouse testing, NVIDIA links directly to Phoronix’s result where Vera led the tested processors across three passes over a 100-million-row dataset. The whitepaper’s 1.2x chart is still selective due to not using the 9575F results, but an outside tester produced the underlying result with a recognizable workload.
Then we reach Figure 24, “Vera drives 1.8x for RL training,” with the figure being a row of little completed-task squares. There is no model, environment, CPU/GPU allocation, framework, batch size, power measurement, repetition count, or error bar. We do not even know whether the squares represent samples, steps, or some random layout of tiles at NVIDIA HQ.
This is not a bad benchmark, it simply is not a benchmark at all.
The surrounding text explains why a faster CPU could improve reinforcement-learning rollouts with faster environment steps and reward computation can feed accelerators more quickly however Figure 24 does not even pretend to measure it.
After 45 pages, my position on Vera is more positive than my position on the Vera whitepaper.
Olympus looks like a serious core with a 10-wide fixed-length decoder, large private caches, value prediction, aggressive branch handling, graph-aware prefetching, and a monolithic 88-core die all being choices pointing to a very high-performance CPU core. The memory setup is no slouch either with the LPDDR5X subsystem delivering up to 1.2 TB/s of memory bandwidth that memory bandwidth-hungry server workloads will love, and early independent benchmarks say the silicon can cash at least some of the checks that the whitepaper writes.
However, the paper’s competitive argument is much shakier, with it mischaracterizing x86 SMT, turning an optional NUMA configuration into a default burden, relabelling standard CPU tests as agentic workloads, hiding a 3% two-socket rate lead behind 1.8x per-core bars, comparing undefined cross-ISA counters, attributing a memory-interface win to monolithic virtue, and presenting an illustration as performance data.
None of that makes Vera slow, it simply makes NVIDIA’s proof smaller than NVIDIA Marketing’s prose.
The next round of Vera testing should be straightforward. Give independent reviewers unrestricted production hardware so that we can publish frequency, package power, and wall power figures along with testing Spatial Multithreading on/off results and of course running whatever benchmark/workload we wish on Vera. If Vera is as good as its architecture suggests, those tests will be much more persuasive than drawing x86 SMT as a tiny two-lane traffic light. The way things stand, NVIDIA’s marketing risks tarnishing Vera. NVIDIA has built enough CPU here, it can stop borrowing performance from the marketing pipeline.
2026-08-03 06:02:44
Arm’s 5-series cores are meant for tasks where performance barely matters, but power and area efficiency are top priorities. Because they’re less performance sensitive, 5-series cores tend to be superseded less often than their more sophisticated counterparts. Cortex A53 started out Arm’s 5-series lineup in 2012, and served across three generations of 7-series cores (A57, A72, and A73). Five years later, Arm refreshed the 5-series line with the Cortex A55. A55 then served as a little core partner to A75, A76, and A78.
Here, I’ll be looking at the A55 primarily in the Mediatek Genio 1200, using the Radxa NIO 12L single board computer. The Mediatek Genio 1200 has four A55 cores and four A78 cores in what Arm calls the DynamIQ Shared Unit (DSU). The DSU is a cluster with 2 MB of L3 cache, and can host different core types without creating cluster boundaries between them. The Genio 1200 runs its A55 cores at 2 GHz, and gives each A55 core 128 KB of L2 cache. For memory, the SBC comes with 8 GB of LPDDR4X running at 4266 MT/s.
I’ve somehow wound up with several A55 implementations. My previous phone, a Pixel 3A, has a Snapdragon 670 with six A55 cores. Zarif98 on Reddit kindly donated a OnePlus 7 Pro, and that comes with four A55 cores in the Snapdragon 855. Microbenchmarking on Linux is easiest because Android tends to be a noisy environment, but I’ll occasionally include data from those devices to show whether an observation comes down to implementation differences.
Arm’s Cortex A55 retains A53’s 2-wide, in-order execution formula and makes targeted improvements. Like its predecessor, A55 uses an eight stage integer pipeline and a nine stage floating point one. All instructions pass through every pipeline stage for their category, which ensures ordering and avoids writeback conflicts from different latency instructions finishing at the same time. That makes the pipeline more fixed length than in a typical out-of-order core.
A55’s improvements are concentrated in the frontend and memory subsystem, though parts of the core proper received tweaks as well.
Arm’s slides talk about a new “neural network based” predictor. This likely means A55 uses a perceptron predictor, which can learn longer patterns while requiring less growth in predictor storage than the A53’s GShare predictor.
Testing with branches that are randomly taken/not-taken in random patterns of increasing lengths shows A55 doing better than its predecessor. However, it’s still far off contemporary high performance cores.
A55 and A53’s performance counters make a like-for-like comparison difficult because A53’s performance monitoring unit (PMU) can’t restrict counting to retired branches. A rough comparison with retired branches on A55 and executed branches on A53 suggests some level of improvement across Geekbench 6’s workloads, though it’s hard to tell how significant that difference is.
Curiously, A55 doesn’t universally improve and suffers more mispredictions in Clang. Photo Library is another case where A55’s newer predictor doesn’t show a clear benefit. Tuning predictors is difficult especially with bare bones power and area budgets, so some applications are bound to fall through the cracks. Perceptron predictors also struggle to predict linearly inseparable patterns. Schemes that index into a history table using the global history don’t have that problem.
Branch prediction speed can affect frontend throughput, and A55 improves that with a new 48 entry micro-BTB. This micro-BTB can handle taken branches without introducing pipeline bubbles, and should speed up performance in tight loops with taken branches. A53 had a 16 byte branch target instruction cache (BTIC) that could accelerate instruction delivery, but only for a single branch target. A 48 entry micro-BTB is still small, but should be relevant for more cases. Branch spacing doesn’t affect micro-BTB capacity.
A55 continues to lack a large BTB with hundreds or thousands of entries like on high performance cores. The core’s decoders calculate target addresses for taken branches not captured by the micro-BTB, giving taken branches 3 cycle latency like on A53. It’s not bad for code that fits in the instruction cache, especially considering that in-order cores are likely to lose lots of throughput from data-side memory latency. However, it does mean the branch predictor can’t run ahead of instruction fetch and drive long distance prefetch.
If each branch jumps to a different 64B cache line, taken branch latency heads for the skies once the test exceeds instruction cache capacity. A55 curiously regresses compared to A53, even though A55’s private L2 cache offers better data-side latency than A53’s larger shared one. With 1536 branches (96 KB), A55 averages 28-29 cycles per taken branch compared to 15-16 cycles on A53.
Cortex A55 comes with 16 KB, 32 KB, and 64 KB options. Compared to A53, Arm dropped the 8 KB option and increased associativity from 2-way to 4-way. Increased associativity reduces the likelihood of conflict misses, where the cache may have enough capacity to contain frequently accessed lines, but ends up getting thrashed because of restrictions on where those lines can be stored.
A55’s instruction cache stores instructions in a pre-decoded format to reduce decode costs, rather than storing raw instruction bytes. 4 byte aarch32 or aarch64 instructions are expanded to a 40-bit intermediate format, while 16-bit T32 (Thumb) instructions take 20 bits. Instruction cache tags indicate the instruction type (aarch32, aarch64, or T32). The predecode approach is common to many of Arm’s cores, and takes a midpoint between a conventional instruction cache and a micro-op cache that completely skips decoding.
Instruction cache tags and data are parity protected, with one parity bit per 32-bit tag entry and one bit per 20 raw data bits. That means A55 can detect a single bit error within each predecoded T32 instruction, or up to two flipped bits in a 4B instruction provided the two bits don’t occur in the same 20-bit half.
Mediatek and Qualcomm chose the 32 KB option for their chips. Testing with a loop of NOPs gives no surprises. All A55 implementations can sustain two instructions per cycle from the instruction cache. Spilling out of L1I drops throughput to well below 1 IPC. Arm’s Technical Reference Manual states that the instruction cache has a “128-bit read interface to the L2 memory system”. However, A55 can’t seem to fully utilize that interface in a sustained fashion in this simple test. A55 also displays lower code throughput than A53 out of L2, mirroring the situation with branches. Past L2, A55 regains the lead because it benefits from a L3 cache.
Geekbench 6 shows A55 and A53’s frontends facing similar challenges. A55/A53 are both organized into an Instruction Fetch Unit (IFU) and Data Processing Unit (DPU). The DPU decodes and executes instructions, and contains an instruction queue. Arm treats cycles with an empty DPU instruction queue as frontend bound. I’m applying the same definition to A53, even though Arm didn’t use top-down metric definitions in A53’s time. Workloads with larger instruction footprints, like Clang, cause the DPU IQ to stay empty for a significant fraction of core cycles. Further breaking down the cause shows most of this happens when there’s an instruction cache miss pending. A55’s higher L1I associativity may help a little bit, but not having a large decoupled BTB continues to hurt.
A55 inherits most of A53’s backend, meaning it features robust dual-issue capability. Most instructions can issue from either position, and the core has two copies of execution units for the vast majority of common instructions. Interlocks take care of instruction dependencies, meaning that adjacent dependent instructions don’t cause issue-stage hiccups. Forwarding paths help deliver low execution latency even though the pipeline is fixed length.
Earlier, I applied the same methodology I used to measure structure sizes on out-of-order CPUs to A53. Those tests showed A53, and A55, could do limited reordering around a cache miss. I assumed that was because the DPU had some kind of buffer for tracking in-flight operations in the backend. After reading the optimization guides for both cores, I think a better explanation is that independent instructions issued after a load can sit in pipeline stages while waiting for the load to retire. If so, observed reordering distance is simply the number of pipeline stages that younger instructions can occupy (or the first pipeline hazard that forces an issue-stage stall). There’s no pseudo-ROB.
A55’s execution pipe layout is basically unchanged. However, floating point fused multiply-add (FMA) instructions see latency reduced to 4 cycles, down from eight in A53. FP adds and multiplies have 4 cycle latency, so A53’s 8 cycle FMA latency meant there was no latency advantage to using FMA. A55 fixes that situation. Elsewhere, A55 receives changes to support the Armv8.2 standard, as well as LDAPR from Armv8.3 and dot product instructions from Armv8.4. Arm’s reluctance to pursue larger execution-side changes is understandable. Performance often comes down to feeding execution units rather than having more of them, and in-order cores face severe challenges in keeping their execution units fed.
Many of Geekbench 6’s workloads are heavy on FP/SIMD instructions. A55 cuts down floating-point/vector related interlocks in Asset Compression, HDR, Photo Filter, and Ray Tracer.
Structure in Motion doesn’t benefit, though that could be a case of A55’s improved memory subsystem shifting bottlenecks away from load latency and towards the execution units. Integer workloads like File Compression and Navigation see little change in interlock delays between the two cores.
At a higher level, A55 and A53 both have weak vector execution units. Even though there are two floating-point/vector pipelines, each pipeline only has 64-bit vector execution width. FP/vector operations also have longer latency than scalar integer ones. Out-of-order cores can easily absorb 4 cycle latency, but that’s not always the case for in-order cores.
Arm delivered more significant improvements to the memory subsystem, which starts with the load/store unit on the data side. A55 can now sustain both a load and a store every cycle, compared to A53’s single memory operation per cycle. Load and store bandwidth individually remain unchanged at 8B/cycle for loads and 16B/cycle for stores. However, the improved dual issue capability can be particularly beneficial for memory copy routines that don’t put loads and stores too far apart.
A55 improves L1D hit latency too thanks to an adjusted load pipeline and a forwarding path from the beginning of the load writeback stage to the AGU’s base operand input. This forwarding path lets A55 achieve two cycle latency for dependent loads. The load writeback stage can also forward to the ALUs, giving 2 cycle load-to-use latency for ALU operations as well.
Curiously, it seems like A55 can only use one of those fast forwarding paths at once. Placing an ALU instruction between two dependent loads increases pointer chasing latency to three cycles, but only if that ALU instruction consumes the prior load’s result. An independent ALU instruction doesn’t increase latency.
Indexed addressing increases load latency to three cycles, which is still one cycle faster than doing the same on A53. A53’s optimization guide interestingly mentions two cycle pointer latency as well, though I have not been able to achieve that even with unrolled back-to-back dependent loads. I’m not sure how A53’s pipeline layout would allow for two cycle load latency in any case. A load would need three clock transitions to get from the AGU to the writeback stage, and A53’s optimization guide says forwarding happens from the beginning of the writeback stage.
Store forwarding latency increases on A55 compared to A53, though cycle count latencies remain low compared to many out-of-order cores. Perhaps supporting dual issue made A55’s load/store unit more complex, and introduced extra penalties. To summarize the notable cases:
A55 continues to prefer 8B alignment for loads and 16B alignment for stores. Both cores avoid harsh penalties when a load only partially overlaps a prior store, likely thanks to their simple in-order designs and short pipeline depth. A55’s latency regression versus A53 probably has little effect because an in-order core can’t have hundreds of instructions in flight, making store forwarding cases less likely.
Cache misses are handled similarly on A53 and A55. Neither core supports hit-under miss, so any L1D miss will delay subsequent memory accesses even if they hit L1D. A55/A53 can achieve some degree of memory level parallelism by letting loads after a cache miss access L1D and generate fill requests. A55/A53 can generate up to three outstanding cache refill requests this way. Arm’s optimization guide says that only works if the three loads are within four pipeline stages of each other.
In typical scenarios, every memory access uses a virtual address that has to be translated on-the-fly to a physical address that corresponds to locations in DRAM or IO. A55 uses two translation lookaside buffer (TLB) levels like many other cores. TLBs cache frequently used translations, helping reduce address translation penalties. A55’s first level TLB has 16 entries, up from 10 in A53. Both figures are small compared to even older out-of-order cores. AMD’s Athlon 64 from long ago had 32 entry L1 TLBs, for reference. L2 TLB size is more reasonable at 1024 entries, matching Intel’s Haswell, and has twice the capacity of A53’s 512 entry L2 TLB.
Getting a translation out of the L2 TLB adds three cycles of latency, compared to just two on the A53. It’s a good tradeoff because the larger first level TLB should reduce reliance on the L2 TLB, and an in-order core will struggle to deal with page walk latency. 3 cycles of L2 TLB latency is good compared to many out-of-order cores, including low clocked ones. A78 for example takes 5 extra cycles of latency on a L2 TLB hit.
To reduce page walk cost, Arm uses both an Intermediate Physical Address (IPA) cache and a walk cache. The IPA cache holds translations between a VM’s physical address and host physical address, helping reduce virtualization overhead. The walk cache holds the second-to-last translation table entry, letting page walks initialize with one step to go. Arm didn’t change IPA cache or walk cache size on A55, likely because 64 entries already go a long way. One translation table holds 512 entries, so 64 walk cache entries would already accelerate page walks for up to 128 MB of address space using 4 KB pages.
L2 TLB structures are parity protected, with parity errors causing an invalidation and a page walk.
A55’s L1D cache has 16 KB, 32 KB, and 64 KB options, and is 4-way set associative like the L1I cache. L1D data and tags are ECC protected, with single-bit errors handled by evicting the line, correcting the error as part of the eviction process, and reloading the line from L2. Compared to A53, increased L1D associativity should provide a slight hitrate improvement. Arm also changed L1D addressing from physically indexed, physically tagged (PIPT) in A53 to a virtually indexed, physically tagged (VIPT) scheme in A55. Note that the 32 KB and 64 KB options are still VIPT. A common misconception is that VIPT requires the virtual address bits that give the offset into a page to exactly match the cache index bits.
Cache misses are the most common long latency instruction in most workloads, and present a severe problem for in-order cores. A55 and A53 can keep no more than about eight instructions in flight after a cache miss, which is four cycles of core throughput. Any L1D miss will incur a longer penalty than four cycles, so reducing cache miss latency is crucial to making the situation less bad. A55 also needs to support Arm’s new DynamIQ Shared Unit (DSU), which has different cache setup expectations.
As a result, A55 gets an optional core-private L2 cache, which aligns with the DSU’s expectations and allows for lower L2 hit latency. A55’s L2 mostly acts as a victim cache, though software prefetch instructions can bring data into L2 without first getting that data evicted from L1. Arm provides 64 KB, 128 KB, and 256 KB L2 capacity options, all of which use 64B cache lines and are 4-way set associative. The L2 has a 128-bit write path and 64-bit read path to the core, matching load/store bandwidth. An extremely area constrained design can have no L2 cache, though doing so would likely be catastrophic for performance. 128 KB seems to be a popular choice, and is what Mediatek picked in the Genio 1200.
L2 latency comes in at a brisk 9 cycles, or just over half the 17 cycle latency seen with A53’s 256 KB shared L2 A small 32 KB L1D will see a lot of misses, and a 128 KB L2 could service a significant portion of those misses without the latency of a shared cache. The downside is that four 128 KB caches require twice the storage capacity of a single shared 256 KB cache.
With the new core-private L2, the DSU’s L3 takes over the role of a large, last level shared cache. Arm’s first DSU, the one introduced with A55 and A75, supports up to 4 MB of L3. L3 capacity options larger than 1 MB use two slices, meaning the Genio 1200’s 2 MB L3 is built from two slices. The power of two size also means it’s 16-way set associative. Arm allows 1.5 MB and 3 MB L3 options by disabling ways to create a 12-way cache.
L3 latency varies depending on implementation options, with Arm giving a 21 cycle latency example. On the Genio 1200, L3 latency comes in at around 35 cycles. It’s much higher latency than the 256 KB L2 on the Amlogic S922X’s A53 cluster, but that’s understandable considering the Genio 1200 has eight cores in the cluster instead of two. On the capacity front, A53 did support L2 capacities up to 2 MB, but I doubt that was a common configuration.
Bandwidth tends to be de-prioritized for density optimized designs, and Arm’s 5-series cores sit far on the density optimized side of the spectrum. Still, A55 manages significant bandwidth improvements at larger test sizes. A53’s bandwidth crashes into the ground once test sizes spill out of cache. A55’s results vary across implementations, but all display milder bandwidth dips even after test sizes exceed L3 capacity. Mediatek’s Genio 1200 barely shows any single thread bandwidth decrease even at very large test sizes.
Part of this could come from prefetching. A55/A53 rely heavily on prefetch to achieve memory level parallelism, because both cores can only sustain three L1D demand misses. Arm’s documentation indicates that A55’s L1D prefetcher can make up to seven outstanding prefetch requests, with a default of five. Curiously, A53 can be configured to make up to eight outstanding prefetches. New in A55 though is a L3 prefetcher that can be configured to prefetch 32 lines ahead, with eight lines as the default. Prefetch into L3 can be more aggressive, because the L3 cache’s larger size reduces the chance of useless prefetches contending with frequently used lines. This more aggressive prefetch likely explains A55’s higher memory bandwidth.
Both A55 and A53 use a streaming mode to help store bandwidth, with a switch at each cache level triggered by reaching a threshold of consecutive “streaming” cache lines. Mediatek’s A55 cores maintain very high store bandwidth even across large test sizes, mirroring behavior with loads. Other A55 implementations show similar behavior to A53.
Performance counters show A55’s memory subsystem delivering impressive improvements. Memory-related stalls were severe on A53, especially on Geekbench 6’s Object Remover, Background Blur, and Navigation workloads. A55 nearly cuts those figures in half.
The A55 cores on Mediatek’s Genio 1200 deliver performance improvements well above Arm’s reference figures across many of Geekbench 6’s workloads. Background Blur enjoys an astounding 210% score uplift, thanks to being less memory bound and suffering fewer FP/SIMD interlocks. Gains are also massive in Object Detection and Photo Filter.
I suspect uplifts will be closer to Arm’s estimates in a more like-for-like comparison. Mediatek’s Genio 1200 does give its A55 cores a much stronger memory subsystem than the Amlogic S922X does for its A53 cores. But better caching would be expected from most A55 implementations, thanks to the new DSU setup’s expectations.
A55’s improvements are less impressive in SPEC CPU2026’s integer suite, though they still go beyond typical generation-on-generation gains from high performance cores. A 35.35% geomean uplift over A53 is nothing to sneeze at, though of course A53 represents a very low performance baseline.
SPEC CPU2026’s floating point suite shows A55 pulling way ahead again, with a 50.45% geometric mean uplift over A53. I didn’t collect performance monitoring data for SPEC CPU2026 because each run takes such a long time on these low clocked, in-order cores. However, one thought is that SPEC CPU2026’s larger instruction footprints in its integer suite punish A55. Instruction cache misses tend to be less common in SPEC’s floating point workloads, which shifts emphasis back to A55’s superior memory subsystem.
A55’s core architecture isn’t exciting, because it leaves most of A53’s framework unchanged. Arm made a sensible choice, because in-order cores do poorly at maintaining throughput around anything that takes more than a few cycles. Better branch prediction and caching down common causes of delay-causing events, and contribute directly to performance improvements.
Even with better branch prediction and caching, A55’s IPC figures show the difficulty of feeding a 2-wide pipeline with in-order execution. Certainly there’s more room to push on the same fronts that Arm did with A55. Larger and faster caches could further mitigate backend memory stalls. More sophisticated branch prediction would help mitigate instruction side stalls.
However, there’s likely a point where out-of-order execution starts to make sense even for sub-1W power targets. Larger caches take area and power. Branch prediction involves caching branch targets and outcomes, and runs into diminishing returns for increasing storage investment. Arm’s own A73 is a small 2-wide out-of-order design, and has no trouble pulling well ahead of A55 even without a clear memory subsystem advantage. For reference, A73 uses a pre-DSU system level setup, and has a 1 MB L2 cache in the Amlogic S922X.
Arm ultimately decided in-order execution still has legs. They followed up A55 with the A510, which implements 3-wide in-order execution and uses shared components in dual core clusters to save area. I can’t imagine feeding a 3-wide core with in-order execution is any easier. But I guess for now, out-of-order execution is still too complex to pull off at sub-1W targets, even with modern process nodes.
2026-07-31 09:39:23
Hello you fine Internet folks,
Today we are talking about one of the stranger x86 stories I have seen in a while. According to both SemiAccurate and Reuters, Intel is giving a new startup called RosaicLabs access to the register transfer language code for its Atom CPU technology. That is already a massive departure for a company that has historically guarded both its x86 license and CPU-core RTL very closely.
Two days before Reuters published its story, however, Phoronix spotted a very weird post on the Linux kernel mailing list. The post describes working x86 implementations with 16 and 32 Advanced Matrix Extension (AMX) tile registers, compared with eight tiles on Intel’s shipping AMX implementations. It was written by the same developer who said in October 2025 that a “corporate entity other than Intel/AMD” were already using its own x86 opcodes, CPUID range, and model-specific register range.
There are no public sources connecting Rosaic to that mystery x86 implementation. I want to be very clear about that from the start.
Still, an Atom-derived x86 core attached to a much larger matrix engine would make an awful lot of architectural sense for a startup staffed by Rivos veterans. Intel’s involvement would also answer the rather important question of how a new American company could legally build a custom x86 processor in the first place. So let’s dig into what we know, what the Linux posts actually say, and where the speculation begins.
Both Reuters and SemiAccurate report that Intel has started providing Atom technology to RosaicLabs Inc., a startup incorporated in Delaware in May 2026. Rosaic is led by Amarjit Gill, a longtime semiconductor executive and investor who worked with current Intel CEO Lip-Bu Tan to assemble the founding team at Rivos. Reuters also says Rosaic’s team includes other Rivos veterans, including former Rivos finance chief Amit Parikh.
The reported arrangement goes well beyond an architectural license, with Intel reportedly supplying register transfer language, or RTL, code. RTL describes the logic and data movement inside a digital design in a form that can be simulated, verified, modified, and eventually synthesized into physical gates for a selected manufacturing process. In less technical terms, Intel is not merely giving Rosaic permission to execute x86 instructions. It is reportedly handing over the source-level description of an Atom CPU design.
That does not necessarily mean Rosaic gets every piece of an Intel processor. Reuters does not identify the Atom generation, the included cache and interconnect logic, the manufacturing process, or the contractual limits on modifying and shipping the design. Access to CPU-core RTL is also not the same as receiving a complete, tapeout-ready system-on-chip.
Even with those caveats, this is highly unusual. Intel has licensed x86 technology in limited circumstances before, but it has not operated like Arm, SiFive, or another merchant CPU-IP supplier. If this arrangement becomes a repeatable business model instead of a one-off deal, it would represent a fundamental change in how Intel monetizes its CPU designs. Intel has previously signaled that it wants to make more IP available through Intel Foundry, so this deal may be part of that broader strategy.
Rosaic itself remains almost completely in stealth. Reuters found no company website or LinkedIn page with any useful information, while an amended July filing reportedly gives the company room to raise a $10 million seed round. That is enough money to assemble a small early-stage team, but it is nowhere near enough to take a large, leading-edge processor from a blank sheet of paper to mass production.
Receiving an existing CPU core changes that equation by letting Rosaic spend its engineering budget on the parts that differentiate the product instead of rebuilding an x86 frontend, execution engine, memory-ordering machinery, privilege model, and software ecosystem from scratch.
The first useful clue appeared in October 2025. Longtime x86 researcher Christian Ludloff posted a message to the Linux kernel and GNU Binutils mailing lists saying he had been asked to relay information about opcode allocations already “in active use by a corporate entity other than Intel/AMD.”
The original Binutils post lists several reclaimed or otherwise unused opcode regions, an E000_xxxx CPUID range, and a matching E000_xxxx MSR range. Ludloff said these ranges were already in active use, implying that this was more than an academic proposal.
As Phoronix noted at the time, that raises two questions: Who was building the processor, and what legal right did it have to implement x86? Existing x86 companies such as VIA, Zhaoxin, or Hygon would be obvious candidates, but they already have public toolchain and kernel histories. Routing the notice through an independent expert while refusing to name the company would be an odd way for one of them to reserve instruction space.
Then on July 27, 2026 Ludloff posted again, this time the subject was an x86 implementation with more than the normal eight AMX tiles. The full mailing-list post documents implementations with 16 and 32 tile registers and explains how they can be fitted into mechanisms that already exist in the AMX architecture.
Intel’s current AMX programming model provides eight tile registers. Each register can hold up to 16 rows of 64 bytes, giving each tile 1 KB of capacity and the complete tile file 8 KB of architectural state. A separate Tile Matrix Multiply unit operates on those registers. Intel’s own AMX documentation describes this as palette 1.
Interestingly enough, the 64-byte TILECFG structure used to configure AMX already contains row and column fields for 16 tiles. Shipping implementations simply use the first eight. Ludloff says a 16-tile implementation can therefore use existing VEX instruction encodings, which can address TMM0 through TMM15, along with fields already present in the configuration structure.
Moving AMX to 32 tiles requires more work. The proposed AMX implementation uses EVEX encodings to address TMM0 through TMM31 and doubles TILECFG from 64 to 128 bytes so it can describe the extra registers. The post says existing CPUID mechanisms can already report more than eight tiles, the total tile-state size, and the width of TILECFG, so the implementation does not require a new feature flag.
In other words, the original AMX framework was more extensible than Intel’s shipping hardware. However, compatibility could become a problem because existing software often assumes that TILECFG is always 64 bytes and that the tile-data state is always 8 KB. Software with those assumptions would have to run in a degraded eight-tile mode. The post even proposes MSR controls to disable the additional registers when necessary.
For a 16-tile implementation, the tile file grows to as much as 16 KB per hardware thread. At 32 tiles, it reaches 32 KB, which is a large chunk of state for an operating system or hypervisor to save and restore, especially on a many-core processor. This points to the device containing the 16- or 32-tile AMX unit being much more focused on matrix multiplication than on general-purpose server tasks.
The mailing-list post refers to both AMX and its cross-vendor successor, AI Compute Extensions (ACE). AMD and Intel developed ACE through the x86 Ecosystem Advisory Group as a common matrix-acceleration target for future x86 CPUs.
The ACE whitepaper defines eight 512-bit-by-16-row tile registers plus a block-scale register. It also places ACE inside the existing AMX palette framework, allowing operating systems to reuse much of AMX’s state-management machinery. The mysterious design’s 16- and 32-tile modes therefore go beyond the baseline state defined by both AMX and ACE. They look like a vendor-specific scaling of the same programming model rather than merely a straightforward implementation of the new common standard.
Again, this points to a matrix-accelerator chip rather than a conventional general-purpose CPU.
Now we get to the fun part, because the circumstantial evidence is fairly compelling with both Reuters and SemiAccurate saying a previously unknown startup staffed by Rivos veterans is receiving Atom RTL from Intel. Days earlier, an unnamed corporate entity asked Ludloff to disclose details of a heavily extended x86 matrix implementation. Ludloff had previously relayed private x86 instruction, CPUID, and MSR allocations on behalf of an unknown corporation. An appropriately broad Intel agreement could explain how such a company gained both a functional x86 core and the legal right to ship it, although neither report discloses the actual licensing terms.
Atom also makes sense as a starting point. Intel’s Atom-derived cores are designed for better area and power efficiency than its large performance cores. Rosaic could use a relatively compact x86 core as the host and control processor, then spend most of its transistor and power budget on matrix execution, cache, memory bandwidth, and interconnect.
That would look less like a conventional Xeon competitor and more like an AI accelerator that happens to execute x86 natively. Instead of pairing a RISC-V control core with a giant data-parallel engine, as many accelerator startups do, Rosaic could pair an Atom-derived core with a giant AMX- or ACE-like engine. Ordinary x86 software would handle orchestration, while long-running kernels would use the expanded tile-register file.
However, there is a fly in the ointment. Rosaic was only incorporated in May 2026. The October 2025 opcode notice predates the company’s first filing by about seven months. That does not mean Rosaic could not have been working on a chip before those filings. However, in the July mailing-list post, Ludloff says that an implementation supporting 16 and 32 tiles “has been running at large scale for some time.” It is very unlikely that a startup receiving Atom RTL could have quietly designed, taped out, and deployed a new x86 core with an AMX implementation that is not directly compatible with current AMX code at “large scale” in that time.
There are ways to make the timeline fit, but each requires another assumption on top of the assumption that Rosaic is the company Ludloff is working with. The design could have started as an internal Intel project that was later handed off to Rosaic. “Running at large scale” could refer to simulation, emulation, or FPGA instances rather than production silicon. The unnamed corporation in 2025 could also have been a predecessor entity whose work is now being commercialized through Rosaic.
It is also entirely possible that the timing is a coincidence. The mailing-list posts never name Rosaic, and Reuters never mentions AMX, ACE, custom opcodes, or a matrix accelerator. Access to Atom RTL does not imply that Intel included AMX, because Intel’s shipping Atom-derived cores do not provide the per-core AMX engines found in its large Xeon cores. Rosaic would still need to design or obtain the accelerator, integrate it with the core, and make the enlarged architectural state work across the operating system and hypervisor.
The hard evidence leaves us with two separate stories. Intel is taking the extraordinary step of providing Atom RTL to Rosaic, while an unnamed company has an x86 AMX implementation with 16 and 32 tile registers that Ludloff says has been running at large scale. Connecting those stories produces a very compelling architecture, but it does not yet produce a confirmed product.
If “running at large scale” means deployed production silicon, Rosaic is difficult to reconcile with the timeline unless it inherited a mature design. If the phrase instead covers simulation, emulation, FPGA prototypes, or work performed by a predecessor organization, Rosaic becomes much more plausible. That single ambiguity is doing an awful lot of work.
Rosaic is not the only possible name. Zhaoxin is the obvious conventional candidate because it has a legal path to x86 through VIA and is actively building modern x86 processors, while Hygon has an AMD-derived x86 lineage, a growing data-center CPU business, and public ambitions around AI acceleration. A large tile file would fit either company’s server ambitions, but both already submit compiler and kernel support under their own names, making the anonymous relay difficult to explain; neither company’s public software support exposes anything resembling 16- or 32-tile AMX.
A hyperscaler’s internal chip program may fit the “large scale” wording better: Google, Meta, Microsoft, and Amazon have the money, deployment volume, and custom-silicon teams needed to build such a device. Google is tempting because Ludloff previously worked there, while Meta has the Rivos connection, but that is extremely thin circumstantial evidence and none has publicly disclosed the x86 license or Intel-core agreement such a design would appear to require. The fun wildcard is the old Centaur/VIA lineage, because Centaur already built an x86 server SoC with a large integrated NCORE AI coprocessor. Most of that team moved to Intel years ago, and there is no evidence NCORE evolved into this mysterious AMX implementation, but it shows that pairing a third-party x86 core with a giant matrix engine is not an entirely new idea.
My best guess is that the mystery design belongs to either an Intel-backed spinout or a confidential custom-silicon customer, with Rosaic remaining the most interesting named candidate. Zhaoxin and Hygon have clearer routes to implementing AMX, while a hyperscaler better fits the claim of large-scale operation. Every option solves one part of the puzzle and creates another.
For now, the most important detail is not whether the mystery company is Rosaic. It is that Intel appears willing to let outside companies build around its CPU RTL at the same time that someone outside Intel and AMD is publicly reserving substantial new x86 architectural space for matrix compute. Even if the two stories are unrelated, both point toward a much stranger and more open x86 ecosystem than the one we are used to.
2026-07-28 03:43:53
Hello you fine Internet folks,
Today we are over AMD's Advancing AI 2026 event to chit chat on their newly revealed CDNA5 architecture with Alan Smith! AMD's Corporate Fellow and Chief Architect for Datacenter GPUs.
Hope y'all enjoy!
The transcript below has been edited for conciseness, readability, and clarifications.
George: So, what do you do here at AMD?
Alan: Yeah. So, I’m a Corporate Fellow of Graphics Architecture at AMD and responsible for Instinct GPU architecture, which we launched today in MI455 and Helios.
George: Starting off, all the way down into the grunt of the engine of MI455. Prior CDNA architectures were based on the GCN architecture, which dates all the way back to Tahiti from I believe 2012, if memory serves, but with CDNA 5, you have now rebased to RDNA. What were some of the considerations for that rebasing of the architecture?
Alan: Yeah, that's a great question. I think there were many, many considerations for making that transition, right? First of all, you know, we wanted to move onto a modern architecture. There were some of the things that we were carrying from the previous GCN architecture, as you mentioned, that we wanted to enhance, including the cache system and also just the execution engine itself, right? So in terms of work scheduling, sequencing of the waves, and even instruction issue and things like this. So there were many things that we wanted to improve.
And so, we knew we wanted to make a big change in the architecture. We wanted to get significant gains in efficiency, and we felt like the opportunity was there for us to bring together what we had done for RDNA, what we wanted to do with CDNA, in order to bring the AMD GPU roadmap more closely together, which gives us then the opportunity for additional optimization in the future that can benefit both.
And one of the reasons we think that’s really important is because AI is everywhere, as you saw in the keynote today. And so, even from gaming and neural rendering and upscaling and all of these things that are now leaning into AI for GPU, even for rendering on the RDNA side, and the things that we’re doing with AI for data center, there’s an opportunity to really bring that together and leverage them on both product lines. So that’s why we wanted to do that.
George: Speaking of two different product lines, I believe last week we published an article on what is effectively MI430. What are the differences in terms of design for HPC versus AI, what design considerations do you have to make at sort of the WGP level for those two different product lines?
Alan: Yeah, great question. I mean, I think not just the WGP level, but the entire SOC level, how do we think about HPC workloads versus AI workloads? They have a lot of similarities, right? They both thrive on high-bandwidth memory, they both need cluster interconnects, they both really enjoy having a high-performance CPU connected to them with a CPU memory system, cache coherency, and all these things. So all of that is already common, right?
And the only thing that's different is the type of numerics that you need for the codes that you're running. And so, the way that we looked at this is we could have included double-precision floating point in the GPU like we've done in the past, or, you know, we could do even better for both if we took advantage of our chiplet architecture and optimized two versions of the compute chiplet: one for traditional simulation with high-precision formats like double-precision floating point, where we really need all of the full IEEE compliance with 64-bit; and then the same thing for the AI workloads, leveraging as highest throughput that we can achieve for both vector operations and the tensor operations for AI.
So having said that, even within the workgroup processor, still many things are in common, right? We need similar bandwidth out of the register file, out of the VGPR. So we're able to leverage all of that, and also all the scheduling, etc. So all of the support hardware within the workgroup processor from the LDS, the caching, the front-end sequencers, all those things, and the SIMDs themselves for the vector ALUs are all shared. And all we really need to do is modulate the number of double-precision units that we implement within the SIMD.
George: Speaking of the SIMDs, previously, a feature or one of the fundamental building blocks of how GCN worked and prior CDNA architectures was that there were four SIMD16 units that were being issued a Wave64 instruction with one instruction per four cycles. With CDNA 5, you have now moved to four SIMD32 units that are being issued one Wave32 per cycle and you have said that you have deprecated Wave64 support. Why did you choose to deprecate Wave64 support considering that RDNA with a similar SIMD setup has Wave64 capabilities?
Alan: Yeah, great question. I think, you know, the way we were looking at it is there’s some additional overhead that we need to carry for Wave64. There are some opportunities to leverage that in some special cases where you can get a little bit of additional efficiency out of issuing a Wave64 on SIMD32. But we felt like the tradeoffs of branch divergence and register pressure and so forth for continuing to keep that capability in most cases, or the common cases from our traces and workload profiling that we’ve done, we think that running in Wave32 almost all the time is a better solution.
And so we were able to, you know, maybe reduce the overhead that you need to carry for Wave64 and deploy that in order to get better common-case performance, and make up for the little bit of optimization that you can get from Wave64 on SIMD32.
George: Okay. So in RDNA 3 and 4, they have four SIMD32 units, but there’s also an extra bank of ALUs per SIMD. Does CDNA 5 have that extra bank of 32 ALUs basically, or is it just 32 ALUs per SIMD?
Alan: Yeah, I think in our SIMDs we have, it’s not always exactly 32 units per lane, right, because in some cases we have more throughput than can be achieved. So we have things like VOP2 instructions...
George: Where you need dual issue?
Alan: Yeah, where you’re effectively doubling the rate, if you will. So you need effectively multiple units per lane in order to get that higher throughput.
George: Speaking of higher throughputs and specifically the register file, with CDNA 5, you have massively increased the amount of VGPRs that a single wave can access: from 256 VGPRs to 1024 VGPRs. Why was that change made?
Alan: The reason we increased the addressing was just to give the waves less register pressure effectively because more of the waves are better able to share more VGPRs.
George: And in terms of the capacity of the register file, it is also 1024 VGPRs in terms of the physical number of registers, which is less than RDNA has. Why fewer in this bigger compute architecture?
Alan: I'll need to go check that.
In a follow-up email, Alan clarified that the VGPR register file size on CDNA5 is sized for the programs that CDNA5 will be used for which has different register file behaviors to what RDNA3/4 will usually run into.
George: On the diagrams it shows a WGP cache of 384 kilobytes and it splits it as 320 and 64 for the LDS and data cache/vector cache whereas some of your LLVM commits show that that number is fungible. So can you change the capacities of the LDS and the vector on the fly, or is that set?
Alan: Yeah. So the architecture was designed to support it to be programmable, but on MI455, we're going to have it as a fixed ratio. We find that for the MI455 deployments, having the larger LDS is almost always the balance that you need for the high tensor throughputs. And so we'll have it always fixed at 320 kilobytes and 64 kilobytes, as you mentioned, for the vector data cache.
George: So then, would MI430 potentially have a different split, or would it be, again, that sort of being able to change on the fly for different workloads?
Alan: It may have a different split.
George: Speaking of caches, that’s a big change specifically at the XCD level. Whereas for CDNA 3 and 4, we had sort of the L1 that was private per CU, then you had the L2 that was per XCD, and then you had the MALL AKA Infinity Cache that acted as sort of the glue to everything, that has changed. Why was that change made?
Alan: Yeah. So there were a couple of reasons that we were addressing with these changes. One of the things that we really wanted to do for MI450 series beyond what we had in 300 and 350 series was die-scope atomics or global atomics. So we wanted to move that back into the L2 cache, where we could get a higher throughput implementation, so that was one of our design goals.
We also wanted to eliminate the kernel boundary flush that we had on the L2 cache that was implemented in the XCD for global visibility on MI350 series. And then we also just wanted to increase the bandwidth for global L2 beyond what would be capable to go across the chip boundaries between the two cache fabric dies.
And so, the way that we did that is we effectively implemented a client-side cache instead of a memory-side cache. So we have these two client-side caches. We used the area that would have been implemented, or would have implemented an Infinity Cache on MI355 on the base die, built a large global L2 cache on each of them, and now all of the shader engines that are in the accelerator complex dies situated on one of those base dies, they all share that L2 cache.
So the last thing that you get the benefit of is from L2 cache data reuse: we get more sharing across more shader engines. So we have eight shader engines now that share a common L2 cache versus what we had on MI300 and 350 series.
And then what we do with those two caches: now each one of them is able to cache all of the memory. So all of the global addresses within the whole GPU, each of them can cache locally. And so those cache accesses don’t have to traverse across the chiplet boundary on the base die, versus on MI300 and 350, where in order to access the capacity of the MALL, the transactions would have to go across longer latency and higher power to go across all of the base die chiplets on the I/O dies.
George: So then, can a WGP on FCD0 access the cache in FCD1?
Alan: They do not.
George: Okay. So what is the bandwidth between the two FCDs?
Alan: We sized the interface between the two FCDs so that we can achieve full memory bandwidth in what we call NPS1. So if you look at the bisectional bandwidth of the two dies, it’s effectively sized relative to the total HBM bandwidth of the device, such that 50% of the bandwidth when you’re sharing from one side to the other... each side needs to access 50% of their accesses on their side and 50% on the other side.
And so that interface is sized so that both sides can do that simultaneously and achieve the full bandwidth of the memory system across the two, plus, as you alluded to, we have some coherency traffic because the Infinity Fabric is maintaining cache coherency between those two GL2 caches. So it’s sized appropriately to handle that traffic in addition to the memory traffic, and then also the I/O traffic that we have coming in from the scale-up and scale-out networks.
In a follow-up email, Alan clarified that the die to die bandwidth is 14 TB/s bi-directional.
George: A way to think of this is like sort of the way that EPYC does its cache coherency, I guess?
Alan: It is very similar. From their L3s. So if you think about the L3 cache implementation on EPYC and the way that that’s implemented, it’s very similar here.
George: Cool. And so we come to the ultimate question. While I usually ask “What is your favorite cheese?”, we already know your favorite cheese! So have you tried any new cheeses as of late?
Alan: (Laughs) Um, yeah, I’m still a fan of cheddar, right?
George: I believe your favorite’s Cabot?
Alan: Cabot is, yeah, what we talked about. But you know what? Since I know that you’re a Tillamook fan... I just want to say that I have been eating a lot of Tillamook ice cream lately!
George: Oh yes! Anyway, thank you so much for sitting down with me.
Alan: Absolutely, George.
2026-07-24 01:37:10
Editor’s Note (7/25/2026): The article has been edited with more information about the L2 behavior along with the bandwidth of the die to die interface.
Hello you fine Internet folks, here at AMD’s Advancing AI event we are looking at AMD’s brand new Instinct MI455X, replacing the older Instinct MI355X at the top of their AI stack. It is AMD’s first GPU designed for rack-scale AI deployments and is based on the new CDNA5 architecture with major changes to the compute unit and SoC, including enhancements to compute performance, improved memory bandwidth, larger memory capacity, and packaged using TSMC’s CoWoS-L.
It comes along with the new Helios rackscale solution, enabling scaling to 72 GPUs in a single pod, up from 8 GPUs for previous MI355X systems. It also features a new fault-tolerant scale-up networking architecture based on UALink over Ethernet (UALoE) with single-hop all-to-all communication across the entire rack.
A single MI455X contains 256 Work Group Processors (WGPs) across 8 Accelerator Complex Dies (XCDs), with a max “engine” clock of 2.4GHz. This enables peak compute figures ranging from 315 TFLOP for matrix/vector FP32 and vector FP16, and up to 40.26 PFLOP for OCP MXFP4. This is paired with 12 stacks of HBM4 each on a 2048 bit bus for a total of 192 channels, giving each GPU a total of 432 GB of memory at 23.3 TB/sec.
Starting with the changes in the “Compute Unit”, AMD now counts the WGPs instead of the CUs for CDNA5. Unlike what they have done on their consumer counterparts where each WGP counts as two CUs.
But just like RDNA4, each WGP comprises four dual issue Wave32 SIMD32 units alongside 4 scalar units which is a massive change from CDNA4’s four single issue Wave64 SIMD16 units.
This means that each WGP can do up to 256 packed FP32 operations per cycle (512 FLOPS if using FMA).
To support this new SIMD design, the VGPR register file has been reorganized with any wave now able to address up to 1,024 VGPRs which is four times the number of VGPRs that a wave could access in prior CDNA and RDNA architectures. Each SIMD still has 128kB of vector registers just like CDNA4 which means it has twice as many (1024) registers available in practice due to Wave32 vs Wave64, but it is still less than the 192kB available on RDNA4. It also means that a single wavefront is in some cases expected to occupy the entire SIMD.
The matrix units also have been beefed up with each matrix unit being able to do up to 8,192 FP4 operations per cycle and with 4 matrix units per WGP you can do up to 65,536 matrix operations per cycle per WGP.
This means that only FP4 and FP8 are practically faster in the new architecture, and the rest of the performance comes from increasing the SIMD width from 16 to 32.
The inter-WGP caches have also changed to support this new design. Both the L1 Data Cache and the LDS have doubled in size to 64 KB of L1 Data Cache and 320 KB. The bandwidth of these caches has also doubled and can now do 2 256 bytes per clock.
Each XCD has 32 WGPs active, 34 physical WGPs of which 2 are fused off, which are broken up into 2 Shader Engines (SE) per XCD each with 16 WGPs. A shader engine is paired with a “Broadcast Arbitrator” replacing the L1 buffer (GL1) from RDNA4, which is now at the SE-level instead of the lower shader array-level. It works both as a write-combine buffer along with providing “up to 4x” bandwidth amplification by presumably broadcasting data.
The memory subsystem now allows multicast loads, accelerating matrix multiplication significantly by reducing redundant memory traffic. Imagine a GEMM that calculates C=A×B, then normally we would have each wavefront individually load a tile of, for example the A operand into the LDS of each WGP. With multicast loads we can instead load this tile of A into all relevant WGPs with a single multicast load instruction.
Each wavefront still loads a different B tile, but MI455X can fetch the common A data from L2 once and use the Broadcast Arbitrator to replicate it into all relevant WGPs private LDS allocations.
AMD calls this up to 4x bandwidth amplification, though the amplification happens after L2 and it does not quadruple L2 or HBM bandwidth. Instead, one unit of L2 traffic becomes four units of locally delivered data, reducing redundant cache reads and chiplet-link traffic while leaving each WGP with a nearby copy of the data.
Moving down to the base dies, MI455X has 192 MB of Global L2 cache split across two Fabric Cache Dies (FCDs). Each FCD contains 96 1 MB SRAM blocks, giving it 96 MB of L2 capacity for the 128 WGPs attached to that cache domain. Each FCD can deliver up to 27 TB/s of L2 bandwidth, for an aggregate of 54 TB/s across the package.
This arrangement also changes how the cache domains behave. On CDNA 3 and CDNA 4, CUs attached to one base die could access the cache on another base die within the same package. CDNA 5 removes that capability: a WGP attached to one FCD cannot access the L2 cache on the other FCD. In that respect, MI455X behaves more like an AMD EPYC CPU, where a core on one CCD cannot directly access the cache attached to another CCD.
AMD made this change to improve the behavior of atomic operations across XCDs, eliminate the kernel flush boundary that MI300-series XCDs required for global visibility, and increase L2 data reuse. Although the two L2 domains are isolated from each other, each 96 MB cache can hold data associated with any global address in the GPU. That allows a WGP’s local L2 domain to cache data that might otherwise require traffic across the die-to-die interface.
Speaking of that interface, the two FCDs are connected by a die-to-die interconnect providing approximately 14 TB/s of bidirectional bandwidth. The FCDs are also connected to 12 stacks of HBM4, with each stack providing 36 GB of capacity over a 2,048-bit interface. At approximately 7.6 GT/s per pin, those stacks provide a combined 432 GB of HBM4 and 23.3 TB/s of memory bandwidth.
For any I/O that is external to a MI455X package, each FCD is attached to a IO die which does the very important job of connecting the GPU package to the host CPU, the scale-up network, and to the scale-out NICs. The CPU to GPU link that was previously serviced by PCIe has been replaced by a dedicated 16 lane AMD Infinity Fabric providing 256GB/s of bi-directional bandwidth which allows a coherent link between the GPU package and the host CPU.
Speaking of scaling up, along with the MI455X accelerator AMD is launching a validated rackscale platform for large scale AI infrastructure deployment. This is enabled by an increase in scale-up interfaces with MI455X having 36 x 400Gbit/s UALoE interfaces that implement 2 x 200G ethernet lanes each, comprising 3.6 TB/s of peak bidirectional bandwidth per GPU.
MI455X also introduces a split DMA architecture, which automatically associates traffic with the optimal link that reduces the topology awareness required for communication.
Helios uses OCP’s new Open Rack Wide form factor, compromising a cabinet 1.2m wide and 1.3m deep that provides. GPU’s are arranged in two groups of nine Compute Trays, each 1 OU in height. Each tray has four MI455X’s and one 96 core EPYC 9006 SP7. Each CPU is paired with 16 x 64GB DIMMS (for a total of 1TB of memory) and five E1.S slots for SSD’s. Six Helios Switch Trays provide a total of 12 switches connected directly via 3 UALoE links per GPU. Each switch provides 432 links at 200Gb/s, for an aggregate of 21.6TB/s per switch. This amounts to a scale-up bandwidth of 260TB/s bidirectionally.
Scale-out is provided via two boards that can contain either 4 or 6 Pensando NICs, depending on how much scale-out the end customer would like, providing up to 43TB/s of backend bandwidth. With 72 GPUs in one rack, a Helios deployment is able to provide up to 2.9 EF at MXFP4 or 22.6 PF at FP32, alongside 31TB of shared HBM4 at a combined 1.7PB/s of memory bandwidth.
The basic GCN microarchitecture underpinned every single one of AMD’s compute accelerators for nearly 15 years starting with Tahiti, followed by Fiji, Vega, and the first 4 generations of CDNA. With CDNA5 AMD has moved over to a microarchitecture that is based on the RDNA series putting a bookend to the long-lived line that was the GCN microarchitecture.
And with that bookend comes the start of a new story for AMD’s Datacenter Accelerators, one that now isn’t just about a single GPU but scaling to 72 GPUs in a rack along with scaling the number of racks. For that AMD is relying on nearly every part of their business from EPYC Server CPUs, to Pensando Networking, to the base GFX12 from the Radeon Division, to combine all of them into the AMD Helios Rack.
