2026-11-12 06:22:57
Hello you fine Internet folks,
I sat down with AMD’s Mike Clark to talk about Zen 6, the evolution from Zen 5 and what some of the differences are between Zen 6 and Zen 6c and what drove some of those changes.
Hope y’all enjoy!
The transcript below has been edited for readability and conciseness.
George Cozma: Hello, you fine Internet folks. We’re here at AMD In-Depth, where we’re talking about Venice. With me, I have a returning guest, Mike Clark. Would you like to tell the audience who you are?
Mike Clark: Hi, I’m Mike Clark, the original architect of Zen. Some people refer to me as “Zen Daddy.” I’ve been working on cores here at AMD for 33 years, and we’ve got another great one coming, so I’m very proud of the design team I get to represent here today.
George Cozma: I’ve had some hands-on time with Zen 6c. Let’s talk about the change that’s happened with Zen 6 and Zen 6c, what I refer to in my piece as the Zen 6 family. Why is it different from previous generations?
Mike Clark: Right. Starting with Zen 4, we’ve been going through this evolution of having two core families. Before, we were really just using the denser technology as an area play. In this generation, having learned a lot, as we were filling out that denser version of the core, we wanted to leverage it to add some extra IPC within the core. So if you look more in-depth at what we have, what used to be our classic design, we now think of as our high-frequency design. And our dense technology is now our top-of-the-line, no-questions-asked performance core, because we’ve also enabled the full L3 in that implementation, at 4MB per core. So we have 32 of those cores, 128MB of L3, and the highest IPC you can get in that core.
George Cozma: Awesome. So far, one thing I’ve noticed is that the vector scheduler is a bit bigger on Zen 6c. What was the reasoning for increasing that on Zen 6c?
Mike Clark: Well, that wasn’t the only thing we increased. One principle we follow is that we don’t want to create a delta between the two architectures that software has to be optimized for. We want them to have the same fit and feel, so the same optimizations apply to both. But because of its reduced frequency target, Zen 6c can have a little more capacity in the queues, since it doesn’t have to reach the maximum frequency the high-frequency core has to hit. The floating-point scheduler is just one example. There are other queues throughout the core that are larger in Zen 6c, to pull out a little more IPC and get a little more leverage in that SMT environment as well.
George Cozma: Interesting. Both Zen 6 and Zen 6c build on the design you had in Zen 5, and one of the interesting things with Zen 5 was the dual decode setup. Has the front end changed much to let a single thread make better use of both decode clusters?
Mike Clark: Yes. As you said, Zen 5 was a new foundation, with the ability to fetch two basic blocks and get them down the line. But it gets quite complicated with one thread running larger-footprint workloads, where some of it might be hitting the I-cache and some of it might be hitting the op cache, and you’re trying to get everything synced up. So we spent a lot of time optimizing that and making it more efficient from both a delivery and a power perspective: less power wasted, and more efficient delivery from that new front end.
George Cozma: Awesome. Going down into the execution engine, I also noticed that the number of integer multiply units has gone down from three to two. Could you talk a bit about what drove that choice?
Mike Clark: It really came out of the Zen 5 design, where we had a more unified scheduler. To make a scheduler simpler, you want the units it schedules into to be very uniform, and the way we had set up those schedulers, adding a third multiplier to the group it was scheduling into was easy and didn’t add complication. In Zen 6, we realized that to really scale out the architecture, we needed to go back to more distributed schedulers, which now schedule into the ALUs. We hadn’t seen much uptick from that extra multiplier on Zen 5, so to optimize the area and the schedulers, we went back to two multipliers.
George Cozma: Speaking of integer multiply, on the vector side you’ve added two more VNNI units. What was the impetus for adding them, and at what point in the design did you add them?
Mike Clark: As I said in the talk, these are long design programs. We weren’t designing Zen 6 to be an agentic core. When we started, no one was really even using that term. But Zen 5 was a whole new foundation for AI workloads, and we’ve always believed the core has a place in AI: orchestration, data type manipulation, smaller models. So we were always focused on making the core good at AI. It’s not going to beat a GPU, and that’s not the point. We realized we probably should have some more VNNI, given where we were seeing inference go, and the team rallied around that. It was kind of late, since these are long-lead projects and the floor plan was set, but the team rallied, we were able to squeeze a couple more VNNI pipes in there, and it was a big success.
George Cozma: Again on the vector side, one of the oddities of Zen 5 was that although the wires were all there, the load unit could only do two vector loads at a time. What changed within the Zen 6 core to allow four vector loads, assuming they’re either 128 or 256 bits?
Mike Clark: We can do the 256-bit loads. Since we were able to do two 512-bit loads, we already had 1,024 wires headed over there. Obviously, using them piecemeal versus as one contiguous set complicates the design. I think it’s something we always wanted to do on Zen 5, but with all the change and getting it working, we left that for the Zen 6 team, and they were awesome. We’re all really one team at AMD anyway, and sometimes things intersect at different points in the roadmap. This was the time to do it.
George Cozma: Speaking of bandwidth into the caches, the L2 in Zen 5 had some oddities where it couldn’t always hit 64 bytes per cycle on reads. What changed in Zen 6 to allow that? Because I’ve been able to measure it, and it does hit that.
Mike Clark: Honestly, sometimes you just don’t see some of these things. A lot of it was that we would be over-prefetching, and prefetches that hit in the L2 would be robbing L2 bandwidth. So we made sure we could sustain those prefetches alongside demand requests, reduce the conflicts, and deliver the full 64 bytes per cycle that we want.
George Cozma: Awesome. Circling all the way back to the front: the all-important branch predictor. There seems to have been a reduction in branch misprediction latency, so when you miss a branch, the number of cycles it takes to recover has gone down. What I also noticed, which is really neat, is that there wasn’t any reduction in accuracy. So how did you pull off not only a faster branch predictor, but one that’s even slightly more accurate than Zen 5’s already very accurate predictor?
Mike Clark: The branch prediction team, as you know, is very algorithmic. There’s lots of design space and lots of exploration, and AI has even been getting involved in helping us figure out how to optimize it. It’s a great optimization problem: the number of TAGE tables, how much information they hold, accuracy. So we set out to make a more efficient one. That was an area where we didn’t really want to grow; we thought we could be more efficient in the way we lay out the tables for prediction. And we’re always focused on our pipeline length. No matter how good a branch predictor you build, there are always hard-to-predict branches, and you’re always going to mispredict. So each generation, we’re monomaniacally focused on getting going again as fast as we can when we’re wrong. There are lots of unique tricks we pull to get that redirect back into the front end and get it flowing as fast as possible. And you’ve observed it, which is great.
George Cozma: Yeah. I haven’t had time on Zen 6 yet, but I have had time on Zen 6c, and in SPECint 2026 it matches your fastest Zen 5 server CPU. That’s very impressive, especially considering the clock deficit. Wrapping this all up and moving outside the core into the L3, there are obviously topology differences between the 12-core and 32-core CCDs. How much were you able to mitigate those differences inside the core, building up structures to tolerate latency, to compensate for the slightly increased L3 latency?
Mike Clark: As I always talk about, the design has to be balanced. As we grow the capacity of a structure, the new technology helps it shrink, so the physical distance doesn’t grow linearly, but unfortunately the distance does grow. Inside the L3, there are a lot of tricks I can’t talk about...
George Cozma: The secret sauce!
Mike Clark: [In order to] maintain bandwidth within the L3 mesh so we can keep delivering to the cores. And in the cores, we do grow the queues to handle those higher latencies.
George Cozma: Awesome. My penultimate question: Verano is adding LPDDR5X to the EPYC ecosystem. Were there any design considerations between LPDDR5X and DDR5 that you had to think about when designing the core?
Mike Clark: At the CCX level, we focus a lot on bandwidth per core. I think Mahesh talked a lot about the SDP ports, with 32-byte reads per port, so we try to make sure we have enough bandwidth between the CCD and the IOD. But we were already well provisioned for bandwidth with MRDIMMs, so that construction, the winglets and choosing a different memory technology, didn’t really have a major impact on how we built the core.
George Cozma: Awesome. Usually this is where I’d ask what your favorite type of cheese is, but since we already know that: have you tried any new cheeses lately?
Mike Clark: I have. My new favorite is Cotija cheese.
George Cozma: Oh!
Mike Clark: I’ve been eating a lot of tacos.
George Cozma: You’ve got to love Cotija on tacos. It’s absolutely amazing. Well, thank you so much for sitting down with me. If you like videos like this, hit like and subscribe; it does help with the algorithm. There are PayPal and Patreon links in the description, as well as a transcript of this on the Chips and Cheese website. Thank you so much.
Mike Clark: Thanks for having me, George. I always look forward to coming and talking with you about what this awesome AMD CPU design team has created, and you always have so many good questions. I wish I could answer more, since we do have to protect our secret sauce, but I really enjoy helping you understand some of our thought processes, the challenges we face, and how we’ve been able to overcome them.
George Cozma: Well, it’s always fun to sit down with you and talk, not just about what you’re doing, but also what the team is doing and what you’re looking forward to in the future. So thank you so much.
Mike Clark: Thank you.
George Cozma: I’ll see you in the next one. Have a good one, folks!
2026-11-11 01:00:10
Hello you fine Internet folks,
Today we’re looking at AMD’s next-generation Zen 6 CPU core family. We’ll explore the changes from Zen 5 and what the differences between Zen 6 and Zen 6c tell us about AMD’s approach to microarchitecture.
Hope y’all enjoy!
From a bird's eye view, Zen 6 is the natural evolution of Zen5, with the expansion of many OoO structures and iterative improvements found throughout the core.
Digging in deeper however tells a different story, with AMD heavily re-working several parts of the core, reminding me of the changes from Zen 1 to Zen 2.
At a high level the changes AMD made to the integer schedules stand out, returning to a layout reminiscent of Zen 3 and Zen 4.
Zen 5 had the best direction predictor of any core we have measured and Zen 6 seemingly builds upon that same direction predictor.
The biggest difference we do see is in what happens once the direction predictor gets overwhelmed. Both cores hit that wall in about the same place and past it, the predictor is left guessing at random data, so the time per branch mostly reflects how long the core takes to recover from a wrong guess.
That plateau sits at roughly 3.5 to 4 ns of extra time per branch on Zen 5 and roughly 3 to 3.5 ns on Zen 6 so Zen 6 is paying less for each branch it gets wrong, which speaks to AMD reducing the latency of a branch mispredict compared to Zen 5.
Zen 6 has also improved the ability of the core to do 2 branches per cycle with Zen 6 able to do 2 branches per cycle within the L1 instruction cache.
When the footprint exceeds the L1i, then it falls to just a single branch per cycle for the rest of the BTB except for the 64B test which goes up to 4 cycles per branch for the L2 BTB.
When comparing the BTB of Zen 6 to Zen 5, Zen 6 appears to have focused on improving the speed of the BTB compared to beefing up the capacity.
The accuracy of Zen 6’s Branch Predictor Unit is very similar to Zen 5’s while also being faster which is a good thing considering that Zen 5 already had a very accurate Branch Predictor.
Zen 6 carries forward the same dual decode cluster schema that Zen 5 introduced but, unlike Zen 5 where a single thread could only effectively use a single decode cluster, now with Zen 6 a single thread can use both decode clusters.
Outside of the L1i, Zen 6 looks very similar to Zen 5’s behavior on the instruction side of things.
The execution engine of Zen 6 is overall very similar to Zen 5, but there are some important differences to be mindful of.
On the integer side, Zen 6 brings back Zen 4’s NOP pairing behavior, retiring up to 16 NOPs per cycle. Zen 6 also comes closer to theoretical peak throughput for simple integer instructions than Zen 5, sustaining 5.96 64-bit adds per cycle against Zen 5’s 5.25 with the same six ALUs.
AMD has reduced the number of integer multiply units from three to two, with PDEP and PEXT throughput also falling from three instructions per cycle to two. According to AMD, Zen 5’s third integer multiply unit did not justify its area cost.
On the vector side, the floating-point units are unchanged, with Zen 6 still handling two 512-bit FADDs, FMULs, or FMAs per cycle. The change is in VNNI, where Zen 6 doubles the number of units from two to four and our 512-bit VNNI test goes from 1.99 to 3.88 instructions per cycle, with Zen 6c matching it. VNNI instructions are useful for operations like calculating the inner loop of quantized inference as well as for video codecs using them for video filters as well.
Moving over to the other notable change, Zen 5 could already perform four per cycle, but our test shows only two of them could be vector loads, regardless of if they were 256 or 512 bit loads. Zen 6 raises 256-bit loads to four per cycle on both cores while leaving 512-bit loads at two, so both widths now reach the same 128 bytes per cycle which matters because most shipped vector code still targets AVX1/2.
In the past the difference between the Classic cores and the Dense cores in terms of the architecture were, for all intents and purposes, functionally identical with the biggest changes being in the number of cores on a CCD and the amount of L3 cache per core. With the Zen 6 family, that has changed.
Starting inside of the core, the changes occur after micro-op dispatch and seemingly end after the load queue.
The main differences between Zen 6 and Zen 6c lie in the capacities of their out-of-order structures. Several of these structures are larger on Zen 6c, allowing it to keep more work in flight and absorb more latency than Zen 6. The reorder buffer grows from 576 to 640 entries, the integer register file from 288 to 352, the vector scheduler from 138 to 180, and the load queue from 280 to 330.
The differences suggest that AMD is specializing the two designs. Zen 6 appears tuned for high frequency, while Zen 6c has more capacity to tolerate comparatively high memory latency.
Zen 6 and Zen 6c use 12-core and 32-core CCDs, respectively, but both provide 4 MB of L3 cache per core. That gives the 12-core CCD 48 MB of L3 cache and the 32-core CCD 128 MB.
L3 latency has increased from about 47 cycles on Zen 5 to 55 cycles on Zen 6 and 66 cycles on Zen 6c. The larger caches and greater number of mesh stops likely contribute to the increase. In nanoseconds, Zen 6’s higher clock hides most of the increase, at 10.7 ns against 10.3 ns on the Zen 5-based EPYC 9355P, while Zen 6c’s lower clock and larger CCX put it at 16.4 ns.
Moving over to bandwidth, we see that Zen 6 fixes Zen 5’s odd L2 bandwidth behavior, where a core pulled only about 42 bytes per cycle across most of the L2’s capacity, in Zen 6 the read path now reliably delivers 64 bytes per cycle. The rest of the cache hierarchy is largely unchanged in bandwidth per cycle.
Store-to-load forwarding is seemingly unchanged from Zen 5 and is where the behavior of Zen 6 and Zen 6c re-unify.
A 32-bit load from the same address as a 64-bit store completes with effectively zero latency which lets Zen 6 handle two dependent store-load pairs per cycle. Loads fully contained within the store complete in 7 cycles regardless of alignment with partial overlaps costing 14 cycles, matching Zen 5.
The EPYC 9686F is near the top of our estimated single-threaded SPEC CPU 2026 results. It roughly matches the Ryzen 9 9950X3D2, AMD’s highest-end client CPU from the previous generation, in integer rate and beats it in floating-point rate. At a matched 4.1 GHz, the 9686F leads the 9950X3D2 by about 4% in integer and 11% in floating point, despite the X3D part’s much larger L3.
The EPYC 9996 is no slouch either. It roughly matches the previous-generation EPYC 9575F in single-threaded performance, despite running at lower clocks and having four times as many cores. It also edges out NVIDIA’s Vera in integer and leads it by about 20% in floating point.
The EPYC 9686F leads on almost every subtest and finishes about 18% ahead of the EPYC 9996 in both suites. It scores 6.20 versus 5.25 in integer and 10.6 versus 8.99 in floating point, with some of its largest leads in 714.cpython, 766.femflow, and 772.marian.
Against the Zen 5-based EPYC 9575F, the 9996 scores 5.25 versus 5.13 in integer and 8.99 versus 9.01 in floating point while running about 900 MHz slower. The 9996 also edges out the 9686F in 721.gcc and 765.roms, two subtests where the 9686F’s higher clock provides no advantage.
Top-down breakdowns of pipeline slots on the two Zen 6 chips show where the 9686F’s lead comes from. The integer-rate profiles look almost identical in core-bound subtests. In 750.sealcrypto, 710.omnetpp, 735.gem5, and 753.ns3, retiring slots account for nearly the same share on both chips, with differences of less than one percentage point.
Memory-heavy subtests show larger gaps. Moving from the EPYC 9996 to the EPYC 9686F, backend memory stalls rise from 37.9% to 50.0% of pipeline slots in 729.abc, from 19.0% to 34.1% in 723.llvm, and from 24.8% to 33.2% in 708.sqlite. The share of retiring slots falls accordingly. Zen 6c’s lower clock and larger shared L3 cache may help explain its lower memory-stall share.
The floating-point profiles look similar for most subtests. The retiring share in 731.astcenc, 772.marian, and 767.nest differs by about one percentage point or less, and 709.cactus spends over half its slots stalled on front-end bandwidth on both chips.
The larger gaps again appear in memory-heavy subtests. Moving from the EPYC 9996 to the EPYC 9686F, backend memory stalls rise from 42.5% to 55.3% of slots in 765.roms, from 60.9% to 67.6% in 749.fotonik3d, and from 17.8% to 33.8% in 782.lbm. These are also tests where the 9686F’s higher clock helps least: it trails the 9996 in 765.roms, roughly ties in 749.fotonik3d, and leads by about 11% in 782.lbm, despite clocking up to roughly 29% higher.
Running the 9686F at 4.1 GHz to match the 9996 removes the clock advantage, and the overall picture flips: the 9686F scores 4.92 and 8.64 at that clock, putting Zen 6c about 6.7% ahead per clock in integer and 4% in floating point. At the same clock speed, Zen 6 and Zen 6c look almost identical in core-bound integer subtests. In 750.sealcrypto, retiring slots account for 67.3% of the total on the EPYC 9686F versus 67.6% on the EPYC 9996.The floating-point profiles also look similar in several subtests at matched clocks. Both chips spend over half their pipeline slots stalled on frontend bandwidth in 709.cactus. Their retiring shares differ by about two percentage points in 722.palm and 766.femflow.
The 9686F still spends a larger share of its pipeline slots stalled on memory. Backend memory stalls account for 50.3% versus 42.5% on the 9996 in 765.roms, 24.9% versus 17.8% in 782.lbm, and 63.3% versus 60.9% in 749.fotonik3d. Zen 6c’s 128 MB L3 cache is likely serving more of these requests than Zen 6’s 48 MB cache, reducing the need to access main memory.
The clearest difference is in frontend latency stalls, which account for a much larger share of slots on the 9686F in subtests with large codebases. In 727.cppcheck, that share is 35.2% versus 16.6% on the 9996; in 721.gcc, it is 25.8% versus 10.0%. With clock speed matched and both designs sharing the same frontend, those stalls most likely reflect differences in the cache and memory hierarchy.
The floating-point profiles also look similar in several subtests at matched clocks. Both chips spend over half their pipeline slots stalled on frontend bandwidth in 709.cactus. Their retiring shares differ by about two percentage points in 722.palm and 766.femflow.
The 9686F still spends a larger share of its pipeline slots stalled on memory. Backend memory stalls account for 50.3% versus 42.5% on the 9996 in 765.roms, 24.9% versus 17.8% in 782.lbm, and 63.3% versus 60.9% in 749.fotonik3d. Zen 6c’s 128 MB L3 cache is likely serving more of these requests than Zen 6’s 48 MB cache, reducing the need to access main memory.
The Zen 6 Family improves upon its predecessor in various ways, an upgraded frontend allows one thread to utilize both decoders, while a faster branch predictor makes it more forgiving on misses, all tied together with larger OoO structures allowing it to better hide memory stalls. All of this coming together to provide a generational uplift from Zen 5.
However, I think the more interesting change is the split between Zen 6 and Zen 6c. The classic and dense cores now differ within the out-of-order engine, with Zen 6 appearing tuned for higher clocks and Zen 6c providing more capacity to absorb latency. The shared foundation is still clear, but those differences make the choice between them more dependent on workload behavior.
The performance results are encouraging as well with the estimated single-threaded SPEC CPU 2026 scores, the EPYC 9686F leads the Zen 5-based EPYC 9575F by about 21% in integer and 18% in floating point. The EPYC 9996 roughly matches the 9575F despite running about 900 MHz slower which are strong results for both core designs.
AMD’s answer to the classic-versus-dense question used to be the same core at two densities but now with Zen 6 it is two cores, each shaped for where it will run. That now leaves me wondering what AMD will do with the Zen 7 Family and if we will see even more specialization of the same basic core design or if we will see a larger split of the designs.
2026-10-25 07:13:33
Qualcomm’s Snapdragon X2 Elite uses a hybrid core setup, breaking away from the all-big-core strategy that the company previously used with in-house core designs. The Snapdragon X2 Elite Extreme X2E96100 contains 18 cores, divided into 6 “performance” cores and 12 “prime” cores. Qualcomm’s “performance” cores fill the same role as little cores or efficiency cores on other chips, so I’ll be calling them E-Cores for simplicity. Like E-Cores on other chips, Qualcomm’s E-Cores are optimized for lower power and performance points than their big-core counterparts. These smaller cores can contain background tasks and lightweight applications that don’t require maximum performance, which saves power by not waking up the big cores. Under heavy multithreaded workloads, the E-Cores can step in to boost overall throughput.
Here, I’ll be looking at Qualcomm’s E-Core architecture in the Asus Zenbook A16 UX3607OA, which Asus has kindly sampled.
Qualcomm’s E-Core is a 4-wide out-of-order architecture running at 3.6 GHz, making it a modest affair next to the likes of Intel’s Skymont, or even the older Crestmont architecture used in Intel’s last-gen Meteor Lake chip. But despite being “only” a 4-wide core, Qualcomm’s E-Core can still punch pretty hard thanks to a combination of good branch prediction, substantial backend resources and large, low-latency caches.
On the instruction set side, Qualcomm’s E-Cores support SVE2 and SME to maintain consistent ISA extension support across the chip. Both features are new to the Snapdragon X2 Elite, since Qualcomm’s prior Snapdragon X Elite lacked SVE support. Besides vector extensions, Qualcomm supports a grab bag of security-focused ISA extensions like pointer authentication and memory tagging (MTE).
Branch prediction can heavily affect both performance and power efficiency, so even density optimized cores have to make substantial investments into fast and accurate branch prediction. For their E-Core, Qualcomm has decided on a branch predictor that does well on the basics, but doesn’t incorporate some of the bling seen on the latest branch predictors on recent AMD, Arm, and Intel cores. A 1024 entry branch target buffer (BTB) is fast enough to deliver back-to-back predictions for taken branches. Intel and AMD’s big cores as well as density optimized cores from Arm like the Neoverse N2 can do two taken branches per cycle. Unlike Qualcomm’s prior Oryon core in the Snapdragon X Elite, the 1024 entry BTB here seems to be decoupled from the L1 instruction cache. Measured capacity doesn’t vary depending on code footprint or branch spacing.
Throughput drops to one taken branch every three cycles once the test spills out of the 1024 entry BTB, with another slowdown visible after exceeding the 128 KB L1 instruction cache’s capacity. Qualcomm might be using a second level BTB tied to the instruction cache, or could even be calculating direct branch destinations as those instructions arrive at the decoder. While doing so is simpler and more area efficient than implementing a very large L2 BTB, it does mean the branch predictor can’t follow taken branches after an instruction cache miss. That could limit the core’s ability to accurately prefetch instructions over large code footprints.
For direction prediction, Qualcomm’s E-Core can recognize fairly long patterns and handle a lot of them. The core can track roughly 16K patterns with 512 branches in play, or somewhere around 1K patterns with a single branch. Prediction accuracy across Geekbench 7’s workloads is roughly on par with Arm’s Neoverse N2, and better than Arm’s Neoverse N1.
A 40 entry return stack helps predict returns, and can sustain a call+return pair every two cycles. That’s a taken branch every cycle on average, making Qualcomm’s return stack faster than the one on AMD’s Zen 5 or Intel’s Skymont. In capacity terms, Qualcomm’s return stack lands in the middle of the pack. It’s much smaller than the 128 entry structure in Intel’s Skymont and Crestmont, but larger than the 24 entry one on Lion Cove.
Capacity has to be taken in context though. Some cores like Intel’s Lion Cove seem to fall back to an alternate predictor on a return stack underflow, letting them avoid mispredicts. Qualcomm’s E-Core and Intel’s Skymont in contrast see taken branch latency rise to over a dozen cycles, which is high enough to indicate mispredicts. It’s a story of different designs striking a balance in different ways. One approach is to design a small return stack with only a minor penalty on underflow. Another is to create a larger return stack so you only have to worry about the underflow case once in a blue moon. Qualcomm is leaning towards the latter. A 40 entry return stack should be plenty, though it stops short of Skymont’s crazy structure.
Qualcomm strikes a similar balance with the instruction cache. A 128 KB instruction cache is much larger than the 64 KB or 32 KB ones found on contemporary designs, which helps the core avoid having to fetch instructions from L2. If code does spill out to L2, frontend bandwidth drops to about two instructions per cycle. It’s not bad, but it is lower than what Skymont can sustain from its shared L2.
The L2 cache acts like a last level cache for the CPU, so missing it leads to very poor instruction bandwidth.
After the decoders translate instructions into micro-ops, the core has to allocate backend tracking resources for those micro-ops for bookkeeping throughout the out-of-order execution process. That process is a convenient place to carry out optimizations that expose extra parallelism to the backend, and Qualcomm does some of this. Move elimination is one example, and Qualcomm’s E-Core can do this for up to two dependent register-to-register MOVs per cycle. Eliminated MOVs execute with zero latency, as the renamer “executes” them by manipulating the register alias table to point multiple architectural registers to the same physical one.
Putting chained, dependent MOVs too close together seems to create a hazard that causes move elimination to fail. An uninterrupted chain of dependent MOVs executes at 1.38/cycle, indicating that some MOVs get eliminated, but others have to be carried out by the downstream execution units. Interleaving NOPs so that there’s no more than two MOVs in every four instructions (i.e., a decode group) brings throughput up to two dependent MOVs per cycle. Move elimination efficacy decreases depending on how many dependent MOVs go together in the same decode group.
Independent MOVs can execute at four per cycle. Even though latency isn’t a concern with independent instructions, move elimination still provides benefits because eliminated MOVs don’t need a physical register allocated, which increases reordering capacity.
Qualcomm can also eliminate adds with small immediates, though with an implementation that’s quite limited compared to what Intel has been doing. Add-immediate elimination is subject to the same two per cycle limitation as move elimination, and furthermore requires the sum of immediates in eliminated adds to be 255 or less. To give an example, add x13, x13, 254 and add x13, x13, 1 can be optimized, but two add x13, x13, 128 instructions cannot. Intel’s add-immediate optimization on Golden Cove has higher throughput and works with larger immediates.
Zeroing registers by MOV-ing an immediate value of zero into them doesn’t get special treatment. Qualcomm still has to allocate a physical register to hold the zeroed value.
Qualcomm has selected reorder buffer and register file sizes closer to those of AMD’s Zen 4, rather than the likes of older 4-wide cores like Intel’s Skylake and Arm’s Neoverse N1. Other supporting structures like memory ordering queues are sized more in line with Skylake, though they’re still slightly larger.
Scalar integer execution is a primary focus on Qualcomm’s E-Core, with four ALU ports that match core width. These are fed by a distributed scheduler, which feeds each ALU port with a 19 entry scheduling queue. Three of the ALU ports can handle flag-setting instructions like SUBS/CMP, and one port can execute integer multiplies with 3 cycle latency. Two ports can resolve branches, though that throughput can only be sustained with not-taken branches because taken branches are bottlenecked by branch prediction throughput.
Floating point and vector execution isn’t emphasized as much, but Qualcomm has still created a reasonable two-port FPU for an efficiency focused core. Both FP/vector ports can execute common FP and integer vector operations, as well as cryptography ops like aese. Execution latency is about average for a modern core.
SVE is supported with a 128-bit vector length, meaning that SVE’s benefits will come from new instructions and prediction support rather than making more compute throughput available. One of the FP pipes can handle comparisons that write a predicate register, like cmpeq. Predicate instructions that don’t access vector registers, like SVE’s while instructions, can issue through an integer pipe to conserve vector/FP issue bandwidth. The core can only issue one while* instruction per cycle, so maybe only one of the integer pipes can write to the predicate register file. SVE also introduces non-contiguous memory accesses.
Qualcomm’s E-Core has two memory access pipes, both of which can service loads or stores. Store forwarding latency is 6-7 cycles for all cases where an older store overlaps a load, regardless of the load’s alignment within the store. Impressively, Qualcomm avoids a failed store forwarding penalty in partial overlap cases. On the other hand, Qualcomm doesn’t have a very fast case for exact address matches the way Skymont does.
To speed up virtual to physical address translation, Qualcomm uses very large translation lookaside buffers (TLBs) to cache frequently used translations. Qualcomm’s slides indicate that the Snapdragon X2 Elite’s big cores have a 224 entry, 7-way set associative L1 DTLB backed by a 8K entry L2 TLB.They didn’t specify TLB capacity for the E-Core, but performance counters suggest very large TLBs on it as well.
L1 DTLB misses start ticking up as test sizes exceed 768 KB, suggesting a 192 entry L1 DTLB. L2 misses start after 32 MB, which is consistent with a 8K entry L2 TLB. Getting a translation out of the L2 TLB takes more than eight cycles, based on comparing L2 latency using 4 KB and 2 MB pages.
Qualcomm’s TLBs support 4 KB and 64 KB small pages sizes, though curiously not 16 KB pages. Windows uses 4 KB pages, and Linux usually uses 4 KB pages as well. However, Ubuntu does offer a kernel version with 64 KB pages.
A 64 KB L1D cache serves as the first stop for memory accesses, and delivers 3 cycle load-to-use latency. Indexed addressing increases latency to 4 cycles. It’s a good combination of speed and capacity, considering that many other modern cores have smaller 32 KB or 64 KB L1D caches and start with 4 cycle latency.
L1D misses head to a shared L2 cache. Unlike many other designs, but like Apple’s M1, Qualcomm doesn’t use a core-private mid-level cache. L2 latency is 21 cycles, or just two cycles more than Skymont’s 4 MB shared L2. While Skymont does have better actual L2 latency thanks to higher clock speeds, Qualcomm’s L2 is very fast for the capacity it offers.
SVE adds scatter/gather instructions, which lets one instruction access memory from non-contiguous memory locations. Gathers execute at a pedestrian pace of roughly one element per cycle. There’s no sign of a “combined gather” ability as described by Fujitsu for their A64FX and Monaka cores, where two elements fetched from the same 64B or 128B block can be handled with a single flow. Latency for a gather instruction is high at just under 4 ns, or 14 cycles. High gather latency is typical. For comparison, gathers on Intel’s Skymont execute with 3.7 ns of latency on a L1D hit, or 17 cycles.
L1D bandwidth is modest at 32 bytes per cycle, limited by the core’s 128-bit vector width and two memory pipelines. A linear read pattern achieves slightly more than 16 bytes per cycle from test sizes that fit in L2, with no bandwidth decrease when the array spills out to DRAM. Qualcomm’s L2 can deliver more than 32 bytes per cycle to a core even though the core can’t operate on that much bandwidth. That can be shown by only loading a subset of the data in each 64B cacheline. Any L1D miss will cause a fill for the entire 64B cacheline regardless of how much of that data the core ultimately uses, so “raw” L2 bandwidth can be inferred from the number of 64B lines touched.
The L2 bandwidth can provide just over 400 GB/s of read bandwidth to the six E-Cores it’s connected to. Per-core L2 bandwidth therefore doesn’t decrease under load, and remains at about 18 bytes per core cycle. Touching only the first 128 bits of each 64B cacheline as described above implies that Qualcomm’s L2 can offer over 500 GB/s of raw bandwidth, even though the cores can’t operate on all of it. Intel’s Skymont for comparison has smaller caches at the cluster level, with more per-core cache bandwidth to feed the higher throughput Skymont architecture.
A 4-wide, 3.6 GHz core isn’t flashy next to Intel’s 8-wide, 4.6 GHz Skymont, or AMD’s Zen 5c that uses the same architecture as its higher performance counterpart, albeit with a different physical implementation. But Qualcomm’s conservative E-Core architecture is a deliberate decision. The company is perfectly capable of delivering fast, high throughput cores, as shown by the 9-wide, 5 GHz P-Cores on the same chip. For their E-Cores though, Qualcomm seems to be focusing harder on reducing power consumption than increasing throughput for multithreaded applications. While not explicitly stated in Qualcomm’s slides, the “performance” core cluster seems aimed at containing lightweight applications and background tasks. “Efficiency” is probably a better term for the cluster.
For handling less performance critical tasks, a 3.6 GHz, 4-wide core looks like a good design point. Higher throughput cores are harder to feed and run into diminishing returns as any hiccup tends to leave more pipeline slots unused. Meanwhile, a narrower core is easier to build. A renamer for example has to check each instruction for dependencies on all previous instructions in that rename group. An instruction queue needs enough read and write bandwidth to handle core width. A decoder might need checks across multiple slots to determine whether multi-uop instructions can be handled in the same cycle. All of that complex logic can be seen as unused transistors whenever a pipeline slot goes unused. Intel stayed with 4-wide designs all the way from Core 2 in 2006 to Skylake in 2015, and likely with good reason. A 3-wide design probably runs into too many situations where core width becomes a limiter, while a 5-wide one starts dipping more into diminishing returns.
2026-10-20 02:26:50
Nvidia’s Vera server chip aims for high per-core performance. Its Olympus cores have massive out-of-order structures and use high per-clock throughput to meet that goal within a high core count server chip. Core design however is only one part of the equation. Off-core memory access can heavily impact core performance, and higher core counts complicate interconnect design. The task of feeding Vera’s Olympus cores falls to Nvidia’s second generation Scalable Coherency Fabric, or SCF. SCF plays a similar role as Arm or Intel’s mesh interconnect, acting as a large chip-wide network that connects cores, cache, and off-chip IO. Like the first generation SCF used in Nvidia’s Grace chip, Vera’s SCF probably uses a mesh topology.
SCF is built from several node types. Coherency Switch Nodes (CSNs) host cores and cache, acting a bit like Intel’s mesh stops. Unlike Intel’s mesh stops, Nvidia has each CSN host a pair of cores. Doing so helps keep node count under control, and is a tactic also used by Ampere’s Altra Max. Ampere uses Arm’s CMN-600, and has each fully coherent request node (RN-F) host an Arm DynamIQ Shared Unit (DSU) containing a pair of Neoverse N1 cores.
Memory Switch Nodes (MSNs) serve as a bridge from SCF to Vera’s LPDDR5X memory controllers. Vera has a 1024-bit LPDDR5X-9600 setup, so the MSNs can move up to 1.228 TB/s of bandwidth. Bridge Switch Nodes (BSNs) host PCIe controllers and the cross-socket link. Vera has 88 PCIe lanes, which together can provide 665.5 GB/s of bandwidth to peripherals. NVLink C2C serves as the cross-socket link, and provides 1.8 TB/s of theoretical bandwidth.
Grace broke ground in the server scene by using LPDDR5X memory, but had mediocre DRAM performance compared to its peers. On Grace, memory latency was high at over 200 ns. Grace’s memory bandwidth was fine in an absolute sense, but fell a bit short compared to contemporary DDR5-based server chips. Vera turns everything around. The massive 1024-bit LPDDR5X interface now gives Nvidia’s server chip more bandwidth than its peers. Latency has been fixed and is now comparable to that of DDR5-based setups. Vera also adopts SOCAMM2 modules, making its memory setup field serviceable and potentially upgradeable. Grace by contrast used soldered memory, so memory chip failures could be problematic.
Vera maintains excellent control over latency under increasing bandwidth load. A pointer chasing thread doesn’t observe more than 200 ns of latency, even with other threads pushing the chip’s memory bandwidth limits. Other server platforms show a more distinct “hockey stick” pattern, where latency rises sharply as bandwidth load approaches the memory controller’s limits.
Nvidia’s combination of high bandwidth and low latency under load should let it control noisy neighbor effects better than other servers. Another way of looking at it is that Vera can out-scale other chips in DRAM-bound workloads, even if Vera has fewer cores.
Even a massive LPDDR5X subsystem won’t come close to being able to handle every program memory access, so Vera tackles the memory wall with large caches like other high performance CPUs. Vera’s caching strategy resembles the ones used on other large mesh-based designs. A 164 MB L3 cache is shared across all cores, and maintains a non-inclusive relationship with core-private L2 caches. This arrangement gives Vera up to 340 MB of effective last level cache capacity if there’s no data duplication across the L2 caches. For addresses that are frequently accessed by different cores, the L3 can economize cache capacity by only keeping one copy of the data at a shared address. Vera’s L3 read bandwidth comes out to about 3.35 TB/s, or 33.1 GB/s per core. A single core can achieve 88.5 GB/s without contention from its peers, so 33.1 GB/s represents a significant drop. However, that’s still a reasonable amount of bandwidth in an absolute sense, and is likely to be adequate because the 2 MB L2 caches should reduce L3 bandwidth requirements.
Nvidia’s caching strategy contrasts with AMD’s, which emphasizes cache capacity across the chip while delivering very high L3 bandwidth. AMD’s Genoa-X for example can give a single core 90 GB/s of L3 bandwidth, and only drops that to 85 GB/s when loading all cores in the die. Dividing Genoa-X (and other AMD server chips) into octa-core clusters means that each L3 instance only has to deliver high bandwidth to eight cores. Genoa-X’s 1248 MB of L2+L3 capacity could also be advantageous for multithreaded applications that work primarily on thread-private data, or when running many different jobs in parallel. However, a workload that shares a lot of data between threads could end up using that cache capacity inefficiently because shared data may be duplicated across multiple L3 instances. Finally, a single thread can only allocate into its 96 MB L3 instance on Genoa-X, while Vera can theoretically let one thread use its entire 164 MB L3.
NVLink C2C acts as a point-to-point link between sockets, and lets Vera scale to dual socket configurations. A dual socket Vera system has two NUMA nodes, each of which represents a socket and its directly attached memory. Cross-node memory accesses have 309.27 ns of load-to-use latency, which is 177.41 ns more than a local memory access. Achieving that 309.27 ns figure curiously requires generating a few dozen GB/s of bandwidth on the link. A pointer chasing test run in isolation measures over 370 ns of latency, suggesting that NVLink C2C only transitions to a higher performance state if there’s enough bandwidth demand and requests start backing up in front of it.
NVLink C2C delivers very high bandwidth for a cross-socket link. Reads achieve just over 587 GB/s, while a read-modify-write pattern that moves data in both directions can reach 780 GB/s. Vera not only has more cross-socket bandwidth than any other dual-socket configuration I’m aware of, but also has more cross-socket bandwidth than most servers have bandwidth to directly attached memory. However, NVLink C2C falls behind in latency terms. AMD’s Genoa-X for comparison has 222.7 ns load-to-use latency for cross-socket memory accesses. Vera also doesn’t maintain the same tight control over NVLink C2C latency that it does for local DRAM accesses. Latency spikes at high bandwidth demand, and funnily enough, reach 780 ns when using a read-modify-write pattern to achieve 780 GB/s of cross-socket bandwidth.
Interconnects have to present a coherent view of memory by making a core’s writes globally visible, even if written data may only be present in that core’s private cache. A “core to core” latency test bounces a value between two threads using atomic operations, and measures an average of 148.5 ns on core pairs in the same socket. Latencies range from 87.5 to 199.3 depending on the tested core pair, and on which agent owns the tested cache line. SMT siblings have less than 10 ns of latency, indicating that a core can maintain coherency across its SMT threads without involving the slower cross-core interconnect.
Latency between cores on different sockets is approximately 660 ns, which is unfortunately high. I remember Anandtech measured similar figures in a dual-socket Ampere Altra system, but latency above 600 ns isn’t typical.
AMD’s Genoa-X (EPYC 9V33X) achieves much better cross-socket core to core latency, at roughly 209 ns. Amazon’s Graviton 4 is even better, with cross-socket latency on par with older, lower core count designs.
Vera’s intra-socket latency is consistent, matching an advantage mentioned in Nvidia’s whitepaper. However, that latency is also consistently high, and worse than on Nvidia’s older Grace chip. It’s also worse on average than Amazon’s Graviton 4 and AMD’s Genoa-X.
SPEC’s CPU rate suite can be used to measure throughput by running multiple copies of each workload. When running one copy per hardware thread, Vera has no trouble beating Graviton 5 node-for-node. Unsurprisingly, Vera also beats a lower core count Zen 5 part that was curiously configured to have one NUMA node spanning both sockets.
However, Vera falls behind Graviton 5 at the socket level. It’s not a completely fair comparison because Graviton 5 can’t expose the entire socket as a single NUMA node, but it does show a tradeoff between core count and having more powerful cores. A density optimized design with less powerful cores can often provide higher throughput, assuming there’s work available to bring all those cores into play. SMT helps close the gap, but it’s still hard for Vera to take on a design with more than twice as many cores.
Vera still has an advantage even against Graviton 5 in a socket-for-socket comparison in workloads with high memory bandwidth demands. 749.fotonik3d tends to be very heavy on DRAM bandwidth even when running a single copy on the Ryzen 7 9800X3D.
Performance monitoring data wasn’t collected from the socket-wide Vera and Graviton 5 runs, but it’s safe to assume 749.fotonik3d is DRAM bandwidth bound for both of those chips. Running multiple copies will mean less cache capacity on average for each copy, and bandwidth demands from each copy will stack. Despite having a much lower core count than Graviton 5, Vera achieves nearly a 20% performance lead over Graviton 5 in 749.fotonik3d. That advantage extends to several other SPEC CPu2026 floating point tests. On the flip side, Graviton 5 has an overwhelming advantage in more cache friendly workloads.
Vera is a very strong contender in today’s server landscape, with exceptionally strong per-core performance backed by tremendous memory bandwidth. It’s a far more convincing offering than Nvidia’s older Grace chip, which suffered from having small 1 MB L2 caches and very high L2 miss latency. Compared to other server chips from Arm, and even AMD and Intel, Vera trades core count for per-core performance. In use cases where individual tasks need to be finished quickly and workloads may not fill all cores, Vera will be hard to beat.
For the Arm server landscape as a whole, Vera’s entrance brings a welcome increase in diversity. When Arm first gained a foothold in the server market, Arm chips competed purely on density. Cores like Neoverse N1 couldn’t match their AMD or Intel counterparts head-to-head, so chips like Ampere Altra competed by offering higher core counts. Arm server chips today still compete on that density focused front with chips like AmpereOne, but chips based on Arm’s Neoverse V series cores have expanded Arm’s boundaries. Vera pushes that boundary even further in the per-core and per-thread performance direction. More diversity and more competition in the server scene is always a good thing, and Vera’s strength is a sign of more to come.
2026-10-06 06:49:26
Server chips traditionally offered less single threaded performance than contemporary client parts. Higher core counts translate to less power per core, and the complex interconnect required to connect many cores often suffer from higher latency as well. However, that trend has been narrowing. AMD’s recent server chips have been pushing higher frequencies over the last couple generations. NVIDIA’s Olympus is another aggressive move to push server single threaded performance forward, but approaches the problem from a different angle by focusing more on performance per clock.
To deliver on that front, Olympus takes a similar approach to Arm’s Cortex X925. Growing core structures tends to run into diminishing returns, so Olympus diverges from Arm’s approach with a simultaneous multi-threading (SMT) implementation. The result is a monster core that is a force to be reckoned with in the server scene, and one that gets very close to desktop performance.
Olympus is a 10-wide out-of-order core with almost comically large out-of-order structures depending on where you look. It runs at a modest 3.3 GHz clock speed, which isn’t high these days even by server standards, so Olympus needs very strong per-cycle performance to meet NVIDIA’s performance goals.
At a high level, Olympus’s general layout has a lot in common with Arm’s Cortex X925. Both cores use a semi-distributed scheduler with a similar execution unit layout. Both aim to deliver high performance without having to reach the clock speeds that AMD and Intel’s cores do. However, Olympus takes that goal even further by increasing out-of-order structure sizes compared to X925. NVIDIA’s core also differs in a number of areas, like having a non-scheduling queue in front of the floating point schedulers.
A wide core with deep reordering capacity has a lot to lose from branch mispredicts, so Olympus needs a strong branch predictor. In a test with various numbers of branches that are taken/not-taken with different pattern lengths, Olympus stops just short of Arm’s X925. X925’s direction predictor can track approximately 16-24K global history patterns with a few branches in play, or over 64K patterns with 512 branches. Olympus can get to roughly 5-6K patterns with a couple branches, or 48K with 512 branches.
In SPEC CPU2026, prediction accuracy is right up there with the best of Intel and AMD’s cores. Olympus overall is slightly behind AMD’s Zen 5, and slightly ahead of Lion Cove. Each core has individual wins and losses on different workloads, highlighting the difficulty of optimizing a branch predictor to cater for a wide range of program behaviors.
SPEC CPU2026’s floating point suite is generally easier on branch predictors, but still includes a couple of workloads with slight branch prediction challenges. Olympus comes out ahead of Intel and AMD in the floating point suite, mostly because of a large win in 731.astcenc.
Branch prediction speed is also crucial to feeding a wide, high throughput design because taken branches are quite common in code. Like AMD’s Zen 5, Olympus can do two taken branches per cycle across a large branch footprint. Unlike Zen 5 and Cortex X925, Olympus’s ability to do two taken branches per cycle is tied to instruction footprint rather than branch count, which could mean it’s tied to the instruction cache. Instruction footprints beyond 48 KB break the two branches per cycle capability. For those larger instruction or branch footprints, Olympus has a 16K entry BTB with 4 cycle latency.
NVIDIA partitions the branch target caching structures to give each thread about half of available capacity when the core is running two threads. The two threads cannot share targets, even if they belong to the same process and therefore share an address space. Sustained taken branch throughput remains at two branches per cycle from the fastest BTB level. When servicing two threads, the 16K entry L2 BTB can deliver a taken branch target every five cycles from each thread’s point of view, or roughly every 2-3 cycles from the core’s point of view.
NVIDIA and AMD both use ahead indexing to improve branch predictor throughput, meaning that one lookup provides predictions for the next two branches rather than a single branch. NVIDIA implies they can do this for their direction predictor and not just the BTB, letting them carry out more predictions per cycle for branches with dynamic behavior.
Olympus implements advanced predictors with robust ahead pipelining mechanisms delivering up to 2.3x higher predictions per cycle vs the competition
-Vera Whitepaper
Predicting with BTB pairs allows two fetches to be predicted in one prediction cycle
-AMD’s Zen 5 Optimization Guide
I tried setting up a test with conditional branches that were not-taken 1/8 of the time in a trivially predictable manner. I was hoping that would force the core to use the conditional predictor to at least validate what comes out of the BTB via ahead indexing. But if that causes an impact, I can’t see it in either Zen 5 or Olympus. Both cores can manage two taken branches per cycle if they have to use the branch predictor. I tried making the branches taken/not-taken in more complex patterns, but stopped putting more time into after the first couple attempts caused mispredicts on both cores.
NVIDIA further noted that Olympus could sustain more predictions per cycle compared to the competition, and cited specific SPEC CPU2026 integer workloads.
Validating this is difficult because neither architecture provides hardware counters to track how many predictions they made. I also don’t think predictions per cycle is a useful metric by itself, because predictions made after a mispredict are useless and more predictions per cycle may not impact performance if the frontend is already adequately fed. Another factor is the core’s design goals around performance per cycle rather than clock speed. Olympus may be averaging more predictions per cycle, but that could be because it’s moving along more of the instruction stream in a given number of cycles compared to Zen 5. Zen 5 in contrast is designed to clock high while still being able to sustain two taken branches per cycle, and the rate at which the two designs complete branches isn’t too far apart when clock speed is taken into account.
Fetch addresses from the branch predictor go through address translation with a 64 entry fully associative instruction TLB. Olympus then fetches instructions from a 64 KB, 4-way set associative instruction cache that can deliver 128 bytes per cycle into a 48 instruction decode queue. A 128 byte fetch can contain up to 32 instructions, which far exceeds the core’s decode throughput and possibly serves to let the frontend catch up if it meets a straight run of instructions after a long latency branch or mispredict.
Larger instruction footprints are handled well, with the core able to fetch 32B/cycle of instructions from the L2 cache. Per-cycle L2 code throughput for a single thread is much higher on Olympus than on Zen 5 or Cortex X925. Like many other cores, code read throughput drops sharply if instructions spill into L3.
Running two threads in the core causes a sharp drop in instruction fetch bandwidth before 1 MB, indicating that Olympus is statically partitioning the L2 cache. This applies even if the sibling thread is running a trivial loop that consists of several instructions, and doesn’t make any memory accesses that could cause data-side cache capacity contention. Throughput for a high IPC thread (NOPs only) is also cut in half even if the other thread is latency bound and stuck at low IPC.
Micro-ops from decoded instructions need to go through register renaming and have appropriate tracking resources allocated in the backend. This process provides opportunities to carry out other optimizations like move elimination. Olympus can do move elimination to a limited extent. A chain of dependent MOVs executes at just over 1 per cycle, indicating that some MOVs executed with zero latency. AMD’s Zen 5 can impressively do this across 5 MOVs per cycle.
If the core runs two threads, Olympus’s per-thread move elimination capability strangely gets better. Per-thread dependent MOV throughput increases to 1.7 MOVs/cycle per thread, or 3.5 across both threads. Zen 5 improves as well, with per-core throughput increasing to 7 MOVs per cycle. However, that represents a drop in per-thread throughput from 5 to 3.5 MOVs/cycle.
Olympus’s backend has massive register files and queues just about everywhere. It’s a bit like Cortex X925, but with many things taken up a notch. The only structure with a somewhat lower entry count is the FP/vector register file, which has fewer entries than Zen 5’s. However, Olympus compensates by often letting a 128-bit vector register hold a pair of scalar FP values written by separate instructions. It’s a useful optimization because a lot of code cannot be vectorized.
Testing with NOPs puts a very high ceiling on the number of instructions that Olympus can have in flight. The core can overlap two long latency loads with more than 1000 NOPs spaced between them, but the practical number of in-flight instructions is likely to be significantly lower because Olympus can only have a bit over 606 registers allocated. The ~606 allocated register limit applies on the architectural register side, because using the scalar FP optimization mentioned above still results in the number of allocated architectural registers capping out at 606.
Olympus’s schedulers are laid out similarly to those on Arm’s Cortex X925. Eight integer ALUs are arranged into pairs, with each pair fed by a scheduling queue with approximately 25 entries. One of the ALU pairs is special and can handle less common operations. Olympus and Cortex X925 both use a six-pipe FPU, again with a scheduling queue for each pair of pipes. However, NVIDIA uses a non-scheduling queue in front of smaller scheduling queues, while Arm uses three very large scheduling queues.
All of the backend structures I checked are statically partitioned, meaning that each thread gets half (or less) of a structure’s capacity when the sibling thread is not idle. That applies even to scheduling queues, which are often watermarked or competitively shared on other architectures. Static partitioning even extends to the execution units, with execution throughput cut in half regardless of whether the sibling thread is running instructions that hit the same ports. AMD’s SMT implementation for comparison flexibly shares most structures, though it sets limits (watermarking) to ensure one thread cannot unfairly monopolize a resource.
Olympus executes memory accesses down a set of four pipelines, all of which can handle loads, and two of which can handle stores. The core’s L1 data cache has a nominal latency of four cycles, but a simple memory latency test can measure 2-3 cycles of L1D latency possibly because of value prediction.
Memory disambiguation behavior on Olympus has more in common with older cores from Arm or Intel’s Atom line. Olympus can do fast forwarding from a 64-bit store to a dependent 32-bit load as long as the load is aligned to either half of the store. Forwarding latency is 3-4 cycles for exact address matches, or 5-6 cycles when forwarding the upper half. All other overlap cases lead to a 12-13 cycle penalty, likely because the core blocks the load and waits for the store to commit.
Olympus encounters a strange hiccup when doing store forwarding across the 32B boundary in the middle of a 64B cacheline. It strangely doesn’t hiccup in the same way when accesses cross a 64B cacheline boundary, even though every 64B aligned boundary is of course also 32B aligned. Also, independent loads and stores that touch the same 32-bit block at the end of the first 32B half of a cacheline sometimes aren’t allowed to proceed in parallel.
Wider vector accesses show the same strange behavior at the middle 32B boundary, and show an additional hiccup at the first 16B aligned boundary into the cacheline. Latency for vector store forwarding is higher, which is typical for many cores.
For address translation, Olympus has a large 112 entry fully associative data TLB, backed by the unified 3K entry L2 TLB. Zen 5 for comparison has a 96 entry fully associative data TLB backed by a 4K entry L2 data TLB. AMD doesn’t use a unified L2 TLB, and stores instruction-side translations in a separate 2048 entry L2 instruction TLB. NVIDIA ran a kernel with 64 KB pages on the system we had access to, which extends TLB reach compared to using typical 4 KB pages. 64 KB pages basically meant that address translation latency was a non-factor for a lot of testing, but made direct comparisons difficult.
Large out-of-order structures, prefetchers, branch prediction, and value prediction help a core tolerate memory access latency. Caches attack the problem from the other side by cutting memory latency for frequently accessed addresses. Vera’s caching strategy is greatly improved compared to NVIDIA’s prior Grace CPU. Olympus’s data memory hierarchy starts with a large 96 KB 6-way set associative data cache, which offers more capacity than the 32, 48, or 64 KB data caches often found on AMD, Arm, and Intel’s server cores.
L1 misses can be caught by a 2 MB 8-way set associative L2, which delivers 10-cycle load-to-use latency. The low cycle count latency means that Olympus’s L2 can deliver data nearly as fast as the Cortex X925’s 12-cycle L2, despite the X925’s higher 4 GHz clock speed. Having 2 MB of L2 capacity is a welcome improvement compared to Grace’s Neoverse V2 cores, which had to make do with 1 MB L2 caches. Vera’s L3 latency is comparable to Grace’s and isn’t great in absolute terms at over 120 cycles. However, that’s typical of designs that implement a L3 cache over a massive monolithic interconnect. Vera at least increases L3 capacity from 114 to 164 MB.
NVIDIA covers for high L3 latency with aggressive prefetching. A linear read pattern from a single thread achieves 88.53 GB/s from L3, and applying Little’s Law indicates that Olympus was able to sustain 50-51 outstanding requests to L3 (88.53 GB/s /(64B lines * 3.3 GHz) = 0.42 cachelines per cycle, multiplied by 121.5 cycle latency = 50.9 requests in flight). Doing the same calculation for Grace/GH200 gives 35-36 requests in flight. As with many other cores, Olympus likely has a deeper queue for L2 misses than L1 misses. Testing with Lemire’s memory level parallelism microbenchmark shows that Olympus can have 18-19 demand L1D misses in flight. Lemire’s test uses a randomized pointer chasing pattern that won’t work with L2 prefetchers, and therefore won’t take advantage of a larger L2 miss queue.
Olympus’s SMT implementation partitions the L1D and L2 caches for data access too, meaning that each thread gets half the capacity of all core-private caches if the sibling thread is active. As with the instruction side, this partitioning applies even if the sibling thread makes no data-side accesses.
SPEC CPU2026 scores show Olympus delivering very strong single threaded performance for a server core. It isn’t far off Intel and AMD’s latest cores in desktop settings, represented here with the Core Ultra 9 285K and Ryzen 7 9800X3D. Against Arm server competitors like Neoverse V2 and Neoverse N2, Olympus has no trouble pulling off a significant lead. A significant problem here is that the kernel on the test system we had access to uses 64 KB pages, while all other results are obtained with 4 KB pages. Estimating page size impact will be difficult, but I expect Olympus will still compare well against its Neoverse counterparts even with a smaller page size.
Examining individual scores show NVIDIA doing well across many integer workloads including the ones highlighted in their whitepaper (723.llvm_r, 727.cppcheck_r, 721.gcc_r, 714.cpython_r). Olympus’s large loss to Zen 5 in 706.stockfish_r has more to do with instruction set differences than core architecture, because Olympus had to execute roughly twice as many instructions as Zen 5 did in that workload. ISA differences also affect 772.marian_r on the floating point side. Instruction count can go the other way too, with aarch64 requiring fewer instructions for 750.sealcrypto_r, but Zen 5 is able to maintain a strong performance in that workload thanks to its overall higher throughput design when taking clock speed into account.
Olympus’s goal is to achieve high single threaded performance by emphasizing performance per clock, and performance counters unsurprisingly indicate very high IPC. NVIDIA’s IPC is often higher than Zen 5’s, even in workloads where Zen 5 manages to deliver higher performance.
Wider pipelines are difficult to feed, and checking top-down performance events reveals that nearly all workloads leave over 50% of dispatch slots unused on average. SPEC CPU2026’s integer workloads lose a lot of throughput because the frontend couldn’t supply micro-ops fast enough (frontend bound), showing that even a fast two-ahead branch predictor and large instruction cache can’t always keep up. The same is true of Zen 5 as well.
SPEC’s floating point workloads tend to be more backend bound, meaning that micro-ops couldn’t dispatch because a backend resource was full. In other words, the core reached a limiting factor for how many instructions it could keep in flight.
SMT is a way to claw back unused core width and more efficiently utilize backend resources when a second thread is available. NVIDIA’s “spatial multithreading” takes a rather unconventional approach to SMT, and definitely shows promise when looking at SPEC CPU2026’s integer workloads. There, Olympus’s SMT gains are on par with Zen 5’s. Zen 5 is ahead in actual performance, but I’m focusing on gains here because again a client CPU is expected to maximize per-core performance.
Olympus’s gains are less convincing on the floating point side, though SPEC CPU2026’s floating point suite still enjoys a significant overall uplift.
Individual workloads show varied gains, with Zen 5 or Olympus getting more out of SMT on different subtests compared to the other. However, the general pattern is the same for both cores. Low IPC, frontend bound workloads like 723.llvm tend to see massive gains from SMT. Higher IPC, throughput bound workloads like 750.sealcrypto and 714.cpython don’t gain as much.
SPEC CPU2026’s floating point suite tends to be more backend bound, and Olympus’s SMT implementation surprisingly suffers throughput losses on 709.cactus and 782.lbm. SMT implementations ideally don’t lose overall throughput when using both threads, so Olympus’s spatial multithreading doesn’t seem ideal in those two workloads. Zen 5 also gains less from SMT in SPEC’s floating point workloads, but manages to avoid situations where using both SMT threads would be worse than running two tasks in serial.
SPEC CPU2017 remains an interesting suite because it features difficult workloads without analogues in SPEC CPU2026. I also have more SPEC CPU2017 data on hand, including from systems I no longer have access to. That data shows Olympus beating several more server cores, including a Zen 5 server implementation. It also shows Olympus drawing even with AMD’s top end Zen 4 desktop chip, which is an impressive feat for a server core.
SMT gains in SPEC CPU2017 are even higher than in SPEC CPU2026, thanks to workloads that wreak havoc on branch predictors and suffer from backend memory latency. 505.mcf comes to mind.
Olympus again takes a few concerning SMT losses in the floating point suite. AMD’s Zen 4 isn’t immune, but its losses are comparatively light. A 3% or 1% performance decrease is far from ideal, but is likely less noticeable than 11.98% or 9.78% losses.
Explaining Olympus’s SMT behavior is difficult, because analyzing SMT test cases is harder than looking at single threaded ones. 709.cactus’s situation can be explained by L2 data MPKI increasing from 3.58 to 6.52 when the core is running two copies of the workload in its SMT threads. Then, the partitioned resources in the backend may be insufficient to deal with high L3 latency.
However, I can’t apply the same explanation for SPEC CPU2017’s 503.bwaves and 549.fotonik3d. 503.bwaves thrashes the L2 in both cases, with 32.12 MPKI for a single thread and 33.03 MPKI when running two threads. Top-down events indicate performance losses for non-memory bound reasons. 549.fotonik3d also suffers a lot of L2 misses (52.45 MPKI for a single thread, 57.13 for two threads), but again the relative difference isn’t too big. In general, Olympus’s spatial multithreading seems to deliver strong gains on par with a conventional SMT implementation for frontend bound workloads, but doesn’t handle as well with backend bound ones.
NVIDIA’s spatial multithreading is less like a conventional SMT implementation, and more like dynamically splitting Olympus into two smaller cores when it’s running two threads. Just about every imaginable core resource down to the L2 cache is statically partitioned, so each thread can only access half of what’s available. This approach certainly eases performance analysis, because viewing the two SMT threads as two 5-wide cores is simpler than thinking about how two threads might dynamically use various core resources in, say, AMD’s Zen 5. I imagine it’s easier on the core design front too. NVIDIA doesn’t have to tune complicated mechanisms that ensure thread fairness because splitting everything in half already gives a fairness guarantee.
I think the jury is still out on whether spatial multithreading is a good idea or a simplification meant to ease NVIDIA’s first generation SMT implementation. Spatial multithreading shows promise by delivering gains on par with conventional SMT implementations in SPEC CPU’s integer workloads. In the best cases, Olympus demonstrates that how efficiently you can let two threads share resources might not matter if your core resources are sufficiently massive in the first place. However, gains can be inconsistent and even negative in backend-bound scenarios in SPEC’s floating point tests. It’s hard to tell whether those rough spots come from teething issues in a first generation SMT implementation, or from fundamental inefficiencies that arise from locking each thread into half of a core. I imagine a certain portion of SMT gains come from dynamically letting each thread dip into resources left under-utilized by its sibling, and limiting each thread’s ability to do that could negatively impact SMT gains.
As with any architectural feature, it’s probably best to wait and see if NVIDIA continues the same strategy. Successful architectural techniques tend to stay around for multiple generations and gain adoption across the industry. Whether that’ll happen with spatial multithreading is anyone’s guess.
Arm’s early server chips emphasized core count over per-core performance, but the Arm server scene has diversified as newer Arm cores took aim at higher performance targets. Now, NVIDIA’s Vera delivers top-notch single threaded performance and turns the situation around. NVIDIA is now the one with a lower core count (though 88 cores is still a lot in an absolute sense), and AMD’s server chips have more cores with less single threaded performance. There is some wiggle room in the SPEC results because of different page sizes, but I suspect that would only narrow the gap between Olympus and the fastest AMD server cores rather than changing their relative rankings.
Against its Arm Neoverse competition, Olympus dominates on per-core throughput terms thanks to both its high single threaded performance and its ability to increase throughput by running a second thread. Debating the merits of a conventional SMT approach versus Olympus’s spatial multithreading one doesn’t matter when Arm cores can’t simultaneously run a second thread at all. Olympus gains throughput from running a second thread in the vast majority of cases, and any throughput it gains widens the per-core performance gap between it and its Neoverse contemporaries. NVIDIA’s prior Grace chip is included in that mix. Vera is a huge step up from Grace, with both faster cores and more cores.
Olympus overall is a very exciting server core. Its performance shows the merits of a low frequency design that aims to maximize work done per clock cycle, and definitely pushes boundaries for single-threaded performance in a server. Other server core designers should take note, because performance in low-threaded workloads continues to be important even as server core counts continue to increase. I look forward to seeing NVIDIA further evolve this impressive core.
2026-09-30 00:07:30
Laptop chips have gotten very powerful in recent years, with increasing CPU core counts, more powerful iGPUs, and higher DRAM bandwidth to feed that compute. All of that creates pressure on the chip’s system level architecture, which must deliver high bandwidth, service an increasing number of blocks, and keep latency under control. Qualcomm’s Snapdragon X2 Elite Extreme is a good example of how powerful recent laptop chips have become, with 18 CPU cores and an iGPU packing nearly as much compute as a Nvidia GeForce GTX 1070 Ti. It’s an aggressive entry into the laptop market, and needs a strong interconnect design to support everything packed into it.
Qualcomm simplifies the chip-level interconnect by splitting CPU cores into clusters, much like AMD. All cores within a cluster can be treated as a single client of the system level interconnect, reducing the number of clients that interconnect has to deal with. A shared cache within a cluster only has to concern itself with that cluster’s cores, simplifying cache design and making it easier to ensure fast cache access. Qualcomm uses 6-core clusters on the Snapdragon X2 Elite Extreme, compared to quad core clusters on the older Snapdragon X Elite. The first cluster contains E-Cores, or “Performance” cores in Qualcomm parlance, and has 12 MB of shared L2 cache. The two other clusters contain P-Cores, or “Prime” cores, and have 16 MB of shared L2 cache each.
Qualcomm employs a caching strategy with parallels to the one used in Apple’s M1. Each cluster’s L2 is designed to provide very low latency and high bandwidth, while each CPU core comes with relatively large L1 caches. This combination lets Qualcomm dispense with the mid-level caches often found on AMD, Arm, and Intel designs. To deliver the required performance, Qualcomm runs the L2 caches at core clocks and tightly couples them to the cores. L2 latency comes out to 20-21 cycles on both the P-Core and E-Core clusters.
Curiously, a couple cores within each P-Core cluster have slightly higher latency at 22-23 cycles, while all E-Cores have the same L2 latency. I suspect Qualcomm uses a centralized crossbar design for their L2. The high 5 GHz might mean two P-Cores need a couple extra pipeline stages to get to L2 while meeting cycle time requirements. Perhaps the E-Cores didn’t need to make that concession because of their lower 3.6 GHz clock speed.
L2 bandwidth is 32B/cycle in both the read and write direction, so each core can achieve 64B/cycle of off-core bandwidth with a read-modify-write pattern. This is true even in the E-Core cluster, even though the load/store unit on Qualcomm’s E-Cores can only handle two 16 byte accesses per cycle. With a test where an E-Core only accesses half of each 64B cacheline’s data, it’s possible to infer that the interface to L2 can sustain 64B/cycle of fills and writebacks.
Bandwidth scales well with increasing active core counts. Each P-Core is able to get more than 100 GB/s of read bandwidth on average when hitting all cores in a cluster with a bandwidth test. AMD does come out ahead with a read-modify-write pattern, but Qualcomm should have more than enough shared cache bandwidth on hand for the vast majority of use cases.
Going outside the cluster reveals an impressive level of external bandwidth. Each cluster appears to have a 32B/cycle path to the system, but cycle here is a cycle at the cluster’s clock. AMD’s Strix Halo for comparison has 32B/cycle read and write paths to the system, but that’s running at the chip’s 2 GHz Infinity Fabric clock (FCLK). The result is that Qualcomm’s CPU core clusters enjoy higher DRAM read bandwidth than Strix Halo clusters.
AMD again does well with a read-modify-write pattern that exercises both of the cluster’s external 32B/cycle paths. However, most applications will make far more reads than writes, so Qualcomm’s ability to get higher bandwidth with reads alone is a notable advantage.
DRAM latency on the Snapdragon X2 Elite Extreme is rather good for a LPDDR5X setup at 115 ns. It can’t compare with the sub-100 ns latencies typical on desktop DDR5 setups, but it’s up there with the best LPDDR5(X) implementations. As CPU bandwidth demands increase, Qualcomm’s system fabric maintains excellent control over latency and doesn’t let bandwidth-hungry threads excessively penalize a latency sensitive one. CPU read bandwidth in absolute terms is impressive too, surpassing even chips with 256-bit memory buses. AMD’s Strix Halo and Nvidia’s GB10 have 256 GB/s and 273 GB/s of theoretical DRAM bandwidth, respectively. The Snapdragon X2 Elite Extreme has 228 GB/s of DRAM bandwidth with a 192-bit bus. However, AMD and Nvidia’s CPU clusters don’t have enough external bandwidth to fully utilize their beefier DRAM setups.
Switching to a read-modify-write pattern lets Strix Halo’s CPU cores utilize more DRAM bandwidth, but hurts Qualcomm. Mixing reads and writes can challenge the memory controller, because the DDR memory bus can only be in read or write mode at any given time. Switching the bus between read and write mode (a turnaround) wastes bus cycles and therefore reduces achievable bandwidth. Perhaps Qualcomm suffers more from that, but getting to 68.5% of theoretical DRAM bandwidth is still a good result.
Cluster designs can experience contention at cluster boundaries, but that’s very well controlled on Qualcomm’s design. Latency stays under 200 ns from a CPU core regardless of how much DRAM bandwidth its peers are pulling through the off-cluster interface. It’s a better showing than some of AMD’s designs, which could let bandwidth-hungry threads squeeze out a latency sensitive one.
Adding in bandwidth demands from other clusters pushes latency up, but it never reaches unacceptable levels.
Qualcomm’s performance in this testing also compares well to Nvidia’s GB10, which could let latency creep over 200 ns when X925 cores contend with each other for off-cluster bandwidth.
Interconnects have to maintain cache coherency by ensuring that a write from one core can be made visible to another. A “core-to-core” latency test evaluates how long it takes for one core to observe another’s write using compare and exchange operations. Qualcomm probably handles transfers within a cluster through the L2 cache, which maintains an inclusive relationship with core-private caches within the cluster. Because the L2 includes L1 contents, it can easily tell when it might need to probe another core. Intra-cluster transfers enjoy very low latency, which is up there with the best of what Intel or AMD’s designs can achieve.
Cross-cluster accesses go through Qualcomm’s system level fabric and incur higher latency. However, that latency still remains well controlled and is similar to cross–cluster latency on AMD’s client designs. Latency to the E-Core cluster is somewhat higher than between the two P-Core clusters.
In all cases, Qualcomm’s newest design is a huge improvement over the older Snapdragon X Elite. Core clustering still means there’s a cross-cluster latency penalty, but it’s now much better controlled than on Qualcomm’s previous design.
Intel, and more recently Nvidia, have criticized cross-cluster penalties in their marketing publications. However, everything is a tradeoff, and focusing on uneven latencies misses the benefits of clustering. Clusters let Qualcomm deliver excellent shared cache latency and low core-to-core latency within a cluster, and that’s true of AMD’s designs too. I doubt Qualcomm could deliver L2 latencies in the low 20 cycle range if L2 accesses have to traverse an interconnect servicing 18 cores. Similarly, sub-20 ns core-to-core latencies would be hard to achieve with larger, logically monolithic interconnects.
I think Qualcomm’s clustering approach strikes a good balance. Worst case latencies aren’t much higher than average latency on a high core count, mesh-based server chip. Trying to build a big monolithic interconnect risks getting consistent core-to-core latency at the cost of making that core-to-core latency consistently bad. A monolithic interconnect would also impact cache performance for typical accesses that don’t need cross-cache transfers.
GPUs can be huge bandwidth consumers on modern laptop chips. When mixing CPU and GPU bandwidth demands, the Snapdragon X2 Elite Extreme can let a CPU thread’s DRAM latency reach nearly 350 ns. It’s not too bad though. Worst case latency is on the same level as Meteor Lake, and achieved bandwidth is much higher than what a 128-bit LPDDR5(X) setup can hope to deliver.
CPU+GPU bandwidth doesn’t approach theoretical DRAM bandwidth as closely as it does on other chips. I achieved the highest bandwidth in this test run with the GPU using 74.86 GB/s and the CPU pulling 94.78 GB/s, for a total of 169.335 GB/s. That’s 74.2% of theoretical, and just short of the 176.98 GB/s (77.6% of theoretical) achievable with a CPU-only bandwidth test. For comparison, AMD's Strix Halo and Nvidia's GB10 can achieve 226.08 GB/s and 243.05 GB/s, or 88% and 81% of theoretical respectively.
Unlike Nvidia’s GB10 and AMD’s Strix Halo, the Snapdragon X2 Elite Extreme doesn’t let the GPU squeeze out the CPU. If I let the CPU and GPU grab as much bandwidth as they can get their hands on (very little compute relative to memory accesses), Qualcomm gives the CPU and GPU each about half of available bandwidth. Decreasing GPU bandwidth demands lets the CPU pick up whatever’s left. GB10 and Strix Halo can both give the CPU more bandwidth if the GPU isn’t asking for too much. However, they leave DRAM bandwidth on the table because their interconnects can’t deliver the same level of bandwidth to the CPU side.
Qualcomm’s bandwidth patterns under this test are closest to Meteor Lake’s, though the Core Ultra 7 155H has much lower bandwidth in general. It’s an older chip, and its 128-bit LPDDR5X-7467 configuration can’t hope to compete with a faster 192-bit one.
High CPU core counts and large iGPUs can challenge interconnect design, but Qualcomm’s interconnect is more than up to the task. The Snapdragon X2 Elite ensures its CPU cores can enjoy low latency and high bandwidth access to shared caches thanks to its clustered setup. Then, Qualcomm’s system level fabric offers a compelling combination of high bandwidth and well-controlled latency. It compares well against other mainstream laptop chips, though admittedly I only have Intel’s slightly dated Meteor Lake to compare against. More impressively, it beats out AMD’s Strix Halo and Nvidia’s GB10 in terms of CPU-side bandwidth. AMD’s Strix Halo and Nvidia’s GB10 both have larger 256-bit DRAM buses, but they can feel like large GPUs that happen to have a pile of CPU cores integrated. Qualcomm’s Snapdragon X2 Elite doesn’t give off the same vibes, and treats its CPU cores as first class citizens.
Qualcomm’s excellent interconnect performance and high bandwidth DRAM setup speak to the company’s ambitions. I remember when years ago, Gerard Williams III (who no longer works at Qualcomm today) mentioned at Hot Chips that architectural features can be built with a vision to enable advances in future designs, even if those features don’t deliver a huge benefit for a current product. Back then, he was talking about CPU core design, but the same principle can apply at the chip level too. The Snapdragon X2 Elite’s 192-bit LPDDR5X setup and system level fabric borders on being over-engineered considering its CPU and GPU setup. CPU cores typically hit cache for the vast majority of memory accesses provided the cache design is competent, so I expect typical CPU-side bandwidth demands to be well within what a 128-bit DRAM bus can deliver. GPU workloads often demand high sustained bandwidth, but Adreno X2 isn’t sized to deliver the level of performance that Strix Halo or GB10’s iGPUs can, and can’t consume the same level of bandwidth.
The Snapdragon X2 Elite Extreme’s system level design is something of a warning to Intel and AMD. For now, AMD’s Strix Halo and Intel’s Panther Lake stand out as halo-segment mobile chips. But no one designs an interconnect and memory subsystem as strong as the one in the Snapdragon X2 Elite if they haven’t at least thought about building a monster to take on AMD and Intel’s halo-segment products. With that in mind, I’m excited to see what Qualcomm comes up with next, on both the CPU and GPU side.