The IEEE is warning that China’s export controls on rare earth minerals could start to impact datacenters, as shipments of some key materials are down 75 percent and Beijing is due to review its policies this November. The techie org's official publication, IEEE Spectrum, focuses on two rare earths in particular, erbium and yttrium, which are used in components for IT equipment. Exports of yttrium are down by about 75 percent this year in comparison with 2025. Yttrium was one of seven medium and heavy rare earth elements that China placed under mandatory licensing in its first wave of restrictions in 2025. Yttrium is used in white LED lights and displays, microwave filters, and solid-state lasers, and also in ceramic capacitors found in pretty much all electronic kit. Erbium is used in the amplifier circuits that boost optical signals in fiber-optic links. The Chinese government enacted restrictions on the overseas sale of rare earth minerals last April, as reported by The Register. This was part of retaliatory measures against the Trump administration’s slapdash trade tariffs. It upped the ante later in the year with tougher measures, including laws that require companies to apply for a license before they ship any goods containing rare earths sourced in China to anywhere in the outside world. This latter rule is a mirror of US laws like those used to block the export of ASML chipmaking equipment to China. That second wave of measures – which was to have included increased export controls of five new elements including erbium – was suspended in November 2025 for 12 months, following trade talks between the US and China. When the year is up this November, Beijing will decide whether to drop the tougher measures or re-apply them, perhaps extending them to all or most of the rare earth minerals it controls. According to the International Energy Agency (IEA), the supply of rare earths remains among the least geographically diversified among all critical minerals. For those minerals used in magnets, China accounted for about 60 percent of global mining output in 2024, and its dominance is even greater in the separation and refining stages, making up about 91 percent of global production. But for yttrium and erbium used in IT kit, China is effectively the sole source, IEEE Spectrum claims. The country mines 90 percent of the world’s yttrium, but it processes all of the world’s mined ore into an industrially usable form such as yttrium oxide, and likewise supplies all erbium oxide. Washington has been taking some steps to try and offset China’s virtual monopoly of key rare earth materials. Recently, the US Department of Defense announced a $750 million investment to secure supplies of rare earth elements for magnets from a production site in central Brazil, while the Department of Energy announced $500 million for seven selected projects to expand critical mineral and material processing, battery manufacturing, and recycling. The US signed agreements with representatives of 54 countries at the 2026 Critical Minerals Ministerial meeting in February this year, as well as talking up commitments to build secure and resilient supply chains for such materials. Also in February, President Trump launched Project Vault, a supply chain security initiative establishing an American Strategic Critical Minerals Reserve stockpile, with funding of up to $10 billion from the federal Export-Import Bank of the United States. But many of these initiatives will take time to have any impact, and China’s impending decision will put these to the test, should it decide to take a tougher stance. Relations between Washingon and Beijing remain strained because of trade and other factors, and a lot could be riding on a summit between President Trump and President Xi Jinping next month. ®

Shots were fired at the Hot Chips conference in California this week as Nvidia announced its new Groq-3-based LPX racks had entered production, with early tests showing the systems churning an eye-watering 3,400 tokens a second in Gemma 4 31B. That's four times faster than rival Cerebras. A day later, Cerebras fired back, touting nearly equivalent performance from its next-gen CS-4 accelerators revealed last week. These top-line performance figures make inference feel instantaneous relative to the chatbots we've grown accustomed to over the past four years. But the two companies are essentially arguing over numbers their customers will probably never see in production. To be clear, neither is lying. If you wanted to recreate these results, you certainly could — Artificial Analysis, the team responsible for both sets of benchmarks, knows what they're doing — but beyond a marketing gimmick, no inference-as-a-service model operator in their right mind would run either system this way. Not unless you've somehow figured out how to make a profit serving one request at a time. In reality, these numbers are more like the top speed on a race car. It makes for great marketing but you probably aren't driving that fast on a regular basis, and if you did, you wouldn't get very far before your tank runs dry. These aren't the numbers that matter Unsurprisingly, the economics of "premium" or "ultra-low latency" inference are a bit more nuanced, but ultimately come down to how efficiently you can scale that performance at the rightmost end of the Pareto frontier. We've shown the graphic below before. It charts the performance characteristics of how different Nvidia B300 configurations perform across a Pareto front. As you can see, GPUs are great for high-throughput, low-interactivity applications, but run out of steam quite quickly as per-user generation rates climb. Chips like Nvidia's Groq-3 LPUs or Cerebras' dinner plate-sized accelerators offer massive bandwidth thanks to their SRAM-heavy architectures, so they can extend and push the Pareto curve out to the right. Unfortunately, the same thing that makes them so fast also limits their ability to scale that performance to large numbers of users. To understand why we need to walk through a little math. We're going to pick on Gemma 4 31B, in part because we have rough numbers for both Nvidia and Cerebras, but also because it's a small model and therefore presents a best-case scenario. Each Groq-3 accelerator is equipped with 500 MB of on-chip memory capacity. Nvidia achieved its 3,400 tok/s per user run at 8-bit precision. So to run a 31 billion-parameter model, we need about 31 GB of memory or about 64 LPUs. But that's just to hold the model's weights. Since it's running as a standalone system, we also have to account for the KV caches generated during the prefill stage in which the prompt is processed. According to LMcache's KV calculator, at 8-bit precision, a 100,000-token input sequence is going to chew up another 8 GB of memory, and that's per sequence. So for every additional 100,000 token input, that's another 16 LPUs worth of memory. By our estimate, that means the most a single LPX rack with its 256 LPUs can manage on its own before running out of memory is a batch size of 12 at a 100,000 token input length each. The numbers look a little different for Cerebras' CS4, but the conclusion is largely unchanged. Each CS4 rack contains three massive WSE-3T accelerators with 44 GB of SRAM apiece, totaling 132 GB of capacity. That leaves the system a bit more headroom, but we're still looking at a maximum batch size of 12. To be clear, that's not a compute limitation. The systems simply don't have enough memory to support larger batches without adding additional racks or cutting the prompt size. Sure, we're oversimplifying a bit. We're assuming batch inference with fixed input sequences, not concurrent users hitting the systems asynchronously, and with different input and output sequences each time. For a chatbot, especially one with a limited context window — think the model's short term memory — a single CS4 or LPX rack could conceivably serve far more users than the 12 noted here. Lifting the curve Even if both systems can hit 3,400 tok/s, the better platform is going to be the one that delivers better scale at a lower cost and power consumption. The benchmark figures touted by Nvidia and Cerebras this week are really measuring the top speed the systems could run on, not the speed customers will actually run them. In fact, customers probably won’t run these systems on their own. These SRAM-heavy architectures work a lot better as decode accelerators, which is exactly why Nvidia spent $20B to license Groq's tech and poach its engineering team. It's also why Cerebras recently teamed up with AWS and AMD. We've discussed these kinds of heterogeneous compute architectures in the past, but in a nutshell they involve GPUs or other compute heavy XPUs, like Amazon's Trainium chips, with something like Groq's LPUs, Cerebras' waferscale accelerators, or SambaNova's SN-series parts. This combination largely eliminates, or at least minimizes, the KV cache and compute overheads that made scaling Cerebras' and Groq's chips so difficult. As decode accelerators, they only really need to hold the model weights, as almost everything else can get on just fine on a GPU. Again, top line AI performance makes for great headlines, but the far right of the Pareto is rarely where you'll want to be. As is usually the case with Pareto curves, the sweet spot usually presents itself as a notable hump or knee pushing toward the upper right corner of the chart. Here you'll be trading some per-user interactivity for better concurrency, throughput, and economics. Combining GPUs with Cerebras or Groq accelerators is what Nvidia, AMD, and AWS are all counting on to make "premium inference" cost-effective. There's a better benchmark for this We've yet to see benchmarks combining Cerebras' or Groq's chips with Nvidia, AMD, or AWS' respective GPUs and XPU platforms, but these are the metrics that are actually going to matter. SemiAnalysis' InferenceX offers a better representation of whether or not adding a CS4 or LPX rack would even help, as we can see exactly how each system performs across the full Pareto, ranging from high-volume low-interactivity "bulk" tokens on the left, to ultra-low-latency but also lower-volume tokens on the right. In fact, while Nvidia and Cerebras opted for Artificial Analysis' simpler test methodology, OpenAI actually opted for InferenceX to show how its spicy new Jalapeño accelerators perform across bulk and premium inference regimes. So why didn't Nvidia and Cerebras do the same? If we had to guess, it's because their LPX and CS4 racks are still best suited toward low-latency inference and wouldn't look as impressive on the left and middle sections of the Pareto curve. We fully expect that both companies will share InferenceX results for their combined GPU and LPX/CS4 systems when they're ready, but no marketer ever got a bonus for making their product look worse than it really is, so we'll just have to wait. In any case, the next time you see someone boasting of some impossibly fast inference accelerator, it's worth asking where on the Pareto that number came from. ®

If you’re going to build a custom AI accelerator, inference is a good place to start. The clusters are smaller, the chips can be simpler, and at the end of the day they only need to do one thing: serve tokens. OpenAI’s spicy new Jalapeño inference chips are just the latest example. Social media magnate Meta is bucking this trend. Its first proper generative AI accelerator, the MTIA 400 — short for Meta Training and Inference Accelerator — is aimed squarely at LLM training. The Facebook parent is no stranger to custom silicon. But, much like Amazon and Google, its first AI accelerators weren’t built to run AI chatbots or train generative AI models. They were built to serve ads. The MTIA 400, teased earlier this year and detailed at the annual Hot Chips semiconductor development conference this week, will perform some inference duties. Just not of the GenAI variety. Instead, it’ll be saddled with running the ad recommender systems that actually pay Meta’s bills. The combination of LLM training and the deep learning recommender model (DLRM) inference used for serving ads is unusual, as the two have wildly different performance demands. LLM training is enormously compute-intensive, often requiring tens or even hundreds of thousands of accelerators to train models in a reasonable amount of time. DLRM inference, on the other hand, is a predominantly memory-bound job, which means most of the FLOPS that make the chip good at training are going to be left sitting idle. But given how much cash Meta is burning on compute these days, it’s probably not a bad thing for its chips to pull double duty – even better if one of the two use cases is highly profitable. While MTIA may play a bigger role in how Meta serves its generative AI workloads, its in-house silicon probably won’t replace AMD or Nvidia’s GPUs any time soon. The two chip designers' GPUs are currently much better suited to LLM inference and are almost certainly what Meta Superintelligence Labs is using to train frontier models like Muse Spark. Application-specific hardware has always been better suited to well-understood workloads, and Meta’s accelerators are no exception. But enough about market positioning and workloads. Let’s dig into the chip itself. Dissecting the MTIA 400 At first blush, the MTIA looks a lot like either Nvidia’s Rubin GPUs or AMD’s MI355X accelerators. The MTIA 400 is based on a heterogeneous multi-die architecture, which means it is built from a collection of different chips each with their own job. It features two compute dies, two I/O dies, and an SoC die that handles host connectivity and workload orchestration — all pretty standard stuff. Broadcom’s influence here is hard to miss. The chip almost certainly was built using the IP house’s XPU tech, which we explored in detail last year. Building a competent AI accelerator isn’t a trivial endeavor. So, if you’re trying to bring one to market quickly, it makes sense to focus on what makes your chip special and let someone like Broadcom handle the rest. Digging deeper into the MTIA 400’s heart, the compute chiplets are built on a 3nm process tech, presumably from TSMC, and feature a 6x8 grid of processing elements responsible for the bulk of the accelerator’s AI performance. Combined, the two chiplets are capable of outputting 12 petaFLOPS of MXFP4 compute at 1.7 GHz. To put that in perspective, the MTIA 400 is about 20 percent faster than Nvidia’s top-specced Blackwell accelerators at the higher precisions more commonly used to train models, while sucking back roughly the same power. But the parts don’t hold up nearly as well when compared to Nvidia and AMD’s latest chips – the MTIA 400 is between 3x and 3.3x slower than Rubin and the Instinct MI455X, respectively. Meta’s latest accelerator is fed by eight 36 GB HBM3e stacks that deliver 288 GB of memory and about 9.2 TB/s of bandwidth. Again, that’s about 15 percent faster than Nvidia and AMD’s last-gen parts, but with less than half the bandwidth of their new GPUs. This is probably why Meta is positioning the part as a training chip. It has much better and faster options for LLM inference. The MTIA 400 features a pair of I/O chiplets that provide 1.2 TB/s of chip-to-chip bandwidth over RDMA. We don’t know what transport tech Meta is using at this point, but given Broadcom’s involvement, Ethernet would be the obvious choice. Scaling up MTIA 400’s physical resemblance to Nvidia Rubin and AMD MI350-series parts also extends to the system design. Each compute blade features four MTIA 400 accelerators connected via a PCIe switch to an x86 CPU and a scale-out NIC. A single rack is equipped with 18 compute blades and eight switch blades for a total of 72 accelerators in a single unified domain. Meta shared a pic of the rack earlier this year, so while the compute blade is highly reminiscent of AMD's Helios blades, it's not using the same double wide OCP racks. Meta hasn't shown how large a cluster its chips can support just yet — the slides simply state "multi-thousand accelerator scaling" — but even at rack scale the systems should be quite competent relative to Nvidia’s GB200 and GB300 rack systems for training. More on the way While the MTIA 400 is aimed primarily at LLM training, Meta is already working on an inference-optimized version of the chip. Disclosed back in March, the MTIA 450 will double the chip’s memory bandwidth, presumably by swapping HBM3e for much faster HBM4. That part is expected to enter production next year, and should be extremely well suited to running those LLM-based recommender models Zuck and crew have been talking about the past few quarters. Meanwhile, the MTIA 500, which is also slated for 2027 release (probably in H2 if we had to guess), will increase memory bandwidth by another 50 percent — likely using 4 additional HBM4 stacks — and double the number of compute chiplets. ®