Samsung LPDDR5X-PIM Puts AI Compute Inside DRAM With 8× the Bandwidth

Photo by Brecht Corbeel on Unsplash

Samsung showed working silicon of LPDDR5X-PIM at Hot Chips 2026 that pushes 614 GB/s of internal bandwidth, roughly eight times the 76.8 GB/s of standard LPDDR5X-9600, and runs Llama 3.1 8B inference 3.01× faster than the same accelerator paired with conventional LPDDR5X. The chip drops into the same JEDEC 561-ball footprint as standard LPDDR5X, meaning device makers can adopt it without redesigning the PCB.

The pitch is narrower than the headlines suggest. Samsung is not aiming at HBM in the datacenter. It is aiming at the far larger pool of smartphones, AI PCs, and edge accelerators where HBM's cost and power make it unusable, and where inference workloads have started to strain conventional DRAM bandwidth.

The 614 GB/s Number and What It Actually Measures

Samsung's Hot Chips presentation, covered in detail by Tom's Hardware, frames LPDDR5X-PIM as the industry's first LPDDR-based processing-in-memory device for AI inference. The 614 GB/s figure is the effective bandwidth available to the in-DRAM compute units, not the external interface bandwidth to the host SoC. That distinction matters because the gain comes from keeping data local: small logic units inside the DRAM execute matrix operations against weights already sitting in the same die, eliminating the round trips that dominate energy and time in transformer inference.

On a real workload, BigGo Finance reports that Samsung demonstrated Llama 3.1 8B running at 81.3 tokens per second on an edge accelerator with LPDDR5X-PIM, versus 27 tokens per second with standard LPDDR5X on the same platform. End-to-end task completion for the same prompt dropped from 12.3 seconds to 5.4 seconds, a 2.28× wall-clock speedup. The token-rate improvement is larger than the wall-clock improvement because host-side setup and decode overhead don't move when memory gets faster.

Samsung principal engineer Hwang Garam led the presentation on August 25 at Stanford. His framing, quoted by BigGo, was direct: "HBM is currently the de facto standard for AI memory, but as AI evolves toward inference-centric workloads and expands into devices like smartphones and AI PCs, a new memory form factor has become necessary."

The architectural principle is old and the execution is new. Processing-in-memory places small compute units inside the memory die so data storage and calculation happen in the same physical location, as BigGo Finance describes. Repeated processor fetches from DRAM disappear. For inference workloads that are almost entirely bandwidth-bound at the attention and feed-forward layers, removing those fetches is the single biggest lever available short of switching memory technology entirely.

The Accuracy Footnote Samsung Buried

One detail from the Q&A after the presentation deserves more attention than it has received. TechTimes reported that a conference attendee raised a concern about output accuracy: LPDDR5X-PIM produces results that differ slightly from what the same model produces on standard LPDDR5X. Samsung confirmed the discrepancy exists and said optimization work is ongoing. The gap has not been quantified publicly.

This is not a minor engineering wrinkle. Processing-in-memory typically uses lower-precision arithmetic or restricted numerical formats to fit compute units into the tight thermal and area budgets of a DRAM die. If the accuracy drift is small enough to sit inside the noise floor of quantized inference, most product teams will ship it. If it isn't, the 3.01× speedup starts to look like a comparison between two subtly different models, and any customer running eval suites against reference outputs will have to re-baseline. Samsung has not said which side of that line the current silicon lands on.

A quantified answer matters more for some deployments than others. A smartphone assistant summarizing a message tolerates output drift the same way it tolerates temperature-based sampling variation. A medical transcription tool or a code completion model running against a compliance benchmark does not. The absence of a published accuracy number means every prospective customer will need to run their own comparison suite before a design commit, which is exactly the kind of friction that slows adoption in a category where the incumbent alternative already works.

The historical parallel worth flagging: the same tension has followed every generation of near-memory and in-memory compute, going back to UPMEM's DPU-in-DRAM in 2019. The pattern is consistent. The bandwidth wins are real, the precision compromises are real, and adoption stalls until a large customer decides the tradeoff pencils out for a specific model family.

LPDDR5X-PIM vs HBM: Different Problems, Different Chips

The LPDDR5X-PIM vs HBM comparison is being drawn everywhere, and it's misleading in both directions. HBM4 delivers up to 2 TB/s per stack, per TechTimes, and remains the standard for large-scale AI training and high-throughput cloud inference. Nothing in Samsung's announcement changes that. Even at 614 GB/s, LPDDR5X-PIM does not approach HBM bandwidth, as Tom's Hardware notes explicitly.

The interesting number sits on the other side of the ledger. ServeTheHome's coverage cites Samsung's own market data: HBM's share of AI chip component spending grew from 52 percent in Q1 2024 to 63 percent by Q4 2025. That is the commercial pressure Samsung is responding to. Every dollar of AI infrastructure spend is getting pulled toward a memory technology that a handful of customers can afford and almost nobody else can put in a phone, a laptop, or an edge inference box.

LPDDR5X-PIM targets that gap. The package Samsung showed carries 16 GB of capacity across four dies per rank, eight dies total, in the JEDEC-standard 561-ball layout, according to ServeTheHome. Cost per gigabyte, power draw, and packaging complexity are all closer to commodity LPDDR than to HBM's silicon interposer and TSV stack. That is the entire point.

Samsung also targets three form factors from a single chip design: server, client, and mobile, per Tom's Hardware. A memory vendor that can amortize the same die across a phone, a laptop, and an edge server has a cost structure HBM cannot match at low and mid volumes.

Where the chip actually fits

• Smartphones running on-device LLMs where sustained token generation is currently thermal- and bandwidth-limited
• AI PCs with NPUs that stall waiting for weights over standard LPDDR5X links
• Edge inference accelerators like Intel's Crescent Island, which Tom's Hardware notes is already cited as an early landing spot
• Automotive inference silicon where power envelopes rule out HBM entirely

Nvidia's Vera CPU has adopted LPDDR5X via SOCAMM2 modules, and Intel's Crescent Island AI accelerator uses LPDDR5X as its primary memory, per TechTimes. Both are natural candidates for a PIM-enabled drop-in upgrade, though neither company has publicly committed. Samsung's semiconductor roadmap work has been building toward this positioning for two years.

Why the 561-Ball Package Is the Real Story

The headline number is bandwidth. The number that will decide adoption is the ball count. BigGo Finance flags backward compatibility as a central selling point: the chip uses the same 561-ball package as standard LPDDR5X, enabling drop-in replacement without PCB redesign.

That single decision collapses the adoption timeline for anyone already shipping LPDDR5X in a product. A phone maker qualifying next year's SoC does not have to change board layout, socket tooling, or reflow profile. They validate the chip against their inference workloads, weigh the accuracy footnote, and decide. For AI PC hardware vendors with 18-month design cycles, that matters more than any spec sheet.

The catch: drop-in compatibility only holds at the mechanical and electrical layer. Software has to be rewritten to dispatch matrix operations to the in-memory compute units instead of the host NPU or GPU. That work touches the model compiler, the runtime, and the driver stack. Samsung has not detailed which frameworks are supported today.

That software gap is where earlier PIM efforts have consistently died. A drop-in chip with no compiler support is a chip nobody uses. Samsung's advantage as an incumbent memory vendor is that it can lean on existing customer relationships to push driver and runtime work into partner stacks. Its disadvantage is that Samsung is not a compiler company, and the runtime work required to schedule ops between a host NPU and in-DRAM compute units is not trivial. Nothing in the Hot Chips presentation resolved that question.

The Standards Path and Who Blinks First

Samsung has already submitted LP-PIM to JEDEC for standardization, which ServeTheHome frames as a meaningful step from research demonstration toward a shipping product category. Industry adoption almost certainly requires a large customer to commit first, and standardization is required before that customer can second-source the part from another vendor. Neither piece is in place yet.

SK Hynix has signaled plans for processing-in-module solutions targeting LPDDR6, expected around 2028, per TechTimes. Micron has not announced an LPDDR-PIM roadmap. Samsung is alone with working silicon in 2026, which is either a two-year lead or a very lonely position to occupy, depending on how the JEDEC process moves and whether a lead customer materializes.

The processing-in-memory AI inference category has a long history of demos that never shipped in volume. UPMEM, Mythic, Syntiant, and a half-dozen research startups have all shown working silicon at conferences over the past decade. What is different about Samsung's presentation at Samsung Hot Chips 2026 is the combination of an incumbent memory vendor, a JEDEC-compatible package, and a submitted standardization proposal. That is the first time all three have lined up.

The second-source question is the one that will determine whether this becomes a category or stays a Samsung SKU. No large phone or accelerator vendor commits to a single-source memory part for a mainstream product line. If SK Hynix's LPDDR6-PIM lands in 2028 with a compatible programming model, Samsung's 2026 lead becomes a genuine head start. If SK Hynix ships a different interface or Micron stays out entirely, the category fragments before it forms.

What to Watch Next

The next decision point is the JEDEC standardization vote and whether a named customer, most plausibly Intel for Crescent Island or Nvidia for a future edge SKU, commits to LPDDR5X-PIM in a shipping product before SK Hynix's LPDDR6-PIM arrives around 2028. If Samsung lands one of those customers within the next 12 months, the AI memory bandwidth edge devices conversation shifts from HBM-or-nothing to a real second tier. If it doesn't, LPDDR5X-PIM becomes another well-engineered conference demo, and the accuracy footnote from the Q&A becomes the reason retold at the next Hot Chips.

Watch for a Samsung product announcement or a customer disclosure at CES 2027 in January. That is the first calendar window where a design win would surface publicly. A published accuracy delta from Samsung, ideally on a standard benchmark like MMLU or HumanEval, would remove the largest remaining objection for customers evaluating the part today. Related coverage on memory and storage trends and the broader AI infrastructure buildout will track the follow-through.