Samsung's LPDDR5X-PIM puts 16 compute blocks beside DRAM and reports 3.01x Llama 3.1 throughput
- Samsung presented a 16 GB LPDDR5X-PIM package at Hot Chips 2026 with 614 GB/s internal bandwidth to its compute units, versus 76.8 GB/s through its external pins.
- On Samsung's edge AI accelerator, Llama 3.1 8B at a 320-token context ran at 81.3 tokens/s on LPDDR5X-PIM versus 27 tokens/s on ordinary LPDDR5X, using SINT8 activations, SINT4 weights, and SINT32 output.
- The package places 16 PIM blocks beside DRAM banks; the host broadcasts FP8 activations and commands, while weights stay in the package and the PIM blocks return smaller partial sums.
- Samsung's Address Align Mode maps DRAM addresses to MAC instructions, allowing a conventional DRAM controller to switch between ordinary single-bank DRAM access and multi-bank PIM operation.
- The article argues PIM best suits batch-1 autoregressive GEMV, where each generated token rereads model weights and throughput is roughly memory bandwidth divided by model size; KV-cache traffic lowers the practical result.
Hacker News opinions
I like compute in RAM, but I want a physical switch that disables it and returns the DIMM to ordinary RAM. If this reaches consumer hardware, it should also have debug ports, proxying, and single-step support. SIMDRAM reported up to 2.5x over Ambit and much better energy efficiency on selected kernels.
I do not expect commodity DIMMs to replace ordinary DRAM with PIM. Added compute raises cost, so it will stay in niches where the benefit pays for itself.
I want general-purpose cores in RAM, not another AI-only unit.
I looked at this for a client almost 20 years ago. The hard part was software support, and open source does not make that happen by itself.
AI inference is unusually suited to this because it repeats one operation across every model weight. General-purpose work needs more silicon area and usually is not as bandwidth-bound, so fewer workloads will benefit from processing in RAM.
Compute-in-memory has dominated much of the electronics research I have seen this year. RAM designers are in demand, and I am curious which approaches actually survive in products.
Processor-in-memory is an old idea. A 1998 Computational RAM thesis proposed many 1-bit processors beside DRAM columns, but DRAM and fast logic use very different fabrication processes, and heat is a problem. HBM logic-base dies and Intel and Micron's Hybrid Memory Cube followed related reasoning.
Duncan Elliott taught my graduate computer architecture course while writing that thesis. We discussed C-RAM then, and even it was probably a rehash of older work.
I can imagine NAND, logic, and DRAM dies stacked together, with each made on its own process and connected by TSVs or bonding. The storage, working-memory, and compute ratios would still need to work out.
I want to know how in-DRAM processing handles the KV cache. The article does not explain it, and I do not think the answer is obvious.
Putting many ALUs near DRAM also creates heat, while DRAM is very heat-sensitive. Process differences between DRAM and digital logic are another long-standing problem.
What happened to using memristors for this kind of compute?
This is a very specific operation built into RAM, so the extra transistors are wasted outside AI. If a different AI operation wins later, the hardware cannot adapt, and a general design starts to resemble many tiny microcontrollers with local memory.