HBM3 DRAM

RH+: Row-Hit-Optimized Scheduling for PIM-based LLM Inference

arXiv:2606.05511v1 Announce Type: new Abstract: Large language model inference on processing-in-memory (PIM) architectures promises to break the memory wall by performing multiply-accumulate (MAC) operations directly within HBM3 DRAM banks. Prior work identifies the power constraint timing parameter nCCDAB as the primary performance bottleneck and optimizes scheduling accordingly. We demonstrate that for GEMV operations that dominate autoregressive decoding, the DRAM row cycle time (nRC) is...