Abstract

The 6T SRAM cell has been the default last-level cache technology for more than two decades. At sub-3nm nodes, the cost-capacity tradeoff for SRAM is worsening in ways that are reopening a debate that seemed settled: whether embedded DRAM (eDRAM), or one of its modern successors, can displace SRAM for large on-chip caches. This is not an academic question - it directly affects how large a last-level cache a processor can afford at a given die area, which in turn affects the hit rates achievable for AI inference workloads where weight tensors are large and reuse is irregular.

SRAM Scaling Woes at Sub-3nm

SRAM bit-cells do not scale with logic as favorably as the node nomenclature implies. At TSMC N3 (3nm-class) and Samsung SF3, the minimum SRAM bit-cell area is roughly 0.021 square microns - a shrinkage of about 15-20% from N5, compared to logic gate shrinkage of 30-40% over the same transition. The result is that SRAM consumes an increasingly large fraction of die area relative to compute. For a high-end CPU die, a 64 MB LLC consumes roughly 50-70 mm2 at N3, which at current wafer costs represents several hundred dollars of silicon per chip just for the cache. Additionally, SRAM leakage power at sub-3nm is not improving proportionally with voltage scaling, because threshold voltage cannot track supply voltage reductions without unacceptable static noise margin degradation. These economic and power realities are pushing architects toward alternatives.

eDRAM’s Comeback and Its Complications

IBM used eDRAM for on-chip cache through the POWER7/POWER8 era before abandoning it due to process complexity. Apple’s A-series chips use high-density SRAM marketed under the SLC/LLC naming, but have not disclosed eDRAM use. The renewed interest in eDRAM at sub-3nm comes from two directions: DRAM process innovations (particularly deep-trench capacitor eDRAM from research groups at imec and Fraunhofer IIS) that can be co-integrated on a logic wafer with manageable yield impact, and the emergence of 3D-stacked SRAM (as used in AMD’s 3D V-Cache technology, which stacks SRAM dies using hybrid bonding). The 3D stacking approach is not eDRAM but achieves similar density goals through packaging rather than process integration, and it is currently the most commercially mature path. AMD’s Genoa-X and Milan-X product lines demonstrated that 3D-stacked SRAM can deliver 96 MB of L3 at latencies competitive with conventional monolithic SRAM. The access latency penalty compared to eDRAM is lower (3D-SRAM accesses the stacked die via hybrid bonding at roughly 2-4 cycles additional latency), while avoiding the refresh overhead eDRAM requires.

Implications for AI Inference Cache Sizing

The capacity-latency tradeoff analysis looks different for AI inference than for general-purpose CPU workloads. Transformer weight access during decode has poor spatial locality (each head accesses a different slice of the weight tensor) but predictable reuse at the layer level. A large LLC - 64 MB or larger - can hold several transformer layers’ weights and avoid repeated bandwidth from DRAM or HBM, meaningfully reducing decode latency. The critical number is not the raw cache size but the effective bandwidth to the accelerator compute, which depends on cache associativity, access width, and the number of parallel read ports. eDRAM, even with its density advantage, has lower bandwidth per bit than SRAM due to the sense amplifier sharing and precharge constraints. For workloads that are bandwidth-bound at the cache rather than latency-bound, this can negate the capacity advantage. Sub-3nm design teams evaluating cache hierarchy choices for AI inference chips should model their specific weight access patterns before concluding that maximum capacity beats maximum bandwidth.