Memory has become a major bottleneck for industry growth drivers, such as AR/VR, Machine Learning (ML), Internet-of-Things (IoT), and Automotive, where the performance and power efficiencies are limited due to data movement between the various memory hierarchies and the operational units. In fact, today, the amount of embedded memory on almost any integrated circuit (chip) is reaching hundreds of megabits, accounting often for up-to 75% of the total chip area. Today’s predominant on-chip memory solution, based on SRAM, no longer provides the density requirements of modern systems, reaching the end of its scaling in advanced CMOS technologies.

Important industry growth drivers, such as ML, IoT, Automotive and AR/VR, operate on ever-growing amounts of data that is typically stored off-chip in an external DRAM. Unfortunately, off-chip memory accesses are up-to 1000x more costly in latency and power compared to on-chip data movement. This limits the bandwidth and power efficiency of modern systems. In order to reduce these off-chip data movements, almost all SoCs incorporate large amounts of on-chip embedded memory caches that are typically implemented with SRAM and often constitute over 50% of the silicon area. This memory bottleneck is further aggravated since SRAM scaling has reached its limits and no longer shrinks in advanced CMOS process nodes beyond 5nm.