Nvidia shrinks LLM memory 20x without changing model weights

Nvidia's KV Cache Transform Coding (KVTC) compresses LLM key-value cache by 20x without model changes, cutting GPU memory costs and time-to-first-token by up to 8x for multi-turn AI applications.

New KV cache compaction technique cuts LLM memory 50x without accuracy loss

MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — ...
EDN

MRAM technologies: from space applications to unified cache memory?

Magneto-resistive random access memory (MRAM) is a non-volatile memory technology that relies on the (relative) magnetization state of two ferromagnetic layers to store binary information. Throughout ...
EurekAlert!

UTA researcher working to use computer cache to speed up memory access

Computer memory capacity has expanded greatly, allowing machines to access data and perform tasks very quickly, but accessing the computer's central processing unit, or CPU, for each task slows the ...
EDN

Last-level cache has become a critical SoC design element

LLC, positioned between external memory and internal subsystems, stores frequently accessed data close to compute resources.
TechSpot

AMD 3D V-Cache CPU memory used to create incredibly fast RAM disk

Why it matters: A RAM drive is traditionally conceived as a block of volatile memory "formatted" to be used as a secondary storage disk drive. RAM disks are extremely fast compared to HDDs or even ...
Nature

Cache Performance and Memory Hierarchy Optimization

The dynamic interplay between processor speed and memory access times has rendered cache performance a critical determinant of computing efficiency. As modern systems increasingly rely on hierarchical ...
Semiconductor Engineering

A Primer On Last-Level Cache Memory For SoC Designs

System-on-chip (SoC) architects have a new memory technology, last level cache (LLC), to help overcome the design obstacles of bandwidth, latency and power consumption in megachips for advanced driver ...
TechRepublic

Cache-Only Memory Architecture

A Cache-Only Memory Architecture design (COMA) may be a sort of Cache-Coherent Non-Uniform Memory Access (CC- NUMA) design. not like in a very typical CC-NUMA design, in a COMA, each shared-memory ...
ExtremeTech

PC OEMs Are Selling Laptops With Optane Cache Drives and Claiming It's Memory

Multiple PC OEMs are selling laptops outfitted with Intel Optane cache drives -- but they're improperly combining that information in ways that makes it seem as if the Optane cache drive represents ...
sciencex

UTA researcher working to use computer cache to speed up memory access

Computer memory capacity has expanded greatly, allowing machines to access data and perform tasks very quickly, but accessing the computer's central processing unit, or CPU, for each task slows the ...