Nature Computational Science · 2025 · 13 citations · 26 references
ComputingComputer EngineeringComputer ScienceMachine LearningMemoryNatural Language ProcessingGraphics Processing UnitsHardware AccelerationLanguage ModelingLow-power Cache DesignNatural Language GenerationText GenerationAttentionEnergy ConsumptionEnergy-efficient ComputationInitializationLarge Language ModelsRandom-access MemorySequence GenerationStatic Random-access Memory
Transformer networks, driven by self-attention, are central to large language models. In generative transformers, self-attention uses cache memory to store token projections, avoiding recomputation at each time step. However, graphics processing unit (GPU)-stored projections must be loaded into static random-access memory for each new generation step, causing latency and energy bottlenecks. Here we present a custom self-attention in-memory computing architecture based on emerging charge-based memories called gain cells, which can be efficiently written to store new tokens during sequence generation and enable parallel analog dot-product computation required for self-attention. However, the analog gain-cell circuits introduce non-idealities and constraints preventing the direct mapping of pre-trained models. To circumvent this problem, we design an initialization algorithm achieving text-processing performance comparable to GPT-2 without training from scratch. Our architecture reduces attention latency and energy consumption by up to two and four orders of magnitude, respectively, compared with GPUs, marking a substantial step toward ultrafast, low-power generative transformers.
26
A compute-in-memory chip based on resistive random-access memory
Weier Wan, Rajkumar Kubendran, Clemens Schaefer et al. · Nature · 2022 · 833 citations · Full text
HellaSwag: Can a Machine Really Finish Your Sentence?
Rowan Zellers, Ari Holtzman, Yonatan Bisk et al. · 2019 · 707 citations · Full text
PIQA: Reasoning about Physical Commonsense in Natural Language
Yonatan Bisk, Rowan Zellers, Ronan Le Bras et al. · 2020 · 594 citations
Computing, Computer Science, Physics +14