Design of Analog-AI Hardware Accelerators for Transformer-based Language Models
Dec 1, 2023·,,,,,
,,,,,,,,,,,,,,,,,,,,,,,,,,,·
0 min read
G. W. Burr
H. Tsai
W. Simon
I. Boybat
S. Ambrogio
C.-E. Ho
Ze-Wei Liou
M. Rasch
J. Büchel
P. Narayanan
T. Gordon
S. Jain
T. M. Levin
K. Hosokawa
M. Le Gallo
H. Smith
M. Ishii
Y. Kohda
A. Chen
C. Mackin
A. Fasoli
K. ElMaghraoui
R. Muralidhar
A. Okazaki
C.-T. Chen
M. M. Frank
C. Lammie
A. Vasilopoulos
A. M. Friz
J. Luquin
S. Teehan
I. Ahsan
A. Sebastian
V. Narayanan
Abstract
Analog Non-Volatile Memory-based accelerators offer high-throughput and energy-efficient Multiply-Accumulate operations for the large Fully-Connected layers that dominate Transformer-based Large Language Models. We describe architectural, wafer-scale testing, chip-demo, and hardware-aware training efforts towards such accelerators, and quantify the unique raw-throughput and latency benefits of Fully- (rather than Partially-)Weight-Stationary systems.
Type
Publication
2023 International Electron Devices Meeting (IEDM)
Authors
PhD student @ Princeton
Check out my recent work on hybrid model acceleration: ReplaySSM
🚀 Deployed in production: GLM-5.3-Flash and Kimi-K3
🔧 Integrated into vLLM, TensorRT-LLM, FlashInfer, and SGLang
📝 Featured on the NVIDIA Technical Blog
I started the second year of my PhD at Princeton, advised by Prof. Tri Dao. My research focuses on ML Systems.
My official name is Ze-Wei Liou, but I also go by the name “Johnny.” I did my undergrad in EE at National Taiwan University. Feel free to reach out!