LightRot: A Light-Weighted Rotation Scheme and Architecture for Accurate Low-Bit Large Language Model Inference

LightRot introduces a lightweight rotation scheme and hardware architecture for energy-efficient, accurate low-bit large language model inference.