GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference

GyRot proposes a quantization framework and hardware accelerator to improve low-bit LLM inference by integrating rotation and fine-grained group quantization.