AutoNorm: Understanding Adaptive Normalization in Transformers through Differentiable Gating
Research investigates adaptive normalization in Transformers, addressing gradient variance challenges in differentiable gating for improved training stability.