FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training

FORGE is a new LLM training technique that eliminates the need to materialize gradients, reducing memory usage by fusing optimizer steps.