LoRA (Low-Rank Adaptation)

What Is LoRA (Low-Rank Adaptation)?

Full fine-tuning updates every parameter and produces a complete model copy per task, which is expensive to train and store. LoRA inserts small low-rank weight matrices and trains only those, leaving the base model frozen. The result is a tiny adapter you can swap per task and stack, cutting training cost dramatically. It is the default parameter-efficient fine-tuning method in 2026.