In Fine-Tuning (Part 3: Evaluation & Sharing), we fine-tuned a small BERT model. But if you try to fine-tune a modern LLM (like Llama 3 or Mistral) the standard way, you’ll run out of memory instantly. This is where tools like Hugging Face LoRA PEFT become essential for efficient model fine-tuning.
The solution is PEFT (Parameter-Efficient Fine-Tuning) and LoRA (Low-Rank Adaptation). Instead of retraining the entire model, LoRA freezes the main model and only trains a tiny “adapter” layer on top. This reduces memory usage by 90%+.
Step 1: Installation
We need the peft library and bitsandbytes for quantization (making the model smaller).
pip install transformers torch peft bitsandbytes datasets
Step 2: Load the Model with Quantization
We load the base model in “4-bit” mode to save massive amounts of RAM.
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
model_id = "facebook/opt-350m" # A smaller LLM for demonstration
# For a real LLM, use "mistralai/Mistral-7B-v0.1" (requires ~16GB VRAM)
# 1. Configure 4-bit quantization
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.float16
)
# 2. Load the model
model = AutoModelForCausalLM.from_pretrained(
model_id,
quantization_config=bnb_config,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_id)Step 3: Apply LoRA
This is the magic step. We attach the LoRA adapters to the model.
from peft import LoraConfig, get_peft_model, TaskType
# 1. Define the LoRA configuration
peft_config = LoraConfig(
task_type=TaskType.CAUSAL_LM,
inference_mode=False,
r=8, # Rank (higher = smarter but slower)
lora_alpha=32, # Scaling factor
lora_dropout=0.1
)
# 2. "Wrap" the model with PEFT
model = get_peft_model(model, peft_config)
model.print_trainable_parameters()Output: trainable params: 1,572,864 || all params: 350,000,000 || trainable%: 0.44 See that? We are only training 0.44% of the model! This is why it’s so fast and cheap.
Step 4: Train!
Now you just use the standard Trainer like we did before, but pass in this new model.
Key Takeaways
- Fine-tuning modern LLMs like Llama 3 or Mistral can quickly lead to out-of-memory errors.
- PEFT (Parameter-Efficient Fine-Tuning) and LoRA (Low-Rank Adaptation) optimise this process by training only a small adapter layer, reducing memory usage by over 90%.
- Key steps include installing the necessary libraries, loading the model in ‘4-bit’ mode, applying LoRA, and training using the standard Trainer.
- This method allows training of just 0.44% of the model parameters, making it both fast and cost-effective.
