LW
AI Research
Nov 2025
Lapis Whale
Continual Learning Framework for Vision Transformers
Problem
Vision Transformers suffer from catastrophic forgetting when learning new tasks sequentially.
Solution
Built a modular continual learning framework introducing a novel Selective Replay Utilization Mechanism (SERUM) — a class-balanced memory buffer with composite replay loss.
Result
Matched near-Naive Replay performance on sequential CIFAR-100 tasks using only a fraction of the memory footprint.
77.66%
Accuracy10.74%
Forgetting2%
Memory Footprint
Python
PyTorch
CUDA
ViT
Continual Learning