IoT Thrust Seminar |  Continual Learning: From Traditional ML to Modern Foundation Models

Deep learning has driven remarkable progress across diverse domains, yet most methods operate under the assumption of stationary data distributions. However, in real-world scenarios, data distributions evolve over time, posing significant challenges such as catastrophic forgetting and the need to balance performance across both old and new tasks. These challenges are particularly critical in (1) resource-constrained environments, such as edge devices, and (2) large language models, where the initial pre-training data is often unavailable after the model's deployment.

In this talk, I will first present approaches to tackle these challenges from three complementary perspectives: data-centric, biology-inspired, and model-centric, all aimed at building solid theoretical foundations for continual learning. Extending beyond conventional continual learning, I will also explore advancements tailored to foundation models, including vision transformers and large language models (LLMs). Finally, I will share my research vision and outline future directions for enabling continual learning in dynamic, real-world settings.