Faculté des Sciences appliquées
Faculté des Sciences appliquées

Master Thesis : Online Distillation with Continual Learning for Cyclic Domain Shifts

Houyon, Joachim ULiège
Promotor(s) : Van Droogenbroeck, Marc ULiège
Date of defense : 26-Jun-2023/27-Jun-2023 • Permalink :
Title : Master Thesis : Online Distillation with Continual Learning for Cyclic Domain Shifts
Author : Houyon, Joachim ULiège
Date of defense  : 26-Jun-2023/27-Jun-2023
Advisor(s) : Van Droogenbroeck, Marc ULiège
Committee's member(s) : Louppe, Gilles ULiège
Cioppa, Anthony ULiège
Language : English
Number of pages : 90
Keywords : [en] Computer Vision
[en] Deep Learning
[en] Continual Learning
Discipline(s) : Engineering, computing & technology > Computer science
Commentary : In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of the University of Liège’s products or services. Internal or personal use of this material is permitted. If interested in reprinting/republishing IEEE copyrighted material for advertising or promotional purposes or for creating new collective works for resale or redistribution, please go to to learn how to obtain a License from RightsLink. If applicable, University Microfilms and/or ProQuest Library, or the Archives of Canada may supply single copies of the dissertation.
Target public : Researchers
Professionals of domain
Institution(s) : Université de Liège, Liège, Belgique
Degree: Master en science des données, à finalité spécialisée
Faculty: Master thesis of the Faculté des Sciences appliquées


[en] The technique of online distillation has become increasingly popular in adapting real-
time deep neural networks using a slow and accurate teacher model. However, one of the
most significant challenges encountered with online distillation is catastrophic forgetting,
which happens when the student model is updated with new domain data and loses the
previously learned knowledge.
The main contribution in this thesis is to apply continual learning techniques to mitigate
the problem fo catastrophic forgetting in online distillation. Indeed, continual learning has
shown to be useful in a more general setting, where the model tend to forget knowledge
from previous tasks when learning the current one. The study aims to assess the efficacy
of various state-of-the-art continual learning methods in reducing catastrophic forgetting
when applied to online distillation, particularly in cyclic domain shifts.
The experimental results presented in this study show improved accuracy and robustness
in the context of online distillation when leveraging continual learning methods to reduce
catastrophic forgetting, with potential applications in fields such as video surveillance or
autonomous driving. As such, this work represents a significant contribution to the fields
of online distillation and continual learning, providing new insights and avenues for future



Access thesis.pdf
Size: 30.05 MB
Format: Adobe PDF


  • Houyon, Joachim ULiège Université de Liège > Mast. sc. don. à fin.


Committee's member(s)

  • Louppe, Gilles ULiège Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Big Data
    ORBi View his publications on ORBi
  • Cioppa, Anthony ULiège Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Télécommunications
    ORBi View his publications on ORBi
  • Total number of views 51
  • Total number of downloads 50

All documents available on MatheO are protected by copyright and subject to the usual rules for fair use.
The University of Liège does not guarantee the scientific quality of these students' works or the accuracy of all the information they contain.