Master thesis : Large Scale and Real-Time Cattle Behaviour Recognition by Deep Learning on Video Data

Abstract

Action recognition on video data is an exploding discipline in the deep learning sector. The consequent increase in available computing power combined with improvements achieved in deep learning methods now allow us to use powerful algorithms on real videos. In the frame of this project, we have worked with AIHerd, a company that designs an intelligent video surveillance tool aiming to improve the management of dairy herds. AIHerd already handle the identification and the tracking of every animals covered by a camera sky. The goal of this project was to add to this architecture a new brick capable of characterizing their behaviours and interactions in real time. We first developed an algorithm for extracting and selecting specific video sequences of animals performing a potentially interesting behaviour. These data were used to encode short videos hosted on an annotation platform modified to our data. Students were selected and trained to annotate these videos, allowing the creation of an action recognition dataset of behaviours on which our models were trained. A prediction algorithm was implemented in order to manage a large number of possible configurations. These configurations allow us to use strategies from the state of the art in terms of action recognition, but also to evaluate several strategies of our creation to ensure a better trade-off between prediction quality and resource cost. Our training and evaluation algorithms have successfully met the technical challenge of classifying in real time a total of 16 behaviours categories simultaneously on 120 cattle covered by 5 high definition cameras. We also demonstrated that adapting old strategies, combined with various components inspired by the state of the art of deep learning, allows us to obtain an algorithm that is light enough to be integrated into the AIHerd software with a restricted decrease in prediction quality compared to the state of the art methods.