Deep Learning for the Classification and Detection of Animals in Artworks

Abstract

Digitization has established itself as an essential process in our daily modern life, as more and more business applications rely on this information processing tool in order to improve their customers' experience, for instance with online shopping. Aside of commercial uses, digitization is also applied to the domain of cultural heritage, for example to provide high-resolution representations of various kinds of artworks. In this thesis, we apply deep learning techniques to the art world with the end goal of pushing forward the digitization of artistic collections, by developing automatic techniques for their annotation process. In particular, we assess the performance of standard classification architectures when coping with a domain transfer from natural images to artworks. We then evaluate the impact of different learning settings and provide insightful observations about model predictions.

Then, we move to the object detection problem, which represents the end task of this work. We investigate state-of-the-art object detection models and try to enhance their performance using several transfer learning strategies. Additionally, we present various error patterns encountered with the models and conclude with propositions of other learning approaches and perspectives to further tackle and improve on this problem.