Efficient Interactive Annotation for Cytomine

Abstract

Labelling and annotating histological images requires expertise but above all precious time (e.g. clinical time for pathologists). This is why improving the tools to annotate is crucial. Fortunately, the development of technologies such as deep learning have already brought non negligible improvements to computer vision and imagery related tasks. In this thesis, a study is conducted on the use of a patch-matching based method for annotations in multiple stained sections of a tissue. The final goal is to identify a significant structure thanks to a patch (or also known as a window) in other sections of the same tissue by making patch-to-patch comparison. It is aimed to avoid the repeating process of labelling several time an identical object. The task is not simple as multiple sections might suffer from deformations and difference in staining, among others. In order to make the comparison, neural networks are used and two similar approaches are tested. The first one consists in extracting some features from the patches thanks to pre-trained networks and comparing them with a measure called the cosine similarity. The second one consists in doing the same thing but the networks and/or the similarity measure are now trained on a specific dataset. The first approach yields the best results. Even if the trained networks have learned to compare, they lack of a huge and diverse dataset to offer convincing results. At this state, a real application can not be built on the basis of the proposed method. This is why, at the end of the report, some suggestions are given to continue investigating it. The study still offers a good starting point.