Deep Learning for Content-Based Image Retrieval in Biomedical applications

Abstract

Due to advances in the digital field, the number of images being generated every day grows exponentially. The field of histopathology is no exception and witnesses the emergence of an increasing number of Whole Slide Images that need to be treated, analyzed and diagnosed. One way to facilitate the diagnostic process is by comparing a particular case with other similar cases. This implies, first, the accessibility to other cases, as well as the ability to retrieve the most useful ones, i.e., the most similar cases. To achieve the latter goal, the technique of Content-Based Image Retrieval (CBIR) was conceived. CBIR involves retrieving the most similar images in a database to a given query image.

The goal of this thesis is to study the different elements that compose a CBIR framework and the options available for them, with a specific focus on the feature extraction part of the framework. It offers an open-source implementation that allows the combination of the researched options to create a fully operational CBIR framework. It provides both supervised and self-supervised models as a way to accommodate all situations and datasets.

All feature extraction models are trained on a single dataset containing over 600,000 histopathological images and evaluated on approximately 200,000 different images from the same dataset. Extensive experiments are conducted to analyze the resilience of the frameworks in different situations, such as when dealing with new data or handling class imbalance.

While the supervised models have displayed great results and the self-supervised methods have demonstrated great potential, the scope of what could have been achieved is limited by the lack of evaluation by trained pathologists and by the few remaining untested combinations.