Rapid Cytomine: foundation models for interactive annotation in computational pathology

Abstract

This thesis presents the integration of an interactive segmentation tool into the Cytomine platform, based on the Segment Anything Models (SAM and SAM2) published by Meta, in order to simplify the manual annotation process. Cytomine is a collaborative application which is designed for sharing and annotating large biomedical images (with a particular interest for histopathology). It allows users to interact with the large-scale images from digital pathology scanners or other sources, and to annotate as well as analyze them collaboratively. Collecting high-quality annotations in such specialized domains is a significant challenge, as it often requires the involvement of experts (such as researchers or medical professionals), whose time is limited and valuable. Therefore, providing an efficient segmentation tool is extremely important.

To address this challenge, both SAM and SAM2 were first evaluated and compared in a zero-shot setting. A series of fine-tuning experiments under various training configurations were then conducted to assess the performance sensitivity to different parameters, and to eventually identify the best-performing setting. Several post-processing strategies were also explored to enhance the mask quality and usability, and the possible advantages of integrating domain-specific encoders alongside SAM were also investigated.

The best-performing model was integrated into the latest release of Cytomine through new API endpoints as well as a new back-end server. To support future development, a tutorial was also created to guide users and developers through the process of modifying Cytomine to integrate custom API endpoints.