Fine-tuning SAM for Grain Segmentation in Reflected-light Microscopy

Abstract

Automated grain segmentation in reflected-light microscopy (RLM) is a key step in quantitative texture analysis and automated mineralogy, yet it remains challenging due to weak and heterogeneous contrast, strong domain variability, and other factors. Classical image-processing approaches often lack robustness, while supervised deep-learning methods require large, domain-specific annotated datasets and frequently generalize poorly across materials. This work investigates the applicability of the Segment Anything Model (SAM), a promptable vision foundation model, for grain segmentation in RLM images and evaluates whether parameter-efficient fine-tuning (PEFT) can improve its performance. First, we benchmark the zero-shot performance of SAM with ViT-B and ViT-H backbones against a fully supervised DeepLabV3+ model on two open-source ore microscopy datasets (iron ore and copper ore). Quantitative evaluation using pixel-wise metrics (IoU, Dice, precision, and recall) shows that SAM achieves segmentation accuracy comparable to DeepLabV3+ without any task-specific training, while demonstrating superior cross-dataset generalization. In contrast, DeepLabV3+ exhibits severe performance degradation when applied outside its training domain. Second, we apply SAM to polished dolomite sections. For this dataset, SAM is compared with classical stereological measurements, different preprocessing strategies are tested, and we show that SAM yields quantitatively consistent textural descriptors. Moreover, we demonstrate that fine-tuning SAM improves both mask-level and full-field segmentation accuracy while updating only a small fraction of the model parameters. Third, we apply SAM to industrial datasets of glass and silica grains from solar panels, which are challenging because the reflectivity of glass is close to that of the resin. The out-of-the-box versions of SAM can segment both glass and silica grains effectively. Overall, the study shows that (i) SAM is a strong starting point for segmenting unseen RLM data, (ii) preprocessing can be critical for achieving good segmentation results with SAM, and (iii) small, targeted fine-tuning is an effective and practical path for improving foundation models in reflected-light microscopy segmentation tasks. Finally, we release a Colab-ready codebase with a clear repository structure to support reproducibility.