Master thesis : 3D object registration with deep learning

Abstract

This thesis was realized within the company Euresys. The main objective is to implement an easy-to-implement deep learning solution to infer the pose of an object on a Zmap image acquisition to replace the existing non-deep learning solution from Euresys whose robustness could be improved. Indeed, the previous solution encounters difficulties when the object is far from being perpendicular to the point of view, as the solution is based mainly on 2D alignment. This new deep learning solution offers more flexibility to capture the object in any view.

Since obtaining annotation for the 3D object registration can be costly and time-consuming. Thus, synthetic images of the Zmap format are created by projecting a 3D model on a reference plane.

By exploring different existing methods, an extension of the model YOLOV5small to solve this task is proposed in two steps. First, a 2D object detector is designed to show us the feasibility of training a model on the synthetic dataset to infer predictions on the acquisition. In the process, we noticed that good results can be achieved by changing the reference plane of the Zmap acquisition during the test and by using appropriate data augmentation during training.

Then, an extension of the 2D model detector to infer the 3D object registration is proposed by discretizing the model space into viewpoints and adding a regression parameter to infer the in-plane rotation. Unfortunately, we did not obtain metrics of the accuracy of the prediction on the acquisition but proposed a comparaison between the acquisition and a synthetic reproduction. This gave us interesting results but we thought that they could be improved. For example, by using the refinement step (ICP) already used by Euresys for the previous solution which could help to have better results. This thesis concludes with information on potential improvements and scope extensions that could be made for further work.