Master thesis : Contribution to an Optical System for Detection and 3D Pose Estimation of Football Players

Abstract

Today, referees often use a video assistance referee (VAR) system to help them make more precise and fair decisions during football matches. However, video verification can take time and may not be obvious. This is why VAR systems can include automated features that make the task of the VAR team operating the system easier, such as automatic detection of offside. Automatic offside detection relies on the precise estimation of the 3D pose of the football players and the ball on the pitch. To achieve this, most technology providers currently propose solutions that use images captured by a dozen of dedicated cameras placed underneath the roof of a football stadium and a sensor in the ball. EVS would like to propose a solution to automatic offside detection without having to deploy dedicated hardware equipment. This requires being able to precisely estimate the pose of players in 3D solely based on non-dedicated, broadcast cameras. Ideally, this estimate should be as fast as possible to b usable under real-time conditions.

This is why in this Master Thesis, different methods of 3D pose estimation based on several broadcast viewing angles were explored. Two methods called VoxelPose and Faster VoxelPose were implemented and tested. These methods were originally tested on datasets containing few people and operating in small spaces, filmed by static and perfectly calibrated cameras. Thanks to our tests and some adaptations, we demonstrated that these methods could be used for the pose estimation of football players using non-static cameras and imperfect calibrations, i.e., in a context where only broadcast cameras are available. We also demonstrated that by using an off-the-shelf pre-trained 2D pose estimator, these methods could be trained only on synthetic data.