Master's Thesis : Machine learning for automatic sport video analysis

Abstract

There is a colossal amount of money at stake in big soccer events and the trend is to increase the number of cameras around the soccer field. The show offered to the spectators is gaining in quality thanks to the multiplicity of image sources being able to capture every moment in the stadium during the soccer game. Although there are fewer cameras for minor sport competition, the trend is the same. Unfortunately, the image acquisition step is costly and a major part of this cost is due to the hiring of professionals to steer those cameras.

This issue motivates the development by the EVS company of an automated, robotized videography system to replace the human operators for certain cameras around the stadium. Automated robotic cameras would be a less expensive alternative allowing both the big broadcaster to cut their image acquisition cost and the smaller broadcasters to provide an affordable quality show to their telespectators. It is in this context that this master thesis takes place and it has been conducted to know if it was possible to mimic the human behavior of a cameraman with ptz robotic camera in a football stadium based on a data-driven approach.

A dataset composed of 10 soccer games was provided by EVS as well as different algorithms already developped by the company in order to extract the useful information from the principal camera images of the soccer field. Information such as the camera calibration in real time or the position of the action on the soccer field. From there, machine learning models have been trained on this dataset to learn how to steer those robotic cameras. Those models output a real-time prediction which resemble professional human operator behavior for similar situation. It is then possible to mimic the behavior of professional cameramen.

In order to evaluate our models, the sequences filmed by professional cameramen from the camera that wanted to be robotized were taken as ground truth making the underlying assumption that the 16M cameramen had perfect control over their camera and that they always filmed the game in an ideal way. The IoU (Intersection over Union) of the areas of the soccer field covered by the images from the robotic cameras and the ground truth image was computed. At the end, the models managed to outperform the heuristics based algorithm developped by EVS.