Model-free reinforcement learning for focal plane wavefront control

Abstract

Direct imaging of exoplanets requires high angular resolution. Adaptive optics systems are used to correct phase aberrations by using deformable mirrors to reach the diffraction-limited angular resolutions of the optical systems. Recently, data-driven methods and reinforcement learning approaches in particular have shown great promise for the control of adaptive optics systems. However, most of them rely on wavefront sensor measurements. Therefore, the phase aberrations that are not detected by the wavefront sensor, such as Non-Common Path Aberrations, can not be corrected. Furthermore, previous work considered policies conditioned on a fixed number of past observations, while the partially observable nature of the problem would suggest that the entire history needs to be considered. This thesis extends previous work in model-free reinforcement learning for the control of adaptive optics systems to correct phase aberrations from focal plane images alone. The models rely on the sequential phase diversity produced by the operation of the system itself and therefore do not require any additional phase diversity observations. The influence of the history size on which the models are conditioned is also studied. Models trained in simulation are able to correct static aberrations spanning $18$ Zernike modes to near optimal Strehl ratios, although the Fast\&Furious baseline considered still outperforms the models due its superior stability. Policies conditioned on two observations alone obtain comparable results to a recurrent policy that considers the entire history, although recurrent policies show better performance when faced with higher noise. However, the application of these models on real systems is still limited by their low sample efficiency which is made worse by the high-dimensionality of the action space.