Imaging performance of annular apertures. 6: Limitations by optical surface deviations