A Machine-Learning-Driven Sky Model