The error surface of the 2-2-1 XOR network: The finite stationary points

We investigate the error surface of the XOR problem for a 2-2-1 network with sigmoid transfer functions. It is proved that all stationary points with finite weights are saddle points with positive error or absolute minima with error zero. So, for finite weights no local minima occur. The proof results from a careful analysis of the Taylor series expansion around the stationary points. For some points coefficients of third or even fourth order in the Taylor series expansion are used to complete the proof. The proofs give a deeper insight into the complexity of the error surface in the neighbourhood of saddle points. These results can guide the research in finding learning algorithms that can handle these kinds of saddle points.