Hölder network for improved adversarial robustness.

Area of Science:

• Machine Learning
• Deep Learning Architectures
• Robustness in AI

Background:

• Lipschitz constants enhance model robustness but can limit function approximation capabilities.
• Existing methods may overly constrain neural networks, hindering performance on complex tasks.

Purpose of the Study:

• Introduce the Hölder network, a novel architecture generalizing Lipschitz constraints.
• Enforce alpha-Hölder continuity using alpha-rectified power units (α-RePU).
• Provide theoretical guarantees for universal approximation of Hölder continuous functions.

Main Methods:

• Developed the Hölder network architecture incorporating α-RePU.
• Theoretically proved the universal approximation property for Hölder continuous functions.
• Conducted empirical evaluations on image classification and tabular data.

Main Results:

• Hölder networks achieve accuracy comparable to existing models.
• Demonstrated superior adversarial robustness against various attacks (e.g., PGD, l∞).
• Showcased flexibility beyond hard Lipschitz constraints.

Conclusions:

• The Hölder network offers a more flexible and robust alternative to standard Lipschitz-constrained networks.
• α-RePU units effectively enable α-Hölder continuity, enhancing model generalization and robustness.
• The proposed framework shows promise for improving AI model resilience in diverse applications.