Capacity bounds for hyperbolic neural network representations of latent tree structures

Area of Science:

• Machine Learning
• Geometry
• Neural Networks

Background:

• Deep hyperbolic neural networks (HNNs) offer unique advantages for representing complex data structures.
• Understanding the representation capacity of HNNs is crucial for their practical applications.

Purpose of the Study:

• To analyze the representation capacity of deep hyperbolic neural networks (HNNs) with ReLU activation.
• To establish theoretical guarantees for embedding finite weighted trees into hyperbolic spaces using HNNs.

Main Methods:

• Theoretical analysis of hyperbolic neural network architectures.
• Mathematical proofs for isometric embedding capabilities.
• Derivation of upper bounds for network complexity.
• Comparison with lower bounds for Euclidean embeddings.

Main Results:

• Proved that HNNs can ɛ-isometrically embed any finite weighted tree into a hyperbolic space (d≥2, κ<0).
• Established rigorous upper bounds for the network complexity of HNNs performing this embedding.
• Demonstrated that HNN embedding complexity is independent of representation fidelity.
• Derived lower bounds for distortion in Euclidean embeddings by MLPs, showing Ω(L^1/d) for trees with L leaves.

Conclusions:

• HNNs provide a powerful and efficient method for representing tree structures in hyperbolic geometry.
• The complexity of HNN embeddings is robust to fidelity requirements, offering a significant advantage over Euclidean methods.