Multimodal Manifold Analysis by Simultaneous Diagonalization of Laplacians

We construct an extension of spectral and diffusion geometry to multiple modalities through simultaneous diagonalization of Laplacian matrices. This naturally extends classical data analysis tools based on spectral geometry, such as diffusion maps and spectral clustering. We provide several synthetic and real examples of manifold learning, object classification, and clustering, showing that the joint spectral geometry better captures the inherent structure of multi-modal data. We also show the relation of many previous approaches for multimodal manifold analysis to our framework.