Machine learning-driven alignment architecture of heterogeneous data with transient varying semantics

Via cross-correlation algorithms or synchronized acquisition of signals, the alignment of heterogeneous data with unknown semantic time shifts and intermittent semantic variations cannot be solved. The shift is caused by different data acquisition principles of sensors, different response discrimination principles using heterogeneous data, etc. Here, we report an unsupervised alignment architecture with a supervised learning model as the kernel to overcome the limitations of brain cognition, perception, and storage in aligning complex heterogeneous data. A set of data with a time shift is input into the kernel model of the architecture to predict the semantic labels, features or continuous values corresponding to another set of data. The time shift corresponding to the maximum testing accuracy or the minimum mean squared error is the alignment parameter for the two heterogeneous datasets. This architecture is expected to serve as a preprocessing step for semantic mining of signals and for information fusion.