Resformer: Time-token transformer with residual compensation for quality prediction in industrial processes

In recent years, deep learning techniques have been increasingly adopted in soft sensor modeling, with the transformer architecture demonstrating notable advantages not only in natural language processing and image analysis but also in time-series modeling. Autoencoders, known for their ability to learn compact representations of process data, have also been widely applied for feature extraction in soft sensors. However, when dealing with multivariate process data, conventional autoencoder-based models often suffer from underfitting due to persistent reconstruction errors or overfitting when the reconstruction loss converges prematurely. These issues hinder effective feature learning and limit the model's generalization capability in real-world applications. To address these challenges, this paper proposes Resformer, a novel transformer-based architecture that incorporates residual feature compensation. Resformer employs a two-stage autoencoding structure to extract both primary and secondary features and fuses them via a cross-attention mechanism to enhance representation completeness. Time tokens are used as the basic modeling units to capture spatiotemporal dependencies among process variables, which are then mapped to the target quality variable through a dedicated decoding structure. Experimental results on the Tennessee Eastman (TE) process and an industrial alkylation process dataset demonstrate that Resformer, with residual compensation and spatiotemporal feature learning, significantly outperforms recent transformer-based variants while maintaining comparable architectural complexity suitable for practical deployment.