MF-TT model for real-time applications in smart tourism English translation under transformer-based fusion network

With the growing demand for cross-lingual real-time communication in smart tourism scenarios, the trade-off between response speed and translation quality in voice translation systems has become increasingly sensitive. Centering on this issue, this study proposes a Multimodal Fusion Transformer Transducer (MF-TT) model for real-time applications and introduces a Transformer-based fusion network into the overall architecture. Meanwhile, it incorporates a segment masking mechanism and a latency constraint loss to adapt to the non-monotonic alignment characteristics between speech and text in translation tasks. Experiments are conducted on the English-Spanish and English-German tasks of the Multilingual Speech Translation Corpus (MuST-C) dataset, as well as the Baidu Speech Translation Corpus (BSTC) Chinese-English dataset. Results show that MF-TT achieves a Bilingual Evaluation Understudy (BLEU) score of 24.32, an Average Proportion (AP) of 0.71, and an Average Lagging (AL) of 1230.8 in the English-Spanish direction. In the English-German direction, this model reaches a BLEU score of 17.64 and an AL of 1466.3, with the overall lag level lower than that of multiple comparative models. On the BSTC Chinese-English dataset, the MF-TT model's BLEU, AP, and AL are 23.02, 0.721, and 1298.4, respectively. AL significantly outperforms other comparative models, p < 0.05. The ablation experiment further verified the independent contributions of fusion network structure, latency loss function, and segment masking mechanism to real-time performance and translation stability. The results show that the proposed method has good performance and application potential in low-latency cross-lingual real-time interaction scenarios. At the same time, it has a certain reference value in scenarios such as smart tourism that require real-time voice translation.