A lightweight Chinese-English translation model integrating compressed BERT attention and phrase discard mechanism

Chinese-English machine translation based on neural network model strictly adopts the sequential modeling method of encoder-decoder. However, this traditional method cannot make effective use of syntactic information and linguistic hierarchy information. Therefore, to integrate syntactic structure information into Chinese-English machine translation to improve its translation performance, this paper proposes a new Chinese-English machine translation method based on graph convolutional network and BERT (Bidirectional Encoder Representation from Transformers) knowledge enhancement. In this work, we present an enhanced approach to neural machine translation that integrates multiple techniques to improve translation quality. The multi-BERT context is first compressed and aligned into the semantic space of the translation model using learnable compression vectors. This alignment ensures that the rich contextual information from BERT is effectively utilized within our translation framework. At the end of source language, we employ a dual encoder to encode both the source sentence and its syntactic dependency tree, thereby capturing both lexical and structural information. To further enrich the source-side semantic representation, the compression vector is concatenated with the input vector of the encoder. Additionally, we introduce a phrase discard mechanism that randomly discards target phrases during training. This mechanism enhances the model's robustness against mistranslated phrases, thereby reducing their impact on subsequent phrase translations. Experiments on NIST dataset demonstrate the effectiveness of our proposed lightweight Chinese-English translation method. Different from general-purpose large chatbot models (e.g., ChatGPT) with high computing costs, this model achieves 39.68 BLEU with low parameters, solving issues of low-resource scenarios and phrase mistranslation. It offers a novel lightweight paradigm for private-oriented translation chatbots, outperforming the baseline Transformer (35.75 BLEU) significantly.