Neural Machine Translation with an Awareness of Semantic Similarity - Li, Jiaxin; Jin, Rize; Paik, Joon Young; Chung, Tae Sun (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2024)
Machine translation requires that source and target sentences have identical semantics. Previous neural machine translation (NMT) models have implicitly achieved this requirement using cross-entropy loss. In this paper, we propose a sentence Semantic-aware Machine Translation model (SaMT) which explicitly addresses the issue of semantic similarity between sentences in translation. SaMT integrates a Sentence-Transformer into a Transformer-based encoder-decoder to estimate semantic similarity between source and target sentences. Our model enables translated sentences to maintain the semantics of source sentences, either by using the Sentence-Transformer alone or by including an additional linear layer in the decoder. To achieve high-quality translation, we employ vertical and horizontal feature fusion methods, which capture rich features from sentences during translation. Experimental results showed a BLEU score of 36.41 on the IWSLT2014 German→ English dataset, validating the efficacy of incorporating sentence-level semantic knowledge and using the two orthogonal fusion methods. Our code is available at https://github.com/aaa559/SaMT-master.
Keywords: Cross entropy; Entropy loss; Fusion mechanism; Machine translation models; Machine translations; Multi-branch attention; Semantic similarity; Semantic-aware; Sentence semantic-aware; Transformer; Fusion Mechanism; Multi-branch Attention; Sentence Semantic-aware; Transformer