Seminar: "Idiom Processing in Transformer, a Translation Case Study"
Abstract
In contrast to compositional literal expressions, idioms’ meanings do not directly follow from their parts, and this non-compositionality poses a challenge for neural machine translation (NMT). NMT models tend to translate idioms word for word, which is inappropriate if the source and target language do not share that particular idiom. In this talk, I will elaborate on what makes idiom translation a hard task for NMT systems to perform and a challenging task for NMT researchers to analyse. We propose a simple method that allows for the subcategorisation of idiom translations as word-for-word translations and paraphrases. Using this method, I first examine how idiom translations develop over the course of training. Secondly, I elaborate on an analysis of the Transformer’s internal mechanisms by diving into how (cross-)attention and hidden representations change when the model is presented with idioms instead of literal phrases.
Invited Speaker: Verna Dankers (University of Edinburgh)
The talk discusses experiments from the following two articles:
- Dankers, V., Bruni, E., & Hupkes, D. (2022, May). The Paradox of the Compositionality of Natural Language: A Neural Machine Translation Case Study. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 4154-4175).
- Dankers, V., Lucas, C., & Titov, I. (2022, May). Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 3608-3626).
