BERGAMOT-LATTE Submissions for the WMT20 Quality Estimation Shared Task
Marina Fomichevaa b, Shuo Sunb c, Lisa Yankovskayad e, Frédéric Blaina b, Vishrav Chaudharyf, Mark Fisheld e, Francisco Guzmáng, Lucia Speciaa h i
- aUniversity of Sheffield
- bUniversity of Sheffield, Sheffield, United Kingdom
- cJohns Hopkins University
- dUniversity of Tartu
- eUniversity of Tartu, Tartu, Estonia
- fUniversity of Wolverhampton, Wolverhampton, United Kingdom
- gMeta (Israel), Tel Aviv, Israel
- hImperial College London , 5 Facebook AI 1
- iImperial College London, London, United Kingdom
Papers open inside SciRah from the original publisher or repository; SciRah never hosts them.
Abstract
This paper presents our submission to the WMT2020 Shared Task on Quality Estimation (QE) 1 .We participate in Task 1 and Task 2 focusing on sentence-level prediction.We explore (a) a black-box approach to QE based on pre-trained representations; and (b) glassbox approaches that leverage various indicators that can be extracted from the neural MT systems.In addition to training a feature-based regression model using glass-box quality indicators, we also test whether they can be used to predict MT quality directly with no supervision.We assess our systems in a multilingual setting and show that both types of approaches generalise well across languages.Our black-box QE models tied for the winning submission in four out of seven language pairs in Task 1, thus demonstrating very strong performance.The glass-box approaches also performed competitively, representing a lightweight alternative to the neural-based models.
TL;DRThe authors' black-box QE models tied for the winning submission in four out of seven language pairs in Task 1, thus demonstrating very strong performance, and the glass-box approaches also performed competitively, representing a light-weight alternative to the neural-based models. (Semantic Scholar)