Abstract:
The article discusses approaches to determining the quality of machine
translation (MT) and several methods of translation quality assessment. The aim of
the article is to review a number of methods and approaches to human and
automatic assessment of MT quality. The first part of the article describes the
methods of relative human evaluation (ranking of translations) and absolute
evaluation based on penalties for errors in translation, as well as software and
algorithms that simplify human assessment. Most attention is paid to the
DQF/MQM
(Dynamic Quality Framework/Multidimensional Quality Metrics)
error typology which is not aimed at a limited subject area as the
most flexible one. The second part of the article is devoted to a review of metrics
for automatic quality assessment of MT that do not use linguistic data as well as
the correlation coefficients of human and automatic evaluation.
Keywords:
machine translation (MT), machine translation quality, rankings of translations and translation systems, MT quality metrics, error typologies, translation quality assessment, human evaluation of MT quality.
Received: 30.08.2019
Document Type:
Article
Language: Russian
Citation:
A. K. Rychikhin, “On methods of machine translation quality assessment”, Sistemy i Sredstva Inform., 29:4 (2019), 106–118
\Bibitem{Ryc19}
\by A.~K.~Rychikhin
\paper On methods of~machine translation quality assessment
\jour Sistemy i Sredstva Inform.
\yr 2019
\vol 29
\issue 4
\pages 106--118
\mathnet{http://mi.mathnet.ru/ssi676}
\crossref{https://doi.org/10.14357/08696527190410}
Linking options:
https://www.mathnet.ru/eng/ssi676
https://www.mathnet.ru/eng/ssi/v29/i4/p106
This publication is cited in the following 3 articles:
Anna Egorova, Mikhail Kruzhkov, Vitaly Nuriev, Igor Zatsman, “Interval evaluation of temporal (in)stability for neural machine translation”, Discov Artif Intell, 5:1 (2025)
V. A. Nuriev, A. Yu. Egorova, “Metody otsenki kachestva mashinnogo perevoda: sovremennoe sostoyanie”, Inform. i ee primen., 15:2 (2021), 104–111
A. Yu. Egorova, I. M. Zatsman, V. V. Kosarik, V. A. Nuriev, “Nestabilnost neironnogo mashinnogo perevoda”, Sistemy i sredstva inform., 30:2 (2020), 124–135