Evaluation of Machine Translation - Round-trip Translation

Round-trip Translation

A typical way for lay people to assess machine translation quality is to translate from a source language to a target language and back to the source language with the same engine. Though intuitively this seem a good method of evaluation, it has been shown that round-trip translation is a "poor predictor of quality". The reason why it is such a poor predictor of quality is reasonably intuitive. A round-trip translation is not testing one system, but two systems: the language pair of the engine for translating in to the target language, and the language pair translating back from the target language.

Consider the following examples of round-trip translation performed from English to Italian and Portuguese from Somers (2005):

Original text	Select this link to look at our home page.
Translated	Selezioni questo collegamento per guardare il nostro Home Page.
Translated back	Selections this connection in order to watch our Home Page.

Original text	Tit for tat
Translated	Melharuco para o tat
Translated back	Tit for tat

In the first example, where the text is translated into Italian then back into English—the English text is significantly garbled, but the Italian is a serviceable translation. In the second example, the text translated back into English is perfect, but the Portuguese translation is meaningless.

While round-trip translation may be useful to generate a "surplus of fun," the methodology is deficient for serious study of machine translation quality.