Generating targeted paraphrases for improved translation

Abstract

Today's Statistical Machine Translation (SMT) systems require high-quality human translations for parameter tuning, in addition to large bitexts for learning the translation units. This parameter tuning usually involves generating translations at different points in the parameter space and obtaining feedback against human-authored reference translations as to how good the translations. This feedback then dictates what point in the parameter space should be explored next. To measure this feedback, it is generally considered wise to have multiple (usually 4) reference translations to avoid unfair penalization of translation hypotheses which could easily happen given the large number of ways in which a sentence can be translated from one language to another. However, this reliance on multiple reference translations creates a problem since they are labor intensive and expensive to obtain. Therefore, most current MT datasets only contain a single reference. This leads to the problem of reference sparsity. In our previously published research, we had proposed the first paraphrase-based solution to this problem and evaluated its effect on Chinese-English translation. In this article, we first present extended results for that solution on additional source languages. More importantly, we present a novel way to generate “targeted” paraphrases that yields substantially larger gains (up to 2.7 BLEU points) in translation quality when compared to our previous solution (up to 1.6 BLEU points). In addition, we further validate these improvements by supplementing with human preference judgments obtained via Amazon Mechanical Turk.