Model Invertibility Regularization: Sequence Alignment With or Without Parallel Data
Tomer Levinboima b, Ashish Vaswanic d, David Chianga b
- aDept. of Computer Science and Engineering University of Notre Dame
- bUniversity of Notre Dame
- cInformation Sciences Institute University of Southern California
- dInformation Sciences Institute#TAB#
Papers open inside SciRah from the original publisher or repository; SciRah never hosts them.
Abstract
We present Model Invertibility Regularization (MIR), a method that jointly trains two directional sequence alignment models, one in each direction, and takes into account the invertibility of the alignment task.By coupling the two models through their parameters (as opposed to through their inferences, as in Liang et al.'s Alignment by Agreement (ABA), and Ganchev et al.'s Posterior Regularization (PostCAT)), our method seamlessly extends to all IBMstyle word alignment models as well as to alignment without parallel data.Our proposed algorithm is mathematically sound and inherits convergence guarantees from EM.We evaluate MIR on two tasks: (1) On word alignment, applying MIR on fertility based models we attain higher F-scores than ABA and PostCAT.(2) On Japanese-to-English backtransliteration without parallel data, applied to the decipherment model of Ravi and Knight, MIR learns sparser models that close the gap in whole-name error rate by 33% relative to a model trained on parallel data, and further, beats a previous approach by Mylonakis et al.
TL;DRModel Invertibility Regularization is presented, a method that jointly trains two directional sequence alignment models, one in each direction, and takes into account the invertibility of the alignment task, and seamlessly extends to all IBMstyle word alignment models as well as to alignment without parallel data. (Semantic Scholar)