AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages

Abteen Ebrahimi^{a b}, Manuel Mager^{a b}, Arturo Oncevay^{b c}, Vishrav Chaudhary^{d e}, Luis Chiruzzo^{a b}, Angela Fan^{a f g}, John E. Ortega^{b c}, Ricardo Ramos^{b h}, Annette Rios^{b i}, Iván Meza^b, Gustavo A. Giménez-Lugo^{a b}, Elisabeth Mager^{b c}, Graham Neubig^{b i}, Alexis Palmer^{a b}, Rolando Coto‐Solano^b, Thang Vu^{b c}, Katharina Kann^{b i}

1. ^aUniversity of Stuttgart
2. ^bCarnegie Mellon University Dartmouth College
3. ^cNew York University Universidad de la República , Uruguay
4. ^dCarnegie Mellon University Dartmouth College Microsoft Turing ∇ Facebook AI Research Ω
5. ^eMicrosoft
6. ^fFacebook AI Research Ω New York University Universidad de la República, Uruguay
7. ^gη Universidad Tecnológica de Tlaxcala Universidad Nacional Autónoma de México ♣ Universidade Tecnológica Federal do Paraná ♦ University of Colorado Boulder ♥ University of Edinburgh ♠ University of Stuttgart ψ University of Zurich
8. ^hUniversidad Tecnológica de Tlaxcala Universidad Nacional Autónoma de México ♣ Universidade Tecnológica Federal do Paraná ♦ University of Colorado Boulder ♥ University of Edinburgh ♠
9. ⁱψ University of Zurich