David Ifeoluwa Adelani

2023-11-14

ArXiv (preprint)

Better Quality Pre-training Data and T5 Models for African Languages

Akintunde Oladipo

Mofetoluwa Adeyemi

Orevaoghene Ahia

Abraham Toluwase Owodunni

Odunayo Ogundepo

Jimmy Lin

In this study, we highlight the importance of enhancing the quality of pretraining data in multilingual language models. Existing web crawl… (see more)s have demonstrated quality issues, particularly in the context of low-resource languages. Consequently, we introduce a new multilingual pretraining corpus for

2023-10-07

EMNLP/2023/Conference (accepted)

openreview.net

Improving Language Plasticity via Pretraining with Active Forgetting

Yihong Chen

Kelly Marchisio

Roberta Raileanu

Pontus Stenetorp

Sebastian Riedel

Mikel Artetxe

Pretrained language models (PLMs) are today the primary model for natural language processing. Despite their impressive downstream performan… (see more)ce, it can be difficult to apply PLMs to new languages, a barrier to making their capabilities universally accessible. While prior work has shown it possible to address this issue by learning a new embedding layer for the new language, doing so is both data and compute inefficient. We propose to use an active forgetting mechanism during pretraining, as a simple way of creating PLMs that can quickly adapt to new languages. Concretely, by resetting the embedding layer every K updates during pretraining, we encourage the PLM to improve its ability of learning new embeddings within limited number of updates, similar to a meta-learning effect. Experiments with RoBERTa show that models pretrained with our forgetting mechanism not only demonstrate faster convergence during language adaptation, but also outperform standard ones in a low-data regime, particularly for languages that are distant from English. Code will be available at https://github.com/facebookresearch/language-model-plasticity.

openreview.net

YORC: Yoruba Reading Comprehension dataset

Aremu Anuoluwapo

Jesujoba Oluwadara Alabi

In this paper, we create YORC: a new multi-choice Yoruba Reading Comprehension dataset that is based on Yoruba high-school reading comprehen… (see more)sion examination. We provide baseline results by performing cross-lingual transfer using existing English RACE dataset based on a pre-trained encoder-only model. Additionally, we provide results by prompting large language models (LLMs) like GPT-4.

2023-08-18

ArXiv (preprint)

Consultative engagement of stakeholders toward a roadmap for African language technologies

Kathleen Siminyu

Jade Abbott

Kọ́lá Túbọ̀sún

Aremu Anuoluwapo

Blessing Kudzaishe Sibanda

Kofi Yeboah

Masabata Mokgesi-Selinga

Frederick R. Apina

Angela Thandizwe Mthembu

Arshath Ramkilowan

Babatunde Oladimeji

2023-08-01

Patterns (published)

NollySenti: Leveraging Transfer Learning and Machine Translation for Nigerian Movie Sentiment Classification

Iyanuoluwa Shode

Jing Peng

Anna Feldman

Africa has over 2000 indigenous languages but they are under-represented in NLP research due to lack of datasets. In recent years, there hav… (see more)e been progress in developing labelled corpora for African languages. However, they are often available in a single domain and may not generalize to other domains. In this paper, we focus on the task of sentiment classification for cross-domain adaptation. We create a new dataset, Nollywood movie reviews for five languages widely spoken in Nigeria (English, Hausa, Igbo, Nigerian Pidgin, and Yoruba). We provide an extensive empirical evaluation using classical machine learning methods and pre-trained language models. By leveraging transfer learning, we compare the performance of cross-domain adaptation from Twitter domain, and cross-lingual adaptation from English language. Our evaluation shows that transfer from English in the same target domain leads to more than 5% improvement in accuracy compared to transfer from Twitter in the same language. To further mitigate the domain difference, we leverage machine translation from English to other Nigerian languages, which leads to a further improvement of 7% over cross-lingual evaluation. While machine translation to low-resource languages are often of low quality, our analysis shows that sentiment related words are often preserved.

2023-05-18

ArXiv (preprint)

AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languages

Odunayo Ogundepo

Tajuddeen Gwadabe

Clara E. Rivera

Jonathan H. Clark

Sebastian Ruder

Bonaventure F. P. Dossou

Abdoulahat Diop

Claytone Sikasote

Gilles HACHEME

Happy Buzaaba

Ignatius Ezeani

Rooweither Mabuya

Salomey Osei

Chris Emezue

Albert Kahira

Shamsuddeen Hassan Muhammad

Akintunde Oladipo

Abraham Toluwase Owodunni

Atnafu Lambebo Tonja … (see 32 more)

Iyanuoluwa Shode

Akari Asai

Tunde Oluwaseyi Ajayi

Clemencia Siro

Stephen Arthur

Mofetoluwa Adeyemi

Orevaoghene Ahia

Aremu Anuoluwapo

Oyinkansola Awosan

Chiamaka Ijeoma Chukwuneke

Bernard Opoku

A. Ayodele

Verrah Akinyi Otiende

Christine Mwase

Boyd Sinkala

Andre Niyongabo Rubungo

Daniel Ajisafe

Emeka Felix Onwuegbuzia

Habib Mbow

Emile Niyomutabazi

Eunice Mukonde

Falalu Lawan

Ibrahim Ahmad

Jesujoba Oluwadara Alabi

Martin Namukombo

Mbonu Chinedu

Mofya Phiri

Neo Putini

Ndumiso Mngoma

Priscilla A. Amuok

Ruqayya Nasir Iro

Sonia Adhiambo34

2023-05-11

ArXiv (preprint)

SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)

Shamsuddeen Hassan Muhammad

Idris Abdulmumin

Seid Muhie Yimam

Ibrahim Ahmad

Nedjma OUSIDHOUM

Abinew Ayele

Saif Mohammad

Meriem Beloucif

2023-04-13

ArXiv (preprint)

Varepsilon kú mask: Integrating Yorùbá cultural greetings into machine translation

Idris Akinade

Jesujoba Oluwadara Alabi

Clement Odoje

Dietrich Klakow

This paper investigates the performance of massively multilingual neural machine translation (NMT) systems in translating Yorùbá greetings… (see more) (kú mask), which are a big part of Yorùbá language and culture, into English. To evaluate these models, we present IkiniYorùbá, a Yorùbá-English translation dataset containing some Yorùbá greetings, and sample use cases. We analysed the performance of different multilingual NMT systems including Google and NLLB and show that these models struggle to accurately translate Yorùbá greetings into English. In addition, we trained a Yorùbá-English model by fine-tuning an existing NMT model on the training split of IkiniYorùbá and this achieved better performance when compared to the pre-trained multilingual NMT models, although they were trained on a large volume of data.

2023-03-31

ArXiv (preprint)

ε KÚ <MASK>: INTEGRATING YORÙBÁ CULTURAL GREETINGS INTO MACHINE TRANSLATION

Idris Akinade

Jesujoba Oluwadara Alabi

Clement Oyeleke Odoje

Dietrich Klakow

This paper investigates the performance of massively multilingual neural machine translation (NMT) systems in translating Yorùbá greetings… (see more) (ε kú ), which are a big part of Yorùbá language and culture, into English. To evaluate these models, we present IkiniYorùbá, a Yorùbá-English translation dataset containing some Yorùbá greetings, and sample use cases. We analysed the performance of different multilingual NMT systems including Google and NLLB and show that these models struggle to accurately translate Yorùbá greetings into English. In addition, we trained a Yorùbá-English model by finetuning an existing NMT model on the training split of IkiniYorùbá and this achieved better performance when compared to the pre-trained multilingual NMT models, although they were trained on a large volume of data.

2023-03-03

ICLR.cc/2023/Workshop/AfricaNLP (published)

openreview.net

AfriMTE and AfriCOMET: Empowering COMET to Embrace Under-resourced African Languages

Jiayi Wang

Sweta Agrawal

Ricardo Rei

Eleftheria Briakou

Marine Carpuat

Marek Masiak

Xuanli He

Sofia Bourhim

Andiswa Bukula

Muhidin A. Mohamed

Temitayo Olatoye

Hamam Mokayed

Christine Mwase

Wangui Kimotho

Foutse Yuehgoh

Aremu Anuoluwapo

Jessica Ojo

Shamsuddeen Hassan Muhammad

Salomey Osei … (see 37 more)

Abdul-Hakeem Omotayo

Chiamaka Ijeoma Chukwuneke

Perez Ogayo

Oumaima Hourrane

Salma El Anigri

Lolwethu Ndolela

Thabiso Mangwana

Shafie Abdi Mohamed

Ayinde Hassan

Oluwabusayo Olufunke Awoyomi

Lama Alkhaled

sana Sabah al-azzawi

Naome Etori

Millicent Ochieng

Clemencia Siro

Samuel Njoroge

Eric Muchiri

Wangari Kimotho

Lyse Naomi Wamba

Daud Abolade

Simbiat Ajao

Tosin Adewumi

Iyanuoluwa Shode

Ricky Macharm

Ruqayya Nasir Iro

Saheed Salahudeen Abdullahi

Stephen Moore

Bernard Opoku

Zainab Akinjobi

Abeeb Afolabi

Nnaemeka Casmir Obiefuna

Onyekachi Ogbu

Sam Brian

Verrah Akinyi Otiende

CHINEDU EMMANUEL MBONU

Toadoum Sari Sakayo

Pontus Stenetorp

Despite the progress we have recorded in scaling multilingual machine translation (MT) models and evaluation data to several under-resourced… (see more) African languages, it is difficult to measure accurately the progress we have made on these languages because evaluation is often performed on n -gram matching metrics like BLEU that often have worse correlation with human judgments. Embedding-based metrics such as COMET correlate better; however, lack of evaluation data with human ratings for under-resourced languages, complexity of annotation guidelines like Multidimensional Quality Metrics (MQM), and limited language coverage of multilingual encoders have hampered their applicability to African languages. In this paper, we address these challenges by creating high-quality human evaluation data with a simplified MQM guideline for error-span annotation and direct assessment (DA) scoring for 13 typologi-cally diverse African languages. Furthermore, we develop A FRI COMET—a COMET evaluation metric for African languages by leveraging DA training data from high-resource languages and African-centric multilingual encoder (AfroXLM-Roberta) to create the state-of-the-art evaluation metric for African languages MT with respect to Spearman-rank correlation with human judgments ( +0 . 406 ).

2023-01-01

arXiv.org (preprint)

AfriMTE and AfriCOMET: Empowering COMET to Embrace Under-resourced African Languages

Jiayi Wang

Sweta Agrawal

Ricardo Rei

Eleftheria Briakou

Marine Carpuat

Marek Masiak

Xuanli He

Sofia Bourhim

Andiswa Bukula

Muhidin A. Mohamed

Temitayo Olatoye

Hamam Mokayede

Christine Mwase

Wangui Kimotho

Foutse Yuehgoh

Anuoluwapo Aremu

Jessica Ojo

Shamsuddeen Hassan Muhammad

Salomey Osei … (see 37 more)

Abdul-Hakeem Omotayo

Chiamaka Chukwuneke

Perez Ogayo

Oumaima Hourrane

Salma El Anigri

Lolwethu Ndolela

Thabiso Mangwana

Shafie Abdi Mohamed

Ayinde Hassan

Oluwabusayo Olufunke Awoyomi

Lama Alkhaled

sana Sabah al-azzawi

Naome A. Etori

Millicent A. Ochieng

Clemencia Siro

Samuel Njoroge

Eric Muchiri

Wangari Kimotho

Lyse Naomi Wamba Momo

Daud Abolade

Simbiat Ajao

Tosin P. Adewumi

Iyanuoluwa Shode

Ricky Macharm

Ruqayya Nasir Iro

Saheed Salahudeen Abdullahi

Stephen E. Moore

Bernard Opoku

Zainab Akinjobi

Abeeb Afolabi

Nnaemeka Casmir Obiefuna

Onyekachi Ogbu

Sam Brian

V. Otiende

CHINEDU EMMANUEL MBONU

Toadoum Sari Sakayo

Pontus Stenetorp

Despite the progress we have recorded in scaling multilingual machine translation (MT) models and evaluation data to several under-resourced… (see more) African languages, it is difficult to measure accurately the progress we have made on these languages because evaluation is often performed on n -gram matching metrics like BLEU that often have worse correlation with human judgments. Embedding-based metrics such as COMET correlate better; however, lack of evaluation data with human ratings for under-resourced languages, complexity of annotation guidelines like Multidimensional Quality Metrics (MQM), and limited language coverage of multilingual encoders have hampered their applicability to African languages. In this paper, we address these challenges by creating high-quality human evaluation data with a simplified MQM guideline for error-span annotation and direct assessment (DA) scoring for 13 typologi-cally diverse African languages. Furthermore, we develop A FRI COMET—a COMET evaluation metric for African languages by leveraging DA training data from high-resource languages and African-centric multilingual encoder (AfroXLM-Roberta) to create the state-of-the-art evaluation metric for African languages MT with respect to Spearman-rank correlation with human judgments ( +0 . 406 ).

2023-01-01

arXiv.org (preprint)