Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
Avi Singh
John D Co-Reyes
Rishabh Agarwal
Ankesh Anand
Piyush Patil
Xavier Garcia
Peter J. Liu
James Harrison
Jaehoon Lee
Kelvin Xu
Aaron T Parisi
Abhishek Kumar
A. Alemi
Alex Rizkowsky
Azade Nova
Ben Adlam
Bernd Bohnet
Hanie Sedghi
Gamaleldin Fathy Elsayed
Igor Mordatch … (see 21 more)
Isabelle Simpson
Izzeddin Gur
Jasper Snoek
Jeffrey Pennington
Jiri Hron
Kathleen Kenealy
Kevin Swersky
Kshiteej Mahajan
Laura Culp
Lechao Xiao
Maxwell Bileschi
Noah Constant
Roman Novak
Rosanne Liu
Tris Brian Warkentin
Yundi Qian
Ethan Dyer
Behnam Neyshabur
Jascha Sohl-Dickstein
Yamini Bansal
Noah Fiedel
Fine-tuning language models~(LMs) on human-generated data remains a prevalent practice. However, the performance of such models is often lim… (see more)ited by the quantity and diversity of high-quality human data. In this paper, we explore whether we can go beyond human data on tasks where we have access to scalar feedback, for example, on math problems where one can verify correctness. To do so, we investigate a simple self-training method based on expectation-maximization, which we call ReST
2023-12-01
arXiv (published)
doi.org
arxiv.org
Brain decoding of the Human Connectome Project tasks in a dense individual fMRI dataset
Shima Rastegarnia
Marie St-Laurent
Elizabeth DuPre
Basile Pinsard
Lune Bellec
2023-12-01
NeuroImage (published)
doi.org
Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model
Parishad BehnamGhader
Santiago Miret
Siva Reddy
Augmenting pretrained language models with retrievers to select the supporting documents has shown promise in effectively solving common NLP… (see more) problems, including language modeling and question answering, in an interpretable way. In this paper, we first study the strengths and weaknesses of different retriever-augmented language models (REALM,
2023-12-01
Findings of the Association for Computational Linguistics: EMNLP 2023 (published)
doi.org
openreview.net
Cross-lingual Open-Retrieval Question Answering for African Languages
Odunayo Ogundepo
Tajuddeen Gwadabe
Clara E. Rivera
Jonathan H. Clark
Sebastian Ruder
David Ifeoluwa Adelani
Bonaventure F. P. Dossou
Abdou Aziz DIOP
Claytone Sikasote
Gilles HACHEME
Happy Buzaaba
Ignatius Ezeani
Rooweither Mabuya
Salomey Osei
Chris Emezue
Albert Kahira
Shamsuddeen Hassan Muhammad
Akintunde Oladipo
Abraham Toluwase Owodunni
Atnafu Lambebo Tonja … (see 32 more)
Iyanuoluwa Shode
Akari Asai
Aremu Anuoluwapo
Ayodele Awokoya
Bernard Opoku
Chiamaka Ijeoma Chukwuneke
Christine Mwase
Clemencia Siro
Stephen Arthur
Oyinkansola Awosan
Tunde Oluwaseyi Ajayi
Verrah Akinyi Otiende
Andre Niyongabo Rubungo
Boyd Sinkala
Daniel Ajisafe
Emeka Felix Onwuegbuzia
Falalu Lawan
Ibrahim Ahmad
Jesujoba Oluwadara Alabi
Habib Mbow
CHINEDU EMMANUEL MBONU
Emile Niyomutabazi
Mofetoluwa Adeyemi
Eunice Mukonde
Mofya Phiri
Orevaoghene Ahia
Ruqayya Nasir Iro
Sonia Adhiambo
Martin Namukombo
Neo Putini
Ndumiso Mngoma
Priscilla A. Amuok
2023-12-01
Findings of the Association for Computational Linguistics: EMNLP 2023 (published)
doi.org
openreview.net
Cross-lingual Open-Retrieval Question Answering for African Languages
Odunayo Ogundepo
Tajuddeen Gwadabe
Clara E. Rivera
Jonathan H. Clark
Sebastian Ruder
David Ifeoluwa Adelani
Bonaventure F. P. Dossou
Abdou Aziz DIOP
Claytone Sikasote
Gilles Q. Hacheme
Happy Buzaaba
Ignatius Majesty Ezeani
Rooweither Mabuya
Salomey Osei
Chris Emezue
Albert Njoroge Kahira
Shamsuddeen Hassan Muhammad
Akintunde Oladipo
Abraham Toluwase Owodunni
Atnafu Lambebo Tonja … (see 24 more)
Iyanuoluwa Shode
Akari Asai
Aremu Anuoluwapo
Ayodele Awokoya
Bernard Opoku
Chiamaka Ijeoma Chukwuneke
Christine Mwase
Clemencia Siro
Stephen Arthur
Tunde Oluwaseyi Ajayi
V. Otiende
Andre Niyongabo Rubungo
B. Sinkala
Daniel A. Ajisafe
Emeka Onwuegbuzia
Falalu Lawan
Ibrahim Ahmad
Jesujoba Alabi
CHINEDU EMMANUEL MBONU
Mofetoluwa Adeyemi
Mofya Phiri
Orevaoghene Ahia
Ruqayya Nasir Iro
Sonia Adhiambo
2023-12-01
Findings of the Association for Computational Linguistics: EMNLP 2023 (published)
doi.org
Current AI applications in neurology: Brain imaging
Tal Arbel
Joshua D. Durso-Finley
Jean-Pierre R. Falet
Raghav Mehta
Douglas Arnold
Nick Pawlowski
2023-12-01
Journal of the Neurological Sciences (published)
doi.org
DiPS: Discriminative Pseudo-Label Sampling with Self-Supervised Transformers for Weakly Supervised Object Localization
Shakeeb Murtaza
Soufiane Belharbi
Marco Pedersoli
Aydin Sarraf
Eric Granger
2023-12-01
Image and Vision Computing (published)
doi.org
arxiv.org
From physics to sentience: Deciphering the semantics of the free-energy principle and evaluating its claims: Comment on "Path integrals, particular kinds, and strange things" by Karl Friston et al.
Zahra Sheikhbahaee
Adam Safron
Casper Hesp
Guillaume Dumas
2023-12-01
Physics of Life Reviews (published)
doi.org
arxiv.org
Growth of TiO2 single crystals by the Verneuil method at different gas flow ratio
Xudong Liu
Hanshu Ma
Wei Wang
Yongqi Hu
Xudong Sun
2023-12-01
Journal of Crystal Growth (published)
doi.org
Large language models: What could they do for neurology?
Guillaume Lajoie
2023-12-01
Journal of the Neurological Sciences (published)
doi.org
A large-scale exploratory study of android sports apps in the google play store
Bhagya Chembakottu
Heng Li
Foutse Khomh
2023-12-01
Information and Software Technology (published)
doi.org
arxiv.org
Measuring the Knowledge Acquisition-Utilization Gap in Pretrained Language Models
Amirhossein Kazemnejad
Mehdi Rezagholizadeh
Prasanna Parthasarathi
Sarath Chandar
2023-12-01
Findings of the Association for Computational Linguistics: EMNLP 2023 (published)
doi.org
openreview.net