Publications

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Teven Le Scao

Angela Fan

Christopher Akiki

Ellie Pavlick

Suzana Ili'c

Daniel Hesslow

Roman Castagn'e

Alexandra Luccioni

François Yvon

Matthias Gall'e

J. Tow

Alexander M. Rush

Stella Biderman

Alex Webson

Pawan Sasanka Ammanamanchi

Thomas Wang

Benoı̂t Sagot

Niklas Muennighoff

Albert Villanova del Moral

Olatunji Ruwase … (see 372 more)

Rachel Bawden

Stas Bekman

Angelina McMillan-Major

Iz Beltagy

Huu Nguyen

Lucile Saulnier

Samson Tan

Pedro Ortiz Suarez

Victor Sanh

Hugo Laurençon

Yacine Jernite

Julien Launay

Margaret Mitchell

Colin Raffel

Aaron Gokaslan

Adi Simhi

Aitor Soroa

Alham Fikri Aji

Amit Alfassy

Anna Rogers

Ariel Kreisberg Nitzav

Canwen Xu

Chenghao Mou

Chris Emezue

Christopher Klamm

Colin D. Leong

Daniel Van Strien

David Ifeoluwa Adelani

Dragomir R. Radev

Eduardo González Ponferrada

Efrat Levkovizh

Ethan Kim

Eyal Bar Natan

Francesco De Toni

Gérard Dupont

Germán Kruszewski

Giada Pistilli

Hady Elsahar

Hamza Benyamina

Hieu Tran

Ian W. Yu

Idris Abdulmumin

Isaac L. Johnson

Itziar Gonzalez-Dios

Javier de la Rosa

Jenny Chim

Jesse Dodge

Jian Zhu

Jonathan Chang

Jörg Frohberg

Josephine L. Tobing

J. Bhattacharjee

Khalid Almubarak

Kimbo Chen

Kyle Lo

Leandro Von Werra

Leon Weber

Long Phan

Loubna Ben allal

Ludovic Tanguy

Manan Dey

Manuel Romero Muñoz

Maraim Masoud

Mar'ia Grandury

Mario Šaško

Max Huang

Maximin Coavoux

Mayank Singh

Mike Tian-Jian Jiang

Vu Minh Chien

Mohammad Ali Jauhar

Mustafa Ghaleb

Nishant Subramani

Nora Kassner

Nurulaqilla Khamis

Olivier Nguyen

Omar Espejel

Ona de Gibert

Paulo Villegas

Peter Henderson

Pierre Colombo

Priscilla A. Amuok

Quentin Lhoest

Rheza Harliman

Rishi Bommasani

Roberto Luis L'opez

Rui Ribeiro

Salomey Osei

Sampo Pyysalo

Sebastian Nagel

Shamik Bose

Shamsuddeen Hassan Muhammad

Shanya Sharma Sharma

Shayne Longpre

Somaieh Nikpoor

S. Silberberg

Suhas Pai

Sydney Zink

Tiago Timponi Torrent

Timo Schick

Tristan Thrush

Valentin Danchev

Vassilina Nikoulina

Veronika Laippala

Violette Lepercq

Vrinda Prabhu

Zaid Alyafeai

Zeerak Talat

Arun Raja

Benjamin Heinzerling

Chenglei Si

Elizabeth E Salesky

Sabrina J. Mielke

Wilson Y. Lee

Abheesht Sharma

Andrea Santilli

Antoine Chaffin

Arnaud Stiegler

Debajyoti Datta

Eliza Szczechla

Gunjan Chhablani

Han Wang

Harshit Pandey

Hendrik. Strobelt

Jason Alan Fries

Jos Rozen

Leo Gao

Lintang A. Sutawika

M. Saiful Bari

Maged S. Al-shaibani

Matteo Manica

Nihal V. Nayak

Ryan Teehan

Samuel Albanie

Sheng Shen

Srulik Ben-David

Stephen H. Bach

Taewoon Kim

T. Bers

Thibault F'evry

Trishala Neeraj

Urmish Thakker

Vikas Raunak

Xiang Tang

Zheng Xin Yong

Zhiqing Sun

Shaked Brody

Y. Uri

Hadar Tojarieh

Adam Roberts

Hyung Won Chung

Jaesung Tae

Jason Phang

Ofir Press

Conglong Li

D. Narayanan

Hatim Bourfoune

Jared Casper

Jeff Rasley

Max Ryabinin

Mayank Mishra

Minjia Zhang

Mohammad Shoeybi

Myriam Peyrounette

Nicolas Patry

Nouamane Tazi

Omar Sanseviero

Patrick von Platen

Pierre Cornette

Pierre Franccois Lavall'ee

R'emi Lacroix

Samyam Rajbhandari

Sanchit Gandhi

Shaden Smith

St'ephane Requena

Suraj Patil

Tim Dettmers

Ahmed Baruwa

Amanpreet Singh

Anastasia Cheveleva

Anne-Laure Ligozat

Arjun Subramonian

Aur'elie N'ev'eol

Charles Lovering

Dan Garrette

D. Tunuguntla

Ehud Reiter

Ekaterina Taktasheva

E. Voloshina

Eli Bogdanov

Genta Indra Winata

Hailey Schoelkopf

Jan-Christoph Kalo

Jekaterina Novikova

Jessica Zosa Forde

Xiangru Tang

Jungo Kasai

Ken Kawamura

Liam Hazan

Marine Carpuat

Miruna-adriana Clinciu

Najoung Kim

Newton Cheng

O. Serikov

Omer Antverg

Oskar van der Wal

Rui Zhang

Ruochen Zhang

Sebastian Gehrmann

Shachar Mirkin

S. Pais

Tatiana Shavrina

Thomas Scialom

Tian Yun

Tomasz Limisiewicz

Verena Teresa Rieser

Vitaly Protasov

V. Mikhailov

Yada Pruksachatkun

Yonatan Belinkov

Zachary Bamberger

Zdenˇek Kasner

Zdeněk Kasner

A. Pestana

Amir Feizpour

Ammar Khan

Amy Faranak

A. Santos

Anthony Hevia

Antigona Unldreaj

Arash Aghagol

Arezoo Abdollahi

Aycha Tammour

Azadeh Hajihosseini

Bahareh Behroozi

Benjamin A. Ajibade

B. Saxena

Carlos Muñoz Ferrandis

Danish Contractor

D. Lansky

Davis David

Douwe Kiela

Duong Anh Nguyen

Edward Chwee Kheng. Tan

Emi Baylor

Ezinwanne Ozoani

F. Mirza

Frankline Ononiwu

Habib Rezanejad

H.A. Jones

Indrani Bhattacharya

Irene Solaiman

Irina Sedenko

Isar Nejadgholi

J. Passmore

Joshua Seltzer

Julio Bonis Sanz

Karen Fort

Livia Macedo Dutra

Mairon Samagaio

Maraim Elbadri

Margot Mieskes

Marissa Kumar Gerchick

Martha Akinlolu

Michael McKenna

Mike Qiu

M. Ghauri

Mykola Burynok

Nafis Abrar

Nazneen Fatema Rajani

Nour Elkott

N. Fahmy

Olanrewaju Samuel

Ran An

R. Kromann

Ryan Hao

Samira Hassan Alizadeh

Sarmad Shubber

Silas L. Wang

Sourav Roy

Sylvain Viguier

Thanh-Cong Le

Tobi Oyebade

T. Le

Yoyo Yang

Zach Nguyen

Abhinav R. Kashyap

Alfredo Palasciano

Alison Callahan

Anima Shukla

Antonio Miranda-Escalada

Ayush Singh

Benjamin Beilharz

Bo Wang

Caio Matheus Fonseca De Brito

Chenxi Zhou

Chirag Jain

Chuxin Xu

Cl'ementine Fourrier

Daniel Le'on Perin'an

Daniel Molano

Dian Yu

Enrique Manjavacas

Fabio Barth

Florian Fuhrimann

Gabriel Altay

Giyaseddin Bayrak

Gully Burns

Helena U. Vrabec

I. Bello

Isha Dash

J. Kang

John Michael Giorgi

Jonas Golde

J. Posada

Karthi Sivaraman

Lokesh Bulchandani

Li Li

Luisa Shinzato

Madeleine Hahn de Bykhovetz

Maiko Takeuchi

Marc Pamies

M. A. Castillo

Marianna Nezhurina

Mario Sanger

Matthias Samwald

Michael Joseph Cullan

Michael Weinberg

Michiel De Wolf

Mina Mihaljcic

Minna Liu

Moritz Freidank

Myungsun Kang

Natasha Seelam

Nathan Dahlberg

Nicholas Michio Broad

Nikolaus Muellner

Pascale Fung

Patricia Haller

Ramya Chandrasekhar

Patrick Haller

Renata Eisenberg

Robert Martin

Rodrigo Canalli

Rosaline Su

Ruisi Su

Samuel Cahyawijaya

Samuele Garda

Shlok S Deshmukh

Shubhanshu Mishra

Sid Kiblawi

Simon Ott

Sinee Sang-aroonsiri

Srishti Kumar

Stefan Schweter

Sushil Pratap Bharati

Tanmay Laud

Th'eo Gigant

Tomoya Kainuma

Wojciech Kusa

Yanis Labrak

Yashasvi Bajaj

Yash Venkatraman

Yifan Xu

Ying Xu

Yu Xu

Zhijun Tan

Zhongli Xie

Zifan Ye

Mathilde Le Bras

Younes Belkada

Thomas Wolf

2022-11-08

ArXiv (preprint)

doi.org

arxiv.org

Flaky Performances when Pretraining on Relational Databases

Shengchao Liu

David Vázquez

Jian Tang

Pierre-Andre Noel

2022-11-08

ArXiv (preprint)

doi.org

arxiv.org

PaReco: patched clones and missed patches among the divergent variants of a software family

Poedjadevie Kadjel Ramkisoen

John Businge

Brent van Bladel

Alexandre Decan

Serge Demeyer

Coen De Roover

Foutse Khomh

Re-using whole repositories as a starting point for new projects is often done by maintaining a variant fork parallel to the original. Howev… (see more)er, the common artifacts between both are not always kept up to date. As a result, patches are not optimally integrated across the two repositories, which may lead to sub-optimal maintenance between the variant and the original project. A bug existing in both repositories can be patched in one but not the other (we see this as a missed opportunity) or it can be manually patched in both probably by different developers (we see this as effort duplication). In this paper we present a tool (named PaReCo) which relies on clone detection to mine cases of missed opportunity and effort duplication from a pool of patches. We analyzed 364 (source to target) variant pairs with 8,323 patches resulting in a curated dataset containing 1,116 cases of effort duplication and 1,008 cases of missed opportunities. We achieve a precision of 91%, recall of 80%, accuracy of 88%, and F1-score of 85%. Furthermore, we investigated the time interval between patches and found out that, on average, missed patches in the target variants have been introduced in the source variants 52 weeks earlier. Consequently, PaReCo can be used to manage variability in “time” by automatically identifying interesting patches in later project releases to be backported to supported earlier releases.

2022-11-08

Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (published)

doi.org

A connectomics-based taxonomy of mammals

Laura E Suarez

Yossi Yovel

Martijn P van den Heuvel

Olaf Sporns

Yaniv Assaf

Guillaume Lajoie

Bratislav Misic

Mammalian taxonomies are conventionally defined by morphological traits and genetics. How species differ in terms of neural circuits and whe… (see more)ther inter-species differences in neural circuit organization conform to these taxonomies is unknown. The main obstacle to the comparison of neural architectures has been differences in network reconstruction techniques, yielding species-specific connectomes that are not directly comparable to one another. Here, we comprehensively chart connectome organization across the mammalian phylogenetic spectrum using a common reconstruction protocol. We analyse the mammalian MRI (MaMI) data set, a database that encompasses high-resolution ex vivo structural and diffusion MRI scans of 124 species across 12 taxonomic orders and 5 superorders, collected using a unified MRI protocol. We assess similarity between species connectomes using two methods: similarity of Laplacian eigenspectra and similarity of multiscale topological features. We find greater inter-species similarities among species within the same taxonomic order, suggesting that connectome organization reflects established taxonomic relationships defined by morphology and genetics. While all connectomes retain hallmark global features and relative proportions of connection classes, inter-species variation is driven by local regional connectivity profiles. By encoding connectomes into a common frame of reference, these findings establish a foundation for investigating how neural circuits change over phylogeny, forging a link from genes to circuits to behaviour.

2022-11-06

eLife (published)

doi.org

A debriefing tool to acquire non-technical skills in trauma courses

Fabio Botelho

Jason M. Harley

Natalie Yanchar

Simone Abib

Ilana Bank

Dan Poenaru

2022-11-06

Surgery Open Science (published)

doi.org

Dissociable brain structural asymmetry patterns reveal unique phenome-wide profiles

Karin Saltoun

Ralph Adolphs

Lynn K. Paul

Vaibhav Sharma

Joern Diedrichsen

B. T. Thomas Yeo

Danilo Bzdok

2022-11-06

Nature Human Behaviour (unknown)

doi.org

Object-centric causal representation learning

Amin Mansouri

Jason Hartford

Kartik Ahuja

Yoshua Bengio

2022-11-06

NeurIPS.cc/2022/Workshop/NeurReps (poster)

openreview.net

Posterior samples of source galaxies in strong gravitational lenses with score-based priors

Alexandre Adam

Adam Coogan

Nikolay Malkin

Ronan Legin

Laurence Perreault-Levasseur

Yashar Hezaveh

Yoshua Bengio

Inferring accurate posteriors for high-dimensional representations of the brightness of gravitationally-lensed sources is a major challenge,… (see more) in part due to the difficulties of accurately quantifying the priors. Here, we report the use of a score-based model to encode the prior for the inference of undistorted images of background galaxies. This model is trained on a set of high-resolution images of undistorted galaxies. By adding the likelihood score to the prior score and using a reverse-time stochastic differential equation solver, we obtain samples from the posterior. Our method produces independent posterior samples and models the data almost down to the noise level. We show how the balance between the likelihood and the prior meet our expectations in an experiment with out-of-distribution data.

2022-11-06

ArXiv (preprint)

doi.org

arxiv.org

Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayes

Mizu Nishikawa-Toomey

Tristan Deleu

Jithendaraa Subramanian

Laurent Charlin

Yoshua Bengio

Bayesian causal structure learning aims to learn a posterior distribution over directed acyclic graphs (DAGs), and the mechanisms that defin… (see more)e the relationship between parent and child variables. By taking a Bayesian approach, it is possible to reason about the uncertainty of the causal model. The notion of modelling the uncertainty over models is particularly crucial for causal structure learning since the model could be unidentifiable when given only a finite amount of observational data. In this paper, we introduce a novel method to jointly learn the structure and mechanisms of the causal model using Variational Bayes, which we call Variational Bayes-DAG-GFlowNet (VBG). We extend the method of Bayesian causal structure learning using GFlowNets to learn not only the posterior distribution over the structure, but also the parameters of a linear-Gaussian model. Our results on simulated data suggest that VBG is competitive against several baselines in modelling the posterior over DAGs and mechanisms, while offering several advantages over existing methods, including the guarantee to sample acyclic graphs, and the flexibility to generalize to non-linear causal mechanisms.

2022-11-03

arXiv (preprint)

doi.org

arxiv.org

A circulating proteome-informed prognostic model of COVID-19 disease activity that relies on routinely available clinical laboratories

William Ma

Antoine Soulé

Karine Tremblay

Simon Rousseau

Amin Emad

A minority of people infected with SARS-CoV-2 will develop severe COVID-19 disease. To help physicians predict who is more likely to require… (see more) admission to ICU, we conducted an unsupervised stratification of the circulating proteome that identified six endophenotypes (EPs) among 731 SARS-CoV-2 PCR-positive hospitalized participants in the Biobanque Québécoise de la COVID-19, with varying degrees of disease severity and times to intensive care unit (ICU) admission. One endophenotype, EP6, was associated with a greater proportion of ICU admission, ventilation support, acute respiratory distress syndrome (ARDS) and death. Clinical features of EP6 included increased levels of C-reactive protein, D-dimers, interleukin-6, ferritin, soluble fms-like tyrosine kinase-1, elevated neutrophils, and depleted lymphocytes, whereas another endophenotype (EP5) was associated with cardiovascular complications, congruent with elevated blood biomarkers of cardiovascular disease like N-terminal pro B-type natriuretic peptide (NT-proBNP), Growth Differentiation Factor-15 (GDF-15), and Troponin T. Importantly, a prognostic model solely based on clinical laboratory measurements was developed and validated on 903 patients that generalizes the EPs to new patients recruited across all pandemic waves (2020-2022) and create new opportunities for automated identification of high-risk groups in the clinic. Thus, this novel way to address pathogenesis that leverages detailed phenotypic information but relies on routinely available information in the clinic to favor translation may find applications in other diseases beyond COVID-19.

2022-11-03

medRxiv (preprint)

doi.org

Representational ethical model calibration

Robert Carruthers

Isabel Straw

James K. Ruffle

Daniel Herron

Amy Nelson

Danilo Bzdok

Delmiro Fernandez-Reyes

Geraint Rees