Publications

SoundChoice: Grapheme-to-Phoneme Models with Semantic Disambiguation
Artem Ploujnikov
Mirco Ravanelli
End-to-end speech synthesis models directly convert the input characters into an audio representation (e.g., spectrograms). Despite their im… (see more)pressive performance, such models have difficulty disambiguating the pronunciations of identically spelled words. To mitigate this issue, a separate Grapheme-to-Phoneme (G2P) model can be employed to convert the characters into phonemes before synthesizing the audio. This paper proposes SoundChoice, a novel G2P architecture that processes entire sentences rather than operating at the word level. The proposed architecture takes advantage of a weighted homograph loss (that improves disambiguation), exploits curriculum learning (that gradually switches from word-level to sentence-level G2P), and integrates word embeddings from BERT (for further performance improvement). Moreover, the model inherits the best practices in speech recognition, including multi-task learning with Connectionist Temporal Classification (CTC) and beam search with an embedded language model. As a result, SoundChoice achieves a Phoneme Error Rate (PER) of 2.65% on whole-sentence transcription using data from LibriSpeech and Wikipedia. Index Terms grapheme-to-phoneme, speech synthesis, text-tospeech, phonetics, pronunciation, disambiguation.
2022-09-17
Interspeech 2022 (published)
doi.org
arxiv.org
Intervertebral Disc Labeling With Learning Shape Information, A Look Once Approach
Reza Azad
Moein Heidari
Julien Cohen-Adad
Ehsan Adeli
Dorit Merhof
Accurate and automatic segmentation of intervertebral discs from medical images is a critical task for the assessment of spine-related disea… (see more)ses such as osteoporosis, vertebral fractures, and intervertebral disc herniation. To date, various approaches have been developed in the literature which routinely relies on detecting the discs as the primary step. A disadvantage of many cohort studies is that the localization algorithm also yields false-positive detections. In this study, we aim to alleviate this problem by proposing a novel U-Net-based structure to predict a set of candidates for intervertebral disc locations. In our design, we integrate the image shape information (image gradients) to encourage the model to learn rich and generic geometrical information. This additional signal guides the model to selectively emphasize the contextual representation and suppress the less discriminative features. On the post-processing side, to further decrease the false positive rate, we propose a permutation invariant 'look once' model, which accelerates the candidate recovery procedure. In comparison with previous studies, our proposed approach does not need to perform the selection in an iterative fashion. The proposed method was evaluated on the spine generic public multi-center dataset and demonstrated superior performance compared to previous work. We have provided the implementation code in https://github.com/rezazad68/intervertebral-lookonce
2022-09-15
Predictive Intelligence in Medicine (published)
doi.org
arxiv.org
Open Design and Validation of a Reproducible Videogame Controller for MRI and MEG
Harel Yann
Cyr André
Boyle Julie
Pinsard Basile
Bernard Jeremy
Fourcade Marie-France
Aggarwal Himanshu
Ponce Ana Fernanda
Thirion Bertrand
Jerbi Karim
Bellec Pierre

Playing video games in a neuroimaging environment is both scientifically promising and technically challenging. Primary among these chall… (see more)enges is the need to use scanner-compatible devices to register player inputs, which limits the type of games that can be comfortably played in a scanner and often reduces the ecological validity of video game tasks. In this paper, we introduce an MRI- and MEG-compatible video game controller that is made exclusively of 3D-printed and commercially available parts, and we release the design files and documentations in the goal of making its production accessible to any research team with minimal engineering resources. In line with the open science philosophy, we made this work accessible under an Open Source Hardware license that aims to promote accessibility and reproducibility. Additionally, we validated the responsiveness and scanner-compatibility of our controller by comparing it to a reference, non-MRI compatible controller, and by assessing the quality of the data recorded with and without the use of the said controller. The analysis of response latencies showed reliable button press accuracies. A higher latency was detected on button releases, both for long and short button presses although this effect was small enough as not to affect gameplay in most situations. Analysis of subject motion during fMRI recordings of various tasks showed that the use of our controller didn’t increase the amount of motion produced. We hope that this tool will stimulate further neuroimaging studies of video games tasks by improving both their accessibility and their validity.

2022-09-15
PsyArXiv (preprint)
doi.org
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Hugo Laurençon
Lucile Saulnier
Thomas Wang
Christopher Akiki
Albert Villanova del Moral
Teven Le Scao
Leandro Von Werra
Chenghao Mou
Eduardo González Ponferrada
Huu Nguyen
Jörg Frohberg
Mario Šaško
Quentin Lhoest
Angelina McMillan-Major
Gérard Dupont
Stella Biderman
Anna Rogers
Loubna Ben allal
Francesco De Toni
Giada Pistilli … (see 34 more)
Olivier Nguyen
Somaieh Nikpoor
Maraim Masoud
Pierre Colombo
Javier de la Rosa
Paulo Villegas
Tristan Thrush
Shayne Longpre
Sebastian Nagel
Leon Weber
Manuel Romero Muñoz
Jian Zhu
Daniel Van Strien
Zaid Alyafeai
Khalid Almubarak
Vu Minh Chien
Itziar Gonzalez-Dios
Aitor Soroa
Kyle Lo
Manan Dey
Pedro Ortiz Suarez
Aaron Gokaslan
Shamik Bose
David Ifeoluwa Adelani
Long Phan
Hieu Tran
Ian Yu
Suhas Pai
Jenny Chim
Violette Lepercq
Suzana Ilic
Margaret Mitchell
Sasha Luccioni
Yacine Jernite
As language models grow ever larger, the need for large-scale high-quality text datasets has never been more pressing, especially in multili… (see more)ngual settings. The BigScience workshop, a 1-year international and multidisciplinary initiative, was formed with the goal of researching and training large language models as a values-driven undertaking, putting issues of ethics, harm, and governance in the foreground. This paper documents the data creation and curation efforts undertaken by BigScience to assemble the Responsible Open-science Open-collaboration Text Sources (ROOTS) corpus, a 1.6TB dataset spanning 59 languages that was used to train the 176-billion-parameter BigScience Large Open-science Open-access Multilingual (BLOOM) language model. We further release a large initial subset of the corpus and analyses thereof, and hope to empower large-scale monolingual and multilingual modeling projects with both the data and the processing tools, as well as stimulate research around this large multilingual corpus.
2022-09-15
NeurIPS.cc/2022/Track/Datasets_and_Benchmarks (accepted)
doi.org
openreview.net
Towards Better Evaluation for Dynamic Link Prediction
Farimah Poursafaei
Andy Huang
Shenyang Huang
Kellin Pelrine
Reihaneh Rabbany
Despite the prevalence of recent success in learning from static graphs, learning from time-evolving graphs remains an open challenge. In th… (see more)is work, we design new, more stringent evaluation procedures for link prediction specific to dynamic graphs, which reflect real-world considerations, to better compare the strengths and weaknesses of methods. First, we create two visualization techniques to understand the reoccurring patterns of edges over time and show that many edges reoccur at later time steps. Based on this observation, we propose a pure memorization-based baseline called EdgeBank. EdgeBank achieves surprisingly strong performance across multiple settings which highlights that the negative edges used in the current evaluation are easy. To sample more challenging negative edges, we introduce two novel negative sampling strategies that improve robustness and better match real-world applications. Lastly, we introduce six new dynamic graph datasets from a diverse set of domains missing from current benchmarks, providing new challenges and opportunities for future research. Our code repository is accessible at https://github.com/fpour/DGB.git.
2022-09-15
NeurIPS.cc/2022/Track/Datasets_and_Benchmarks (accepted)
doi.org
openreview.net
Accurate machine learning prediction of sexual orientation based on brain morphology and intrinsic functional connectivity
Benjamin Clemens
Jeremy Lefort-Besnard
Christoph Ritter
Elke Smith
Mikhail Votinov
Birgit Derntl
Ute Habel
Danilo Bzdok
Sexual orientation in humans represents a multilevel construct that is grounded in both neurobiological and environmental factors. Here, we… (see more) bring to bear a machine learning approach to predict sexual orientation from gray matter volumes (GMVs) or resting-state functional connectivity (RSFC) in a cohort of 45 heterosexual and 41 homosexual participants.  In both brain assessments, we used penalized logistic regression models and nonparametric permutation.  We found an average accuracy of 62% (±6.72) for predicting sexual orientation based on GMV and an average predictive accuracy of 92% (±9.89) using RSFC. Regions in the precentral gyrus, precuneus and the prefrontal cortex were significantly informative for distinguishing heterosexual from homosexual participants in both the GMV and RSFC settings.  These results indicate that, aside from self-reports, RSFC offers neurobiological information valuable for highly accurate prediction of sexual orientation. We demonstrate for the first time that sexual orientation is reflected in specific patterns of RSFC, which enable personalized, brain-based predictions of this highly complex human trait. While these results are preliminary, our neurobiologically based prediction framework illustrates the great value and potential of RSFC for revealing biologically meaningful and generalizable predictive patterns in the human brain.
2022-09-13
Cerebral Cortex (unknown)
doi.org
Designing Biological Sequences via Meta-Reinforcement Learning and Bayesian Optimization
Leo Feng
Padideh Nouri
Aneri Muni
Yoshua Bengio
Pierre-Luc Bacon
The ability to accelerate the design of biological sequences can have a substantial impact on the progress of the medical field. The problem… (see more) can be framed as a global optimization problem where the objective is an expensive black-box function such that we can query large batches restricted with a limitation of a low number of rounds. Bayesian Optimization is a principled method for tackling this problem. However, the astronomically large state space of biological sequences renders brute-force iterating over all possible sequences infeasible. In this paper, we propose MetaRLBO where we train an autoregressive generative model via Meta-Reinforcement Learning to propose promising sequences for selection via Bayesian Optimization. We pose this problem as that of finding an optimal policy over a distribution of MDPs induced by sampling subsets of the data acquired in the previous rounds. Our in-silico experiments show that meta-learning over such ensembles provides robustness against reward misspecification and achieves competitive results compared to existing strong baselines.
2022-09-12
ArXiv (preprint)
doi.org
arxiv.org
Measuring Commonality in Recommendation of Cultural Content: Recommender Systems to Enhance Cultural Citizenship
Andres Ferraro
Gustavo Ferreira
Fernando Diaz
Georgina Born
2022-09-12
Proceedings of the 16th ACM Conference on Recommender Systems (published)
doi.org
arxiv.org
Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity
Kiwan Maeng
Haiyu Lu
Luca Melis
John Nguyen
Michael G. Rabbat
Carole-Jean Wu
2022-09-12
Proceedings of the 16th ACM Conference on Recommender Systems (published)
doi.org
arxiv.org
Learning Multi-Objective Curricula for Robotic Policy Learning
Jikun Kang
Miao Liu
Abhinav Gupta
Christopher Pal
Xue Liu
Jie Fu
2022-09-09
Conference on Robot Learning (poster)
doi.org
proceedings.mlr.press
A neurodynamic model of inter-brain coupling in the gamma band
Quentin Moreau
Lena Adel
Caitriona Douglas
Ranjbaran Ghazaleh
Guillaume Dumas
2022-09-06
Journal of Neurophysiology (published)
doi.org
Unifying Generative Models with GFlowNets
Dinghuai Zhang
Ricky T. Q. Chen
Nikolay Malkin
Yoshua Bengio
There are many frameworks for deep generative modeling, each often presented with their own specific training algorithms and inference metho… (see more)ds. Here, we demonstrate the connections between existing deep generative models and the recently introduced GFlowNet framework, a probabilistic inference machine which treats sampling as a decision-making process. This analysis sheds light on their overlapping traits and provides a unifying viewpoint through the lens of learning with Markovian trajectories. Our framework provides a means for unifying training and inference algorithms, and provides a route to shine a unifying light over many generative models. Beyond this, we provide a practical and experimentally verified recipe for improving generative modeling with insights from the GFlowNet perspective.
2022-09-05
ArXiv (preprint)
doi.org
arxiv.org