Publications

Policy Gradient Methods in the Presence of Symmetries and State Abstractions
Prakash Panangaden
Sahand Rezaei-Shoshtari
Rosie Zhao
David Meger
Doina Precup
Reinforcement learning (RL) on high-dimensional and complex problems relies on abstraction for improved efficiency and generalization. In th… (see more)is paper, we study abstraction in the continuous-control setting, and extend the definition of Markov decision process (MDP) homomorphisms to the setting of continuous state and action spaces. We derive a policy gradient theorem on the abstract MDP for both stochastic and deterministic policies. Our policy gradient results allow for leveraging approximate symmetries of the environment for policy optimization. Based on these theorems, we propose a family of actor-critic algorithms that are able to learn the policy and the MDP homomorphism map simultaneously, using the lax bisimulation metric. Finally, we introduce a series of environments with continuous symmetries to further demonstrate the ability of our algorithm for action abstraction in the presence of such symmetries. We demonstrate the effectiveness of our method on our environments, as well as on challenging visual control tasks from the DeepMind Control Suite. Our method's ability to utilize MDP homomorphisms for representation learning leads to improved performance, and the visualizations of the latent space clearly demonstrate the structure of the learned abstraction.
2023-12-31
Journal of Machine Learning Research (published)
doi.org
arxiv.org
Population Monte Carlo With Normalizing Flow
Soumyasundar Pal
Antonios Valkanas
Mark J. Coates
Adaptive importance sampling (AIS) methods provide a useful alternative to Markov Chain Monte Carlo (MCMC) algorithms for performing inferen… (see more)ce of intractable distributions. Population Monte Carlo (PMC) algorithms constitute a family of AIS approaches which adapt the proposal distributions iteratively to improve the approximation of the target distribution. Recent work in this area primarily focuses on ameliorating the proposal adaptation procedure for high-dimensional applications. However, most of the AIS algorithms use simple proposal distributions for sampling, which might be inadequate in exploring target distributions with intricate geometries. In this work, we construct expressive proposal distributions in the AIS framework using normalizing flow, an appealing approach for modeling complex distributions. We use an iterative parameter update rule to enhance the approximation of the target distribution. Numerical experiments show that in high-dimensional settings, the proposed algorithm offers significantly improved performance compared to the existing techniques.
2023-12-31
IEEE Signal Processing Letters (published)
doi.org
arxiv.org
Quantifying learning-style adaptation in effectiveness of LLM teaching
Ruben Weijers
Gabrielle Fidelis de Castilho
Jean-François Godbout
Reihaneh Rabbany
Kellin Pelrine
This preliminary study aims to investigate whether AI, when prompted based on individual learning styles, can effectively improve comprehens… (see more)ion and learning experiences in educational settings. It involves tailoring LLMs baseline prompts and comparing the results of a control group receiving standard content and an experimental group receiving learning style-tailored content. Preliminary results suggest that GPT-4 can generate responses aligned with various learning styles, indicating the potential for enhanced engagement and comprehension. However, these results also reveal challenges, including the model’s tendency for sycophantic behavior and variability in responses. Our findings suggest that a more sophisticated prompt engineering approach is required for integrating AI into education (AIEd) to improve educational outcomes.
2023-12-31
PERSONALIZE @ AAAI Conference on Artificial Intelligence (published)
doi.org
Reproducible Spinal Cord Quantitative MRI Analysis with the Spinal Cord Toolbox
Jan Valosek
Julien Cohen-Adad
The spinal cord plays a pivotal role in the central nervous system, providing communication between the brain and the body and containing cr… (see more)itical motor and sensory networks. Recent advancements in spinal cord MRI data acquisition and image analysis have shown a potential to improve the diagnostics, prognosis, and management of a variety of pathological conditions. In this review, we first discuss the significance of standardized spinal cord MRI acquisition protocol in multi-center and multi-manufacturer studies. Then, we cover open-access spinal cord MRI datasets, which are important for reproducible science and validation of new methods. Finally, we elaborate on the recent advances in spinal cord MRI data analysis techniques implemented in the open-source software package Spinal Cord Toolbox (SCT).
2023-12-31
Magnetic Resonance in Medical Sciences (published)
doi.org
Resilience and Mental-Health Symptoms in ICU Healthcare Professionals Facing Repeated COVID-19 Waves
Elie Azoulay
Frédéric Pochard
Laurent Argaud
Alain Cariou
Raphael Clere-Jehl
Olivier Guisset
Vincent Labbé
Fabienne Tamion
Fabrice Bruneel
Mercé Jourdain
Danielle Reuter
Kada Klouche
Achille Kouatchet
Virginie Souppart
Alexandre Lautrette
Julien Bohé
Antoine Vieillard Baron
Jean Dellamonica
Laurent Papazian
Jean Reignier … (see 3 more)
François Barbier
Guillaume Dumas
Nancy Kentish-Barnes
2023-12-31
American Journal of Respiratory and Critical Care Medicine (published)
doi.org
RGP: Achieving Memory-Efficient Model Fine-tuning Via Randomized Gradient Projection
Ali Saheb Pasand
Pouya Bashivan
2023-12-31
ENLSP (published)
proceedings.mlr.press
proceedings.mlr.press
Scope Ambiguities in Large Language Models
Gaurav Kamath
Sebastian Schuster
Sowmya Vajjala
Siva Reddy
Sentences containing multiple semantic operators with overlapping scope often create ambiguities in interpretation, known as scope ambiguiti… (see more)es. These ambiguities offer rich insights into the interaction between semantic structure and world knowledge in language processing. Despite this, there has been little research into how modern large language models treat them. In this paper, we investigate how different versions of certain autoregressive language models -- GPT-2, GPT-3/3.5, Llama 2 and GPT-4 -- treat scope ambiguous sentences, and compare this with human judgments. We introduce novel datasets that contain a joint total of almost 1,000 unique scope-ambiguous sentences, containing interactions between a range of semantic operators, and annotated for human judgments. Using these datasets, we find evidence that several models (i) are sensitive to the meaning ambiguity in these sentences, in a way that patterns well with human judgments, and (ii) can successfully identify human-preferred readings at a high level of accuracy (over 90% in some cases).
2023-12-31
Trans. Assoc. Comput. Linguistics (published)
doi.org
arxiv.org
Self-supervised multimodal learning for group inferences from MRI data: Discovering disorder-relevant brain regions and multimodal links
Alex Fedorov
Eloy Geenjaar
Lei Wu
Tristan Sylvain
Thomas P. DeRamus
Margaux Luck
Maria Misiura
Girish Mittapalle
R. Devon Hjelm
Sergey M. Plis
Vince D. Calhoun
In recent years, deep learning approaches have gained significant attention in predicting brain disorders using neuroimaging data. However, … (see more)conventional methods often rely on single-modality data and supervised models, which provide only a limited perspective of the intricacies of the highly complex brain. Moreover, the scarcity of accurate diagnostic labels in clinical settings hinders the applicability of the supervised models. To address these limitations, we propose a novel self-supervised framework for extracting multiple representations from multimodal neuroimaging data to enhance group inferences and enable analysis without resorting to labeled data during pre-training. Our approach leverages Deep InfoMax (DIM), a self-supervised methodology renowned for its efficacy in learning representations by estimating mutual information without the need for explicit labels. While DIM has shown promise in predicting brain disorders from single-modality MRI data, its potential for multimodal data remains untapped. This work extends DIM to multimodal neuroimaging data, allowing us to identify disorder-relevant brain regions and explore multimodal links. We present compelling evidence of the efficacy of our multimodal DIM analysis in uncovering disorder-relevant brain regions, including the hippocampus, caudate, insula, - and multimodal links with the thalamus, precuneus, and subthalamus hypothalamus. Our self-supervised representations demonstrate promising capabilities in predicting the presence of brain disorders across a spectrum of Alzheimer's phenotypes. Comparative evaluations against state-of-the-art unsupervised methods based on autoencoders, canonical correlation analysis, and supervised models highlight the superiority of our proposed method in achieving improved classification performance, capturing joint information, and interpretability capabilities. The computational efficiency of the decoder-free strategy enhances its practical utility, as it saves compute resources without compromising performance. This work offers a significant step forward in addressing the challenge of understanding multimodal links in complex brain disorders, with potential applications in neuroimaging research and clinical diagnosis.
2023-12-31
NeuroImage (published)
doi.org
SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects
David Ifeoluwa Adelani
Hannah Liu
Xiaoyu Shen
Nikita Vassilyev
Jesujoba Oluwadara Alabi
Yanke Mao
Haonan Gao
Annie En-Shiun Lee
2023-12-31
EACL (1) (published)
doi.org
arxiv.org
Signatures of Co-evolution and Co-regulation in the CYP3A and CYP4F Genes in Humans
Alex Richard-St-Hilaire
Isabel Gamache
Justin Pelletier
Jean-Christophe Grenier
Raphaël Poujol
Julie G. Hussin
Cytochromes P450 (CYP450) are hemoproteins generally involved in the detoxification of the body of xenobiotic molecules. They participate in… (see more) the metabolism of many drugs and genetic polymorphisms in humans have been found to impact drug responses and metabolic functions. In this study, we investigate the genetic diversity of CYP450 genes. We found that two clusters, CYP3A and CYP4F, are notably differentiated across human populations with evidence for selective pressures acting on both clusters: we found signals of recent positive selection in CYP3A and CYP4F genes and signals of balancing selection in CYP4F genes. Furthermore, an extensive amount of unusual linkage disequilibrium is detected in this latter cluster, indicating co-evolution signatures among CYP4F genes. Several of the selective signals uncovered co-localize with expression quantitative trait loci (eQTL), which could suggest epistasis acting on co-regulation in these gene families. In particular, we detected a potential co-regulation event between CYP3A5 and CYP3A43, a gene whose function remains poorly characterized. We further identified a causal relationship between CYP3A5 expression and reticulocyte count through Mendelian randomization analyses, potentially involving a regulatory region displaying a selective signal specific to African populations. Our findings linking natural selection and gene expression in CYP3A and CYP4F subfamilies are of importance in understanding population differences in metabolism of nutrients and drugs.
2023-12-31
Genome Biology and Evolution (published)
doi.org
Speech Model Pre-training for End-to-End Spoken Language Understanding
Loren Lugosch
Mirco Ravanelli
Patrick Ignoto
Vikrant Singh Tomar
Yoshua Bengio
Whereas conventional spoken language understanding (SLU) systems map speech to text, and then text to intent, end-to-end SLU systems map spe… (see more)ech directly to intent through a single trainable model. Achieving high accuracy with these end-to-end models without a large amount of training data is difficult. We propose a method to reduce the data requirements of end-to-end SLU in which the model is first pre-trained to predict words and phonemes, thus learning good features for SLU. We introduce a new SLU dataset, Fluent Speech Commands, and show that our method improves performance both when the full dataset is used for training and when only a small subset is used. We also describe preliminary experiments to gauge the model's ability to generalize to new phrases not heard during training.
2023-12-31
TIB (published)
doi.org
arxiv.org
Stochastic Simulated Quantum Annealing for Fast Solution of Combinatorial Optimization Problems
Naoya Onizawa
Ryoma Sasaki
Duckgyu Shin
Warren J. Gross
Takahiro Hanyu
In this paper, we introduce stochastic simulated quantum annealing (SSQA) for large-scale combinatorial optimization problems. SSQA is desig… (see more)ned based on stochastic computing and quantum Monte Carlo, which can simulate quantum annealing (QA) by using multiple replicas of spins (probabilistic bits) in classical computing. The use of stochastic computing leads to an efficient parallel spin-state update algorithm, enabling quick search for a solution around the global minimum energy. Therefore, SSQA realizes quantum-like annealing for large-scale problems and can handle fully connected models in combinatorial optimization, unlike QA. The proposed method is evaluated in MATLAB on graph isomorphism problems, which are typical combinatorial optimization problems. The proposed method achieves a convergence speed an order of magnitude faster than a conventional stochastic simulaated annealing method. Additionally, it can handle a 100-times larger problem size compared to QA and a 25-times larger problem size compared to a traditional SA method, respectively, for similar convergence probabilities.
2023-12-31
IEEE Access (published)
doi.org
arxiv.org