Publications

When does word order matter and when doesn't it?
Xuanda Chen
Timothy John O'donnell
Siva Reddy
Language models (LMs) may appear insensitive to word order changes in natural language understanding (NLU) tasks. In this paper, we propose … (see more)that linguistic redundancy can explain this phenomenon, whereby word order and other linguistic cues such as case markers provide overlapping and thus redundant information. Our hypothesis is that models exhibit insensitivity to word order when the order provides redundant information, and the degree of insensitivity varies across tasks. We quantify how informative word order is using mutual information (MI) between unscrambled and scrambled sentences. Our results show the effect that the less informative word order is, the more consistent the model's predictions are between unscrambled and scrambled sentences. We also find that the effect varies across tasks: for some tasks, like SST-2, LMs' prediction is almost always consistent with the original one even if the Pointwise-MI (PMI) changes, while for others, like RTE, the consistency is near random when the PMI gets lower, i.e., word order is really important.
2024-02-28
ArXiv (preprint)
doi.org
arxiv.org
Acoustic tactile sensing for mobile robot wheels
Wilfred Mason
David Brenken
Falcon Z. Dai
Ricardo Gonzalo Cruz Castillo
Olivier St-Martin Cormier
Audrey Sedal
2024-02-27
ArXiv (preprint)
doi.org
arxiv.org
ICE-SEARCH: A Language Model-Driven Feature Selection Approach
Tianze Yang
Tianyi Yang
Shaoshan Liu
Fuyuan Lyu
Xue Liu
This study unveils the In-Context Evolutionary Search (ICE-SEARCH) method, the first work that melds language models (LMs) with evolutionary… (see more) algorithms for feature selection (FS) tasks and demonstrates its effectiveness in Medical Predictive Analytics (MPA) applications. ICE-SEARCH harnesses the crossover and mutation capabilities inherent in LMs within an evolutionary framework, significantly improving FS through the model's comprehensive world knowledge and its adaptability to a variety of roles. Our evaluation of this methodology spans three crucial MPA tasks: stroke, cardiovascular disease, and diabetes, where ICE-SEARCH outperforms traditional FS methods in pinpointing essential features for medical applications. ICE-SEARCH achieves State-of-the-Art (SOTA) performance in stroke prediction and diabetes prediction; the Decision-Randomized ICE-SEARCH ranks as SOTA in cardiovascular disease prediction. Our results not only demonstrate the efficacy of ICE-SEARCH in medical FS but also underscore the versatility, efficiency, and scalability of integrating LMs in FS tasks. The study emphasizes the critical role of incorporating domain-specific insights, illustrating ICE-SEARCH's robustness, generalizability, and swift convergence. This opens avenues for further research into comprehensive and intricate FS landscapes, marking a significant stride in the application of artificial intelligence in medical predictive analytics.
2024-02-27
ArXiv (preprint)
doi.org
arxiv.org
A density estimation perspective on learning from pairwise human preferences
Vincent Dumoulin
Daniel D. Johnson
Pablo Samuel Castro
Hugo Larochelle
Yann Dauphin
Learning from human feedback (LHF) -- and in particular learning from pairwise preferences -- has recently become a crucial ingredient in tr… (see more)aining large language models (LLMs), and has been the subject of much research. Most recent works frame it as a reinforcement learning problem, where a reward function is learned from pairwise preference data and the LLM is treated as a policy which is adapted to maximize the rewards, often under additional regularization constraints. We propose an alternative interpretation which centers on the generative process for pairwise preferences and treats LHF as a density estimation problem. We provide theoretical and empirical results showing that for a family of generative processes defined via preference behavior distribution equations, training a reward function on pairwise preferences effectively models an annotator's implicit preference distribution. Finally, we discuss and present findings on"annotator misspecification"-- failure cases where wrong modeling assumptions are made about annotator behavior, resulting in poorly-adapted models -- suggesting that approaches that learn from pairwise human preferences could have trouble learning from a population of annotators with diverse viewpoints.
2024-02-26
TMLR (accepted)
doi.org
openreview.net
RAMEN Unveils Clinical Variable Networks for COVID-19 Severity and Long COVID Using Absorbing Random Walks and Genetic Algorithms
Yiwei Xiong
Jingtao Wang
Xiaoxiao Shang
Tingting Chen
Douglas D. Fraser
Gregory Fonseca
Simon Rousseau
Jun Ding
The COVID-19 pandemic has significantly altered global socioeconomic structures and individual lives. Understanding the disease mechanisms a… (see more)nd facilitating diagnosis requires comprehending the complex interplay among clinical factors like demographics, symptoms, comorbidities, treatments, lab results, complications, and other metrics, and their relation to outcomes such as disease severity and long term outcomes ( e . g ., post-COVID-19 condition/long COVID). Conventional correlational methods struggle with indirect and directional connections among these factors, while standard graphical methods like Bayesian networks are computationally demanding for extensive clinical variables. In response, we introduced RAMEN, a methodology that integrates Genetic Algorithms with random walks for efficient Bayesian network inference, designed to map the intricate relationships among clinical variables. Applying RAMEN to the Biobanque québécoise de la COVID-19 (BQC19) dataset, we identified critical markers for long COVID and varying disease severity. The Bayesian Network, corroborated by existing literature and supported through multi-omics analyses, highlights significant clinical variables linked to COVID-19 outcomes. RAMEN’s ability to accurately map these connections contributes substantially to developing early and effective diagnostics for severe COVID-19 and long COVID.
2024-02-26
bioRxiv (preprint)
doi.org
Effective Latent Differential Equation Models via Attention and Multiple Shooting
Germán Abrevaya
Mahta Ramezanian-Panahi
Jean-christophe Gagnon-audet
Irina Rish
Pablo Polosecki
Silvina Ponce Dawson
Guillermo Cecchi
Guillaume Dumas
2024-02-25
TMLR (accepted)
openreview.net
openreview.net
Correction to: Multi-agent reinforcement learning for fast-timescale demand response of residential loads
Vincent Mai
Philippe Maisonneuve
Tianyu Zhang
Hadi Nekoei
Liam Paull
Antoine Lesage-Landry
2024-02-22
Machine-Mediated Learning (published)
doi.org
Intra-Host Evolution Analyses in an Immunosuppressed Patient Supports SARS-CoV-2 Viral Reservoir Hypothesis
Dominique Fournelle
Fatima Mostefai
Elsa Brunet-Ratnasingham
Raphaël Poujol
Jean-Christophe Grenier
José Héctor Gálvez
Amélie Pagliuzza
Inès Levade
Sandrine Moreira
Mehdi Benlarbi
Guillaume Beaudoin-Bussières
Gabrielle Gendron-Lepage
Catherine Bourassa
Alexandra Tauzin
Simon Grandjean Lapierre
Nicolas Chomont
Andrés Finzi
Daniel E. Kaufmann
Morgan Craig
Julie G. Hussin
Throughout the SARS-CoV-2 pandemic, several variants of concern (VOCs) have been identified, many of which share recurrent mutations in the … (see more)spike glycoprotein’s receptor-binding domain (RBD). This region coincides with known epitopes and can therefore have an impact on immune escape. Protracted infections in immunosuppressed patients have been hypothesized to lead to an enrichment of such mutations and therefore drive evolution towards VOCs. Here, we present the case of an immunosuppressed patient that developed distinct populations with immune escape mutations throughout the course of their infection. Notably, by investigating the co-occurrence of substitutions on individual sequencing reads in the RBD, we found quasispecies harboring mutations that confer resistance to known monoclonal antibodies (mAbs) such as S:E484K and S:E484A. These mutations were acquired without the patient being treated with mAbs nor convalescent sera and without them developing a detectable immune response to the virus. We also provide additional evidence for a viral reservoir based on intra-host phylogenetics, which led to a viral substrain that evolved elsewhere in the patient’s body, colonizing their upper respiratory tract (URT). The presence of SARS-CoV-2 viral reservoirs can shed light on protracted infections interspersed with periods where the virus is undetectable, and potential explanations for long-COVID cases.
2024-02-22
Viruses (published)
doi.org
Molar Pregnancy in a Quadruplet Conception Following IVF: A Case Report
Madhuri A Mehendale
Meenal Shailesh Sarmalkar
Prerna Kailashchand Gupta
Aishwarya Agrawal
Agraj S Doshi
2024-02-22
Journal of South Asian Federation of Obstetrics and Gynaecology (published)
doi.org
The Sample Average Approximation Method for Solving Two-Stage Stochastic Programs with Endogenous Uncertainty
Maria Bazotte
Margarida Carvalho
Thibaut Vidal
Real-world decision-making problems involve Type 1 decision-dependent uncertainty, where the probability distribution of the stochastic proc… (see more)ess depends on the model decisions. However, few studies focus on two-stage stochastic programs with this type of endogenous uncertainty, and those that do lack general methodologies. We thus propose herein a general method for solving a class of these programs based on the transformation of random variables, a technique widely employed in probability and statistics. The proposed method is tailored to large-scale problems with discrete or continuous endogenous random variables. The random variable transformation allows the use of the sample average approximation (SAA) method, which provides optimality convergence guarantees under certain conditions. We show that, for some classical distributions, the proposed method reduces to solving mixed-integer linear or convex programs. Finally, we validate this method by applying it to a network design and facility-protection problem, considering distinct decision-dependent distributions for the random variables. Whereas most distributions result in a nonlinear nonconvex deterministic equivalent program, the proposed method solves mixed-integer linear programs in all cases. In addition, it produces attractive performance estimators for the SAA method in a reasonable computational time and outperforms the case in which the endogenous distribution defines a mixed-integer deterministic equivalent.
2024-02-22
ArXiv (preprint)
arxiv.org
arxiv.org
Matrix Factorization Recommendation Algorithm Based on Attention Interaction
Chengzhi Mao
Zhifeng Wu
Yingjie Liu
Zhiwei Shi
2024-02-21
Symmetry (published)
doi.org
Posterior inference of Hi-C contact frequency through sampling
Yanlin Zhang
Christopher J. F. Cameron
Mathieu Blanchette
Hi-C is one of the most widely used approaches to study three-dimensional genome conformations. Contacts captured by a Hi-C experiment are r… (see more)epresented in a contact frequency matrix. Due to the limited sequencing depth and other factors, Hi-C contact frequency matrices are only approximations of the true interaction frequencies and are further reported without any quantification of uncertainty. Hence, downstream analyses based on Hi-C contact maps (e.g., TAD and loop annotation) are themselves point estimations. Here, we present the Hi-C interaction frequency sampler (HiCSampler) that reliably infers the posterior distribution of the interaction frequency for a given Hi-C contact map by exploiting dependencies between neighboring loci. Posterior predictive checks demonstrate that HiCSampler can infer highly predictive chromosomal interaction frequency. Summary statistics calculated by HiCSampler provide a measurement of the uncertainty for Hi-C experiments, and samples inferred by HiCSampler are ready for use by most downstream analysis tools off the shelf and permit uncertainty measurements in these analyses without modifications.
2024-02-21
Frontiers in Bioinformatics (published)
doi.org