Publications

Scattered Mixture-of-Experts Implementation
Shawn Tan
Yikang Shen
Rameswar Panda
Aaron Courville
ScatterMoE is an implementation of Sparse Mixture-of-Experts (SMoE) on GPUs. ScatterMoE builds upon techniques in existing implementations, … (see more)and overcoming some of the current limitations to improve batched inference, training speed, and memory footprint. This implementation achieves this by avoiding padding and making excessive copies of the input. We also fuse expert linear transforms and reordering operations with ParallelLinear, a module that can be used to extend the concept of SMoEs. We benchmark our implementation against Megablocks, and show that it enables a higher throughput and lower memory footprint. We also show how ParallelLinear enables extension of the Mixture-of-Experts concept by demonstrating with an implementation of Mixture-of-Attention.
2024-07-09
colmweb.org/COLM/2024/Conference (accepted)
doi.org
openreview.net
Seeking Interpretability and Explainability in Binary Activated Neural Networks
Benjamin Leblanc
Pascal Germain
2024-07-09
Communications in Computer and Information Science (published)
doi.org
arxiv.org
Should We Attend More or Less? Modulating Attention for Fairness
Abdelrahman Zayed
Goncalo Mordido
Samira Shabanian
A. Chandar
2024-07-09
colmweb.org/COLM/2024/Conference (accepted)
doi.org
openreview.net
A Survey on Deep Learning for Theorem Proving
Zhaoyu Li
Jialiang Sun
Logan Murphy
Qidong Su
Zenan Li
Xian Zhang
Kaiyu Yang
Xujie Si
2024-07-09
colmweb.org/COLM/2024/Conference (accepted)
doi.org
openreview.net
The black box of the relationship between breast cancer patients and accompanying patients: the accompanied patients' point of view
Marie-Pascale Pomey
Monica Iliescu Nelea
Cécile Vialaron
Louise Normandin
Marie-Andrée Côté
Mado Desforges
Pénélope Pomey-Carpentier
Nesrine Adjtoutah
Israël Fortin
Isabelle Ganache
Catherine Régis
Zeev Rosberger
Danielle Charpentier
Lynda Bélanger
Michel Dorval
Djahanchah P. Ghadiri
Mélanie Lavoie-Tremblay
Antoine Boivin
Jean-François Pelletier
Nicolas Fernandez … (see 2 more)
Alain M. Danino
Michèle de Guise
The PAROLE-Onco program was introduced in the province of Quebec, Canada in 2019. It integrates accompanying patients (APs), i.e., people wh… (see more)o have been affected by cancer, into the clinical team as full members. These APs use their experiential knowledge with people undergoing treatment and with clinical teams. The aim of this paper is to evaluate, within the framework of two university medical centers, the perceptions of breast cancer patients who receive support from APs, particularly in terms of their active involvement in their care trajectory. A qualitative study based on semi-structured interviews with accompanied patients was performed. Fourteen individual interviews were conducted between July and September 2021 with women presenting different profiles in terms of age, education, professional status, type of treatment, family situation, and clinical background. The data were analyzed using thematic analysis, focusing on patients’ perceptions of APs’ contributions and suggested improvements for accessing AP support. Three themes emerged from the semi-structured interviews: communication modalities used to connect patients with their APs, the characteristics of the support provided by APs, and the perceived effects of this support on the patients. Patients expressed a preference for telephone communication, highlighting its convenience and accessibility. The support provided by APs included emotional and informational support, neutrality, and adaptability. This relationship improved patient communication, reduced anxiety, helped regain control, and enhanced overall quality of life. The results emphasized the added value of APs in complementing the support offered by healthcare professionals. Patients noted the critical role of APs in helping them navigate the healthcare system, better understand their treatment processes, and manage their emotions. The ability of APs to provide practical advice and emotional reassurance was particularly valued. Overall, the findings underscored the significant impact of AP support on patients’ experiences and highlighted areas for enhancing this service. This study highlights, during the care trajectory of people affected by breast cancer, APs’ contribution to patients’ emotional well-being because they improve, in particular, the management of emotions and communication with health professionals. The online version contains supplementary material available at 10.1186/s12885-024-12585-z.
2024-07-09
BMC Cancer (published)
doi.org
Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wild
Niloofar Mireshghallah
Maria Antoniak
Yash More
Yejin Choi
Golnoosh Farnadi
Measuring personal disclosures made in human-chatbot interactions can provide a better understanding of users' AI literacy and facilitate pr… (see more)ivacy research for large language models (LLMs). We run an extensive, fine-grained analysis on the personal disclosures made by real users to commercial GPT models, investigating the leakage of personally identifiable and sensitive information. To understand the contexts in which users disclose to chatbots, we develop a taxonomy of tasks and sensitive topics, based on qualitative and quantitative analysis of naturally occurring conversations. We discuss these potential privacy harms and observe that: (1) personally identifiable information (PII) appears in unexpected contexts such as in translation or code editing (48% and 16% of the time, respectively) and (2) PII detection alone is insufficient to capture the sensitive topics that are common in human-chatbot interactions, such as detailed sexual preferences or specific drug use habits. We believe that these high disclosure rates are of significant importance for researchers and data curators, and we call for the design of appropriate nudging mechanisms to help users moderate their interactions.
2024-07-09
colmweb.org/COLM/2024/Conference (accepted)
doi.org
openreview.net
V-STaR: Training Verifiers for Self-Taught Reasoners
Arian Hosseini
Xingdi Yuan
Nikolay Malkin
Aaron Courville
Alessandro Sordoni
Rishabh Agarwal
Common self-improvement approaches for large language models (LLMs), such as STaR, iteratively fine-tune LLMs on self-generated solutions to… (see more) improve their problem-solving ability. However, these approaches discard the large amounts of incorrect solutions generated during this process, potentially neglecting valuable information in such solutions. To address this shortcoming, we propose V-STaR that utilizes both the correct and incorrect solutions generated during the self-improvement process to train a verifier using DPO that judges correctness of model-generated solutions. This verifier is used at inference time to select one solution among many candidate solutions. Running V-STaR for multiple iterations results in progressively better reasoners and verifiers, delivering a 4% to 17% test accuracy improvement over existing self-improvement and verification approaches on common code generation and math reasoning benchmarks with LLaMA2 models.
2024-07-09
colmweb.org/COLM/2024/Conference (accepted)
doi.org
openreview.net
Web Retrieval Agents for Evidence-Based Misinformation Detection
Jacob-Junqi Tian
Hao Yu
Yury Orlovskiy
Tyler Vergho
Mauricio Rivera
Mayank Goel
Zachary Yang
Jean-François Godbout
Reihaneh Rabbany
Kellin Pelrine
2024-07-09
colmweb.org/COLM/2024/Conference (accepted)
doi.org
openreview.net
What makes a good metric? Evaluating automatic metrics for text-to-image consistency
Candace Ross
Melissa Hall
Adriana Romero
Adina Williams
2024-07-09
colmweb.org/COLM/2024/Conference (accepted)
doi.org
openreview.net
Automated River Substrate Mapping From Sonar Imagery With Machine Learning
C. S. Bodine
D. Buscombe
Toby Dylan Hocking
2024-07-08
Journal of Geophysical Research: Machine Learning and Computation (published)
doi.org
Canada’s approach to SARS-CoV-2 sero-surveillance: Lessons learned for routine surveillance and future pandemics
Sheila F. O’Brien
Michael Asamoah-Boaheng
Brian Grunau
Mel Krajden
David L Buckeridge
David M. Goldfarb
Maureen Anderson
Marc Germain
Patrick Brown
Derek R. Stein
Kami Kandola
Graham Tipples
Philip Awadalla
Amanda Lang
Lesley Behl
Tiffany Fitzpatrick
Steven J. Drews
2024-07-08
Canadian journal of public health (published)
doi.org
Differentially Private Linear Regression With Linked Data
Shurong Lin
Elliot Paquette
Eric D. Kolaczyk
There has been increasing demand for establishing privacy-preserving methodologies for modern statistics and machine learning. Differential … (see more)privacy, a mathematical notion from computer science, is a rising tool offering robust privacy guarantees. Recent work focuses primarily on developing differentially private versions of individual statistical and machine learning tasks, with nontrivial upstream pre-processing typically not incorporated. An important example is when record linkage is done prior to downstream modeling. Record linkage refers to the statistical task of linking two or more data sets of the same group of entities without a unique identifier. This probabilistic procedure brings additional uncertainty to the subsequent task. In this paper, we present two differentially private algorithms for linear regression with linked data. In particular, we propose a noisy gradient method and a sufficient statistics perturbation approach for the estimation of regression coefficients. We investigate the privacy-accuracy tradeoff by providing finite-sample error bounds for the estimators, which allows us to understand the relative contributions of linkage error, estimation error, and the cost of privacy. The variances of the estimators are also discussed. We demonstrate the performance of the proposed algorithms through simulations and an application to synthetic data.
2024-07-08
Harvard Data Science Review (published)
doi.org
arxiv.org