Publications

Sample-Efficient Reinforcement Learning by Breaking the Replay Ratio Barrier
Pierluca D'Oro
Max Schwarzer
Evgenii Nikishin
Pierre-Luc Bacon
Bellemare Marc-Emmanuel
Aaron Courville
2023-04-30
International Conference on Learning Representations (Accept: notable-top-5%)
openreview.net
openreview.net
Simplicial Embeddings in Self-Supervised Learning and Downstream Classification
Samuel Lavoie
Christos Tsirigotis
Max Schwarzer
Ankit Vani
Michael Noukhovitch
Kenji Kawaguchi
Aaron Courville
Simplicial Embeddings (SEM) are representations learned through self-supervised learning (SSL), wherein a representation is projected into …
2023-04-30
International Conference on Learning Representations (Accept: notable-top-25%)
doi.org
openreview.net
Stateful Active Facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning
Dianbo Liu
Vedant Shah
Oussama Boussif
Cristian Meo
Anirudh Goyal
Tianmin Shu
Michael Mozer
Nicolas Heess
Yoshua Bengio
In cooperative multi-agent reinforcement learning, a team of agents works together to achieve a common goal. Different environments or tasks… (see more) may require varying degrees of coordination among agents in order to achieve the goal in an optimal way. The nature of coordination will depend on the properties of the environment -- its spatial layout, distribution of obstacles, dynamics, etc. We term this variation of properties within an environment as heterogeneity. Existing literature has not sufficiently addressed the fact that different environments may have different levels of heterogeneity. We formalize the notions of coordination level and heterogeneity level of an environment and present HECOGrid, a suite of multi-agent RL environments that facilitates empirical evaluation of different MARL approaches across different levels of coordination and environmental heterogeneity by providing a quantitative control over coordination and heterogeneity levels of the environment. Further, we propose a Centralized Training Decentralized Execution learning approach called Stateful Active Facilitator (SAF) that enables agents to work efficiently in high-coordination and high-heterogeneity environments through a differentiable and shared knowledge source used during training and dynamic selection from a shared pool of policies. We evaluate SAF and compare its performance against baselines IPPO and MAPPO on HECOGrid. Our results show that SAF consistently outperforms the baselines across different tasks and different heterogeneity and coordination levels. We release the code for HECOGrid as well as all our experiments.
2023-04-30
ICLR (Accept: poster)
doi.org
openreview.net
Systematic Rectification of Language Models via Dead-End Analysis
Meng Cao
Mehdi Fatemi
Jackie Chi Kit Cheung
Samira Shabanian
With adversarial or otherwise normal prompts, existing large language models (LLM) can be pushed to generate toxic discourses. One way to re… (see more)duce the risk of LLMs generating undesired discourses is to alter the training of the LLM. This can be very restrictive due to demanding computation requirements. Other methods rely on rule-based or prompt-based token elimination, which are limited as they dismiss future tokens and the overall meaning of the complete discourse. Here, we center detoxification on the probability that the finished discourse is ultimately considered toxic. That is, at each point, we advise against token selections proportional to how likely a finished text from this point will be toxic. To this end, we formally extend the dead-end theory from the recent reinforcement learning (RL) literature to also cover uncertain outcomes. Our approach, called rectification, utilizes a separate but significantly smaller model for detoxification, which can be applied to diverse LLMs as long as they share the same vocabulary. Importantly, our method does not require access to the internal representations of the LLM, but only the token probability distribution at each decoding step. We believe this is important since many LLMs today are hosted in servers and only accessible through APIs. When applied to various LLMs, including GPT-3, our approach generates notably better results compared to the base LLMs and other techniques in terms of the overall language and detoxification performance.
2023-04-30
ICLR (Accept: poster)
doi.org
openreview.net
TCR/Chimeric Antigen Receptor (CAR) Cross-Antagonism to Fine-Tune CAR-T cell Immunotherapy
Grégoire Altan-Bonnet
Taisuke Kondo
François X. P. Bourassa
Sooraj R. Achar
Justyn DuSold
Pablo Cespedes
Madison Wahlsten
Guillaume Gaud
Paul E Love
Mike Dustin
Paul François
Naomi Taylor
Chimeric antigen receptor (CAR) T cells, are created by extracting T cells from a cancer patient, engineering them to express a CAR targetin… (see more)g a tumor specific molecule, then reintroducing them back into the patient. A patient’s T cells contain their own endogenous T cell receptors (TCRs) however, which could potentially interact with the exogenous CAR inserted into the cell. In this study, we examine how TCR and CAR signals interact upon CAR-T activation. We show that weak TCR stimulation can reduce (antagonize) or increase overall CAR-T response, both in vitro and in vivo, across multiple tumor models, in both mouse and human T cells. We further show that the behavior of these TCR/CAR interactions can be manipulated by changing various characteristics of the TCR, CAR, and associated ligands. While this behavior is complex, we show that it can be described by a single mathematical model based on the adaptive kinetic proofreading scheme of ligand discrimination. We conclude by presenting potential applications for cancer immunotherapy. Intramural Research Program of the National Cancer Institute
2023-04-30
Journal of Immunology (published)
doi.org
The StatCan Dialogue Dataset: Retrieving Data Tables through Conversations with Genuine Intents
Xing Han Lu
Siva Reddy
Harm de Vries
We introduce the StatCan Dialogue Dataset consisting of 19,379 conversation turns between agents working at Statistics Canada and online use… (see more)rs looking for published data tables. The conversations stem from genuine intents, are held in English or French, and lead to agents retrieving one of over 5000 complex data tables. Based on this dataset, we propose two tasks: (1) automatic retrieval of relevant tables based on a on-going conversation, and (2) automatic generation of appropriate agent responses at each turn. We investigate the difficulty of each task by establishing strong baselines. Our experiments on a temporal data split reveal that all models struggle to generalize to future conversations, as we observe a significant drop in performance across both tasks when we move from the validation to the test set. In addition, we find that response generation models struggle to decide when to return a table. Considering that the tasks pose significant challenges to existing models, we encourage the community to develop models for our task, which can be directly used to help knowledge workers find relevant tables for live chat users.
2023-04-30
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (published)
doi.org
arxiv.org
The study of plasticity has always been about gradients
Blake Aaron Richards
Konrad Paul Kording
2023-04-30
The Journal of Physiology (published)
doi.org
A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games
Samuel Sokota
Ryan D'Orazio
J. Zico Kolter
Nicolas Loizou
Marc Lanctot
Ioannis Mitliagkas
Noam Brown
Christian Kroer
This work studies an algorithm, which we call magnetic mirror descent, that is inspired by mirror descent and the non-Euclidean proximal gra… (see more)dient algorithm. Our contribution is demonstrating the virtues of magnetic mirror descent as both an equilibrium solver and as an approach to reinforcement learning in two-player zero-sum games. These virtues include: 1) Being the first quantal response equilibria solver to achieve linear convergence for extensive-form games with first order feedback; 2) Being the first standard reinforcement learning algorithm to achieve empirically competitive results with CFR in tabular settings; 3) Achieving favorable performance in 3x3 Dark Hex and Phantom Tic-Tac-Toe as a self-play deep reinforcement learning algorithm.
2023-04-30
International Conference on Learning Representations (Accept: poster)
doi.org
openreview.net
What did the T cell see? A deep-learning model of CD8+ T cell activation reveals sharp antigen discrimination at the single cell level
Madison Wahlsten
Amin Akhshi
Sooraj R. Achar
Anagha Yogam Krishnan
Paul François
Grégoire Altan-Bonnet
Immunotherapies such as checkpoint blockade antibodies to block T cell exhaustion have been successful in several cancers such as non-small … (see more)cell lung cancer and melanoma, but limited in others (e.g., pancreatic or prostate carcinomas) owing to differences in tumor antigenicity. Therefore, quantifying tumor antigenicity is critical for successful immunotherapies. Our lab has shown that antigenicity can be encoded in a single parameter derived from bulk cytokine dynamics in ex vivo co-cultures between antigen presenting cells (APCs) and T cells. Here we built a model that can capture the antigenicity seen by individual cells. Using a custom robotic platform, we generated high-throughput kinetics of T cell activation in co-culture with APCs by analyzing cells at various timepoints across a large set of activation conditions. We performed spectral flow cytometry to measure the expression of up to 30 surface markers and intracellular signals per cell. To analyze our content-rich datasets, we designed a machine learning-based model that can classify the antigen seen by an individual cell using expression values from flow cytometry. The model performs well not only at classifying T cells (ROC-AUC > 0.91), but also APCs (ROC-AUC > 0.88), suggesting that each individual leukocyte may register the quality of antigen being presented. Blocking cytokine signaling disrupted this antigen classification. Our study demonstrates that every individual lymphocyte can bridge local and global response to achieve high discriminatory power of antigens.
2023-04-30
Journal of Immunology (published)
doi.org
Estimating individual minimum calibration for deep-learning with predictive performance recovery: An example case of gait surface classification from wearable sensor gait data
Guillaume Lam
Irina Rish
Philippe C. Dixon
2023-04-29
Journal of Biomechanics (unknown)
doi.org
Social Media as a Vector for Escort Ads: A Study on OnlyFans advertisements on Twitter.
Maricarmen Arenas
Pratheeksha Nair
Reihaneh Rabbany
Golnoosh Farnadi
Online sex trafficking is on the rise and a majority of trafficking victims report being advertised online. The use of OnlyFans as a platfor… (see more)m for adult content is also increasing, with Twitter as its main advertising tool. Furthermore, we know that traffickers usually work within a network and control multiple victims. Consequently, we suspect that there may be networks of traffickers promoting multiple OnlyFans accounts belonging to their victims. To this end, we present the first study of OnlyFans advertisements on Twitter in the context of finding organized activities. Preliminary analysis of this space shows that most tweets related to OnlyFans contain generic text, making text-based methods less reliable. Instead, focusing on what ties the authors of these tweets together, we propose a novel method for uncovering coordinated networks of users based on their behaviour. Our method, called Multi-Level Clustering (MLC), combines two levels of clustering that considers both the network structure as well as embedded node attribute information. It focuses jointly on user connections (through mentions) and content (through shared URLs). We apply MLC to real-world data of 2 million tweets pertaining to OnlyFans and analyse the detected groups. We also evaluate our method on synthetically generated data (with injected ground truth) and show its superior performance compared to competitive baselines. Finally, we discuss examples of organized clusters as case studies and provide interesting conclusions to our study.
2023-04-29
Web Science Conference (published)
doi.org
Can ChatGPT Pass An Introductory Level Functional Language Programming Course?
Chuqin Geng
Yihan Zhang
Brigitte Pientka
Xujie Si
The recent introduction of ChatGPT has drawn significant attention from both industry and academia due to its impressive capabilities in sol… (see more)ving a diverse range of tasks, including language translation, text summarization, and computer programming. Its capability for writing, modifying, and even correcting code together with its ease of use and access is already dramatically impacting computer science education. This paper aims to explore how well ChatGPT can perform in an introductory-level functional language programming course. In our systematic evaluation, we treated ChatGPT as one of our students and demonstrated that it can achieve a grade B- and its rank in the class is 155 out of 314 students overall. Our comprehensive evaluation provides valuable insights into ChatGPT's impact from both student and instructor perspectives. Additionally, we identify several potential benefits that ChatGPT can offer to both groups. Overall, we believe that this study significantly clarifies and advances our understanding of ChatGPT's capabilities and potential impact on computer science education.
2023-04-28
ArXiv (preprint)
doi.org
arxiv.org