Chris Pal

Biography

Christopher Pal is a Canada CIFAR AI Chair, full professor at Polytechnique Montréal and adjunct professor in the Department of Computer Science and Operations Research (DIRO) at Université de Montréal. He is also a Distinguished Scientist at ServiceNow Research.

Pal has been involved in AI and machine learning research for over twenty-five years and has published extensively on large-scale language modelling methods and generative modelling techniques. He has a PhD in computer science from the University of Waterloo.

Current Students

Shubham Agarwal

Postdoctorate - HEC Montréal

Principal supervisor :

Paul Barde

PhD - McGill University

Principal supervisor :

Derek Nowrouzezahrai

paul.b.barde@gmail.com

Master's Research - Université de Montréal

Chris Beckham

PhD - Polytechnique Montréal

Georges Belanger Albarran

Master's Research - Université de Montréal

Simon Chamorro

Master's Research - Polytechnique Montréal

Can (Sam) Chen

PhD - McGill University

Principal supervisor :

PhD - Université de Montréal

Principal supervisor :

Glen Berseth

Amine El-hattami

PhD - Polytechnique Montréal

Chris Emezue

Master's Research - Université de Montréal

Co-supervisor :

Master's Research - Polytechnique Montréal

Roger Girgis

PhD - Polytechnique Montréal

Anthony Gosselin

Master's Research - Polytechnique Montréal

PhD - Université de Montréal

Co-supervisor :

PhD - Université de Montréal

PhD - Université de Montréal

Master's Research - Université de Montréal

Olga Luo

PhD - Université de Montréal

Z Luo

PhD - Polytechnique Montréal

Aristides Milios

PhD - Université de Montréal

PhD - Université de Montréal

Alexandre Piché

PhD - Université de Montréal

Jonathan Pilault

PhD - Polytechnique Montréal

Mats Leon Richter

Postdoctorate - Université de Montréal

Juan Rodriguez

PhD - École de technologie suprérieure

Luke Rowe

PhD - Université de Montréal

Principal supervisor :

Julien Roy

PhD - Polytechnique Montréal

Co-supervisor :

PhD - Polytechnique Montréal

Principal supervisor :

PhD - McGill University

Principal supervisor :

PhD - Université de Montréal

Martin Weiss

PhD - Polytechnique Montréal

Direct Behavior Specification via Constrained Reinforcement Learning

Blog Posts

August 31, 2022

Julien Roy

Roger Girgis

Joshua Romoff

Pierre-Luc Bacon

Chris Pal

Read the article

Publications

XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference

Jo˜ao Monteiro

Étienne Marcotte

Pierre-Andre Noel

Valentina Zantedeschi

David Vazquez

Nicolas Chapados

Perouz Taslakian

In-context learning (ICL) approaches typically leverage prompting to condition decoder-only language model generation on reference informati… (see more)on. Just-in-time processing of a context is inefficient due to the quadratic cost of self-attention operations, and caching is desirable. However, caching transformer states can easily require almost as much space as the model parameters. When the right context isn't known in advance, caching ICL can be challenging. This work addresses these limitations by introducing models that, inspired by the encoder-decoder architecture, use cross-attention to condition generation on reference text without the prompt. More precisely, we leverage pre-trained decoder-only models and only train a small number of added layers. We use Question-Answering (QA) as a testbed to evaluate the ability of our models to perform conditional generation and observe that they outperform ICL, are comparable to fine-tuned prompted LLMs, and drastically reduce the space footprint relative to standard KV caching by two orders of magnitude.

2024-04-23

ArXiv (preprint)

Language Models Can Reduce Asymmetry in Information Markets

Nasim Rahaman

Martin Weiss

Manuel Wüthrich

Yoshua Bengio

Erran L. Li

Bernhard Schölkopf

2024-03-21

ArXiv (preprint)

Multi-Resolution Continuous Normalizing Flows

Vikram Voleti

Chris Finlay

Adam M. Oberman

2024-03-21

Annals of Mathematics and Artificial Intelligence (published)

IntentGPT: Few-shot Intent Discovery with Large Language Models

Juan A. Rodriguez

Nicholas Botzer

David Vazquez

Marco Pedersoli

Issam Hadj Laradji

2024-03-11

ICLR.cc/2024/Workshop/LLMAgents (poster)

Self-evaluation and self-prompting to improve the reliability of LLMs

Alexandre Piché

Aristides Milios

Dzmitry Bahdanau

In order to safely deploy Large Language Models (LLMs), they must be capable of dynamically adapting their behavior based on their level of … (see more)knowledge and uncertainty associated with specific topics. This adaptive behavior, which we refer to as self-restraint, is non-trivial to teach since it depends on the internal knowledge of an LLM. By default, LLMs are trained to maximize the next token likelihood which does not teach the model to modulate its answer based on its level of uncertainty. In order to learn self-restraint, we devise a simple objective that can encourage the model to produce generation that the model is confident in. To optimize this objective, we introduce ReSearch, an iterative search algorithm based on self-evaluation and self-prompting. Our method results in fewer hallucinations overall, both for known and unknown topics, as the model learns to selectively restrain itself. In addition, our method elegantly incorporates the ability to decline, when the model assesses that it cannot provide a response without a high proportion of hallucination.

2024-03-04

ICLR.cc/2024/Workshop/SeT_LLM (published)

Self-evaluation and self-prompting to improve the reliability of LLMs

Alexandre Piché

Aristides Milios

Dzmitry Bahdanau

2024-03-04

ICLR.cc/2024/Workshop/SeT_LLM (published)

LitLLM: A Toolkit for Scientific Literature Review

Shubham Agarwal

Issam Hadj Laradji

Laurent Charlin

Conducting literature reviews for scientific papers is essential for understanding research, its limitations, and building on existing work.… (see more) It is a tedious task which makes an automatic literature review generator appealing. Unfortunately, many existing works that generate such reviews using Large Language Models (LLMs) have significant limitations. They tend to hallucinate-generate non-actual information-and ignore the latest research they have not been trained on. To address these limitations, we propose a toolkit that operates on Retrieval Augmented Generation (RAG) principles, specialized prompting and instructing techniques with the help of LLMs. Our system first initiates a web search to retrieve relevant papers by summarizing user-provided abstracts into keywords using an off-the-shelf LLM. Authors can enhance the search by supplementing it with relevant papers or keywords, contributing to a tailored retrieval process. Second, the system re-ranks the retrieved papers based on the user-provided abstract. Finally, the related work section is generated based on the re-ranked results and the abstract. There is a substantial reduction in time and effort for literature review compared to traditional methods, establishing our toolkit as an efficient alternative. Our open-source toolkit is accessible at https://github.com/shubhamagarwal92/LitLLM and Huggingface space (https://huggingface.co/spaces/shubhamagarwal92/LitLLM) with the video demo at https://youtu.be/E2ggOZBAFw0.

2024-02-02

ArXiv (preprint)

Würstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models

Pablo Pernias

Dominic Rampas

Mats Leon Richter

Marc Aubreville

2024-01-16

ICLR.cc/2024/Conference (oral)

Exploring validation metrics for ofﬂine model-based optimisation

Christopher Beckham

Alexandre Piché

David Vazquez

In ofﬂine model-based optimisation (MBO) we are interested in using machine learning to de-sign candidates that maximise some measure of d… (see more)esirability through an expensive but real-world scoring process. Ofﬂine MBO tries to approximate this expensive scoring function and use that to evaluate generated designs, however evaluation is non-exact because one approximation is being evaluated with another. Instead, we ask ourselves: if we did have the real world scoring function at hand, what cheap-to-compute validation metrics would correlate best with this? Since the real-world scoring function is available for simulated MBO datasets, insights obtained from this can be transferred over to real-world ofﬂine MBO tasks where the real-world scoring function is expensive to compute. To address this, we propose a conceptual evaluation framework that is amenable to measuring extrapolation, and apply this to conditional denoising diffusion models. Empirically, we ﬁnd that two validation metrics – agreement and Frechet distance – correlate quite well with the ground truth. When there is high variability in conditional generation, feedback is required in the form of an approximated version of the real-world scoring function. Furthermore, we ﬁnd that generating high-scoring samples may require heavily weighting the generative model in favour of sample quality, potentially at the cost of sample diversity.

2024-01-01

Trans. Mach. Learn. Res. (published)

dblp.uni-trier.de

Reinforcement Learning for Blind Stair Climbing with Legged and Wheeled-Legged Robots

Simon Chamorro

Victor Klemm

Miguel de La Iglesia Valls

Roland Siegwart

2024-01-01

ICRA (published)

Würstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models

Pablo Pernias

Dominic Rampas

Mats Leon Richter

Marc Aubreville

2024-01-01

International Conference on Learning Representations (published)

Capture the Flag: Uncovering Data Insights with Large Language Models

Issam Hadj Laradji

Perouz Taslakian

Sai Rajeswar

Valentina Zantedeschi

Alexandre Lacoste

Nicolas Chapados

David Vazquez

Alexandre Drouin

The extraction of a small number of relevant insights from vast amounts of data is a crucial component of data-driven decision-making. Howev… (see more)er, accomplishing this task requires considerable technical skills, domain expertise, and human labor. This study explores the potential of using Large Language Models (LLMs) to automate the discovery of insights in data, leveraging recent advances in reasoning and code generation techniques. We propose a new evaluation methodology based on a"capture the flag"principle, measuring the ability of such models to recognize meaningful and pertinent information (flags) in a dataset. We further propose two proof-of-concept agents, with different inner workings, and compare their ability to capture such flags in a real-world sales dataset. While the work reported here is preliminary, our results are sufficiently interesting to mandate future exploration by the community.

2023-12-21

ArXiv (preprint)