Chris Pal

shubham.agarwal@mila.quebec

Biographie

Christopher Pal est titulaire d'une chaire en IA Canada-CIFAR, professeur titulaire à Polytechnique Montréal et professeur adjoint au Département d'informatique et de recherche opérationnelle (DIRO) de l'Université de Montréal. Il est également chercheur émérite à ServiceNow Research. Il est engagé dans la recherche sur l'intelligence artificielle et l'apprentissage automatique depuis plus de 25 ans, publiant souvent des travaux sur les méthodes de modélisation du langage à grande échelle et les techniques de modélisation générative. Il a obtenu un doctorat en informatique à l'Université de Waterloo.

Étudiants actuels

Shubham Agarwal

Postdoctorat - HEC

Superviseur⋅e principal⋅e :

Laurent Charlin

Site web

christopher.beckham@mila.quebec

Paul Barde

Doctorat - McGill

Superviseur⋅e principal⋅e :

Doctorat - Polytechnique

Georges Belanger Albarran

Maîtrise recherche - UdeM

georges.belangeralbarran@mila.quebec

Simon Chamorro

Maîtrise recherche - Polytechnique

chamorrs@mila.quebec

matthew.fortier@mila.quebec

Can (Sam) Chen

Doctorat - McGill

Superviseur⋅e principal⋅e :

Doctorat - Polytechnique

elhattaa@mila.quebec

Chris Emezue

Maîtrise recherche - UdeM

Co-superviseur⋅e :

Derek Nowrouzezahrai

chris.emezue@mila.quebec

Maîtrise recherche - Polytechnique

anthony.gosselin@mila.quebec

Roger Girgis

Doctorat - Polytechnique

girgisro@mila.quebec

Anthony Gosselin

Maîtrise recherche - Polytechnique

Doctorat - UdeM

Co-superviseur⋅e :

Sarath Chandar Anbil Parthipan

Doctorat - UdeM

Baudchon Hugo Baudchon

Collaborateur·rice de recherche - Université de Montréal

Superviseur⋅e principal⋅e :

Étienne Laliberté

hugo.baudchon@mila.quebec

Doctorat - UdeM

Collaborateur·rice de recherche

michelle.lin@mila.quebec

Doctorat - UdeM

Z Luo

Doctorat - Polytechnique

Doctorat - UdeM

Doctorat - UdeM

Doctorat - Polytechnique

Postdoctorat - UdeM

mats-leon.richter@mila.quebec

juan.rodriguez@mila.quebec

Juan Rodriguez

Doctorat - École de technologie suprérieure

Site web

Luke Rowe

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Liam Paull

luke.rowe@mila.quebec

aditya.sharma@mila.quebec

Julien Roy

Doctorat - Polytechnique

Co-superviseur⋅e :

Doctorat

Superviseur⋅e principal⋅e :

Amal Zouaq

Doctorat - McGill

Superviseur⋅e principal⋅e :

Derek Nowrouzezahrai

mattie.tesfaldet@mila.quebec

Doctorat - UdeM

Doctorat - Polytechnique

Billets de blogue

Direct Behavior Specification via Constrained Reinforcement Learning

31 août 2022

Spécification directe du comportement par apprentissage par renforcement sous contrainte

par

Julien Roy

Roger Girgis

Joshua Romoff

Pierre-Luc Bacon

Chris Pal

Lire l'article

Publications

StarVector: Generating Scalable Vector Graphics Code from Images

Juan A. Rodriguez

Shubham Agarwal

Issam Hadj Laradji

Pau Rodriguez

David Vazquez

Marco Pedersoli

Scalable Vector Graphics (SVGs) have become integral in modern image rendering applications due to their infinite scalability in resolution,… (voir plus) versatile usability, and editing capabilities. SVGs are particularly popular in the fields of web development and graphic design. Existing approaches for SVG modeling using deep learning often struggle with generating complex SVGs and are restricted to simpler ones that require extensive processing and simplification. This paper introduces StarVector, a multimodal SVG generation model that effectively integrates Code Generation Large Language Models (CodeLLMs) and vision models. Our approach utilizes a CLIP image encoder to extract visual representations from pixel-based images, which are then transformed into visual tokens via an adapter module. These visual tokens are pre-pended to the SVG token embeddings, and the sequence is modeled by the StarCoder model using next-token prediction, effectively learning to align the visual and code tokens. This enables StarVector to generate unrestricted SVGs that accurately represent pixel images. To evaluate StarVector's performance, we present SVG-Bench, a comprehensive benchmark for evaluating SVG methods across multiple datasets and relevant metrics. Within this benchmark, we introduce novel datasets including SVG-Stack, a large-scale dataset of real-world SVG examples, and use it to pre-train StarVector as a large foundation model for SVGs. Our results demonstrate significant enhancements in visual quality and complexity handling over current methods, marking a notable advancement in SVG generation technology. Code and models: https://github.com/joanrod/star-vector

2023-12-17

ArXiv (prépublication)

Capture the Flag: Uncovering Data Insights with Large Language Models

Issam Hadj Laradji

Perouz Taslakian

Sai Rajeswar

Valentina Zantedeschi

Alexandre Lacoste

Nicolas Chapados

David Vazquez

Alexandre Drouin

2023-11-07

NeurIPS.cc/2023/Workshop/FMDM (publié)

Are Diffusion Models Vision-And-Language Reasoners?

Benno Krojer

Elinor Poole-Dayan

Vikram Voleti

Siva Reddy

Text-conditioned image generation models have recently shown immense qualitative success using denoising diffusion processes. However, unlik… (voir plus)e discriminative vision-and-language models, it is a non-trivial task to subject these diffusion-based generative models to automatic fine-grained quantitative evaluation of high-level phenomena such as compositionality. Towards this goal, we perform two innovations. First, we transform diffusion-based models (in our case, Stable Diffusion) for any image-text matching (ITM) task using a novel method called DiffusionITM. Second, we introduce the Generative-Discriminative Evaluation Benchmark (GDBench) benchmark with 7 complex vision-and-language tasks, bias evaluation and detailed analysis. We find that Stable Diffusion + DiffusionITM is competitive on many tasks and outperforms CLIP on compositional tasks like like CLEVR and Winoground. We further boost its compositional performance with a transfer setup by fine-tuning on MS-COCO while retaining generative capabilities. We also measure the stereotypical bias in diffusion models, and find that Stable Diffusion 2.1 is, for the most part, less biased than Stable Diffusion 1.5. Overall, our results point in an exciting direction bringing discriminative and generative model evaluation closer. We will release code and benchmark setup soon.

Block-State Transformers

Jonathan Pilault

Mahan Fathi

Orhan Firat

Pierre-Luc Bacon

Ross Goroshin

2023-09-21

NeurIPS.cc/2023/Conference (poster)

Parallel-mentoring for Offline Model-based Optimization

Can Chen

Christopher Beckham

Zixuan Liu

Xue (Steve) Liu

Parallel-mentoring for Offline Model-based Optimization

Can Chen

Christopher Beckham

Zixuan Liu

Xue (Steve) Liu

We study offline model-based optimization to maximize a black-box objective function with a static dataset of designs and scores. These desi… (voir plus)gns encompass a variety of domains, including materials, robots, DNA sequences, and proteins. A common approach trains a proxy on the static dataset and performs gradient ascent to obtain new designs. However, this often results in poor designs due to the proxy inaccuracies for out-of-distribution designs. Recent studies indicate that (a) gradient ascent with a mean ensemble of proxies generally outperforms simple gradient ascent, and (b) a trained proxy provides weak ranking supervision signals for design selection. Motivated by (a) and (b), we propose

2023-09-21

NeurIPS.cc/2023/Conference (poster)

Neural Causal Structure Discovery from Interventions

Nan Rosemary Ke

Olexa Bilaniuk

Anirudh Goyal

Stefan Bauer

Hugo Larochelle

Bernhard Schölkopf

Michael Curtis Mozer

Yoshua Bengio

Recent promising results have generated a surge of interest in continuous optimization methods for causal discovery from observational data.… (voir plus) However, there are theoretical limitations on the identifiability of underlying structures obtained solely from observational data. Interventional data, on the other hand, provides richer information about the underlying data-generating process. Nevertheless, extending and applying methods designed for observational data to include interventions is a challenging problem. To address this issue, we propose a general framework based on neural networks to develop models that incorporate both observational and interventional data. Notably, our method can handle the challenging and realistic scenario where the identity of the intervened upon variable is unknown. We evaluate our proposed approach in the context of graph recovery, both de novo and from a partially-known edge set. Our method achieves strong benchmark results on various structure learning tasks, including structure recovery of synthetic graphs as well as standard graphs from the Bayesian Network Repository.

2023-09-10

TMLR (accepté)

Bridging the Gap Between Target Networks and Functional Regularization

Alexandre Piché

Valentin Thomas

Joseph Marino

Gian Maria Marconi

Rafael Pardinas

Mohammad Emtiyaz Khan

2023-09-06

TMLR (accepté)

Goal-conditioned GFlowNets for Controllable Multi-Objective Molecular Design

Julien Roy

Pierre-Luc Bacon

Emmanuel Bengio

In recent years, in-silico molecular design has received much attention from the machine learning community. When designing a new compound f… (voir plus)or pharmaceutical applications, there are usually multiple properties of such molecules that need to be optimised: binding energy to the target, synthesizability, toxicity, EC50, and so on. While previous approaches have employed a scalarization scheme to turn the multi-objective problem into a preference-conditioned single objective, it has been established that this kind of reduction may produce solutions that tend to slide towards the extreme points of the objective space when presented with a problem that exhibits a concave Pareto front. In this work we experiment with an alternative formulation of goal-conditioned molecular generation to obtain a more controllable conditional model that can uniformly explore solutions along the entire Pareto front.

2023-06-23

ICML.cc/2023/Workshop/DeployableGenerativeAI (publié)

Improving Generalization in Task-oriented Dialogues with Workflows and Action Plans

Stefania Raimondo

Xiaotian Liu

David Vazquez

Hector. Palacios

2023-06-02

ArXiv (prépublication)

Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models

Pablo Pernias

Dominic Rampas

Mats Leon Richter

Marc Aubreville

We introduce W\"urstchen, a novel architecture for text-to-image synthesis that combines competitive performance with unprecedented cost-eff… (voir plus)ectiveness for large-scale text-to-image diffusion models. A key contribution of our work is to develop a latent diffusion technique in which we learn a detailed but extremely compact semantic image representation used to guide the diffusion process. This highly compressed representation of an image provides much more detailed guidance compared to latent representations of language and this significantly reduces the computational requirements to achieve state-of-the-art results. Our approach also improves the quality of text-conditioned image generation based on our user preference study. The training requirements of our approach consists of 24,602 A100-GPU hours - compared to Stable Diffusion 2.1's 200,000 GPU hours. Our approach also requires less training data to achieve these results. Furthermore, our compact latent representations allows us to perform inference over twice as fast, slashing the usual costs and carbon footprint of a state-of-the-art (SOTA) diffusion model significantly, without compromising the end performance. In a broader comparison against SOTA models our approach is substantially more efficient and compares favorably in terms of image quality. We believe that this work motivates more emphasis on the prioritization of both performance and computational accessibility.

2023-06-01

ArXiv (prépublication)

ArK: Augmented Reality with Knowledge Interactive Emergent Ability

Qiuyuan Huang

J. Park

Abhinav Gupta

Pan Lu

Paul N. Bennett

Ran Gong

Subhojit Som

Baolin Peng

Owais Khan Mohammed

Yejin Choi

Jianfeng Gao

Despite the growing adoption of mixed reality and interactive AI agents, it remains challenging for these systems to generate high quality 2… (voir plus)D/3D scenes in unseen environments. The common practice requires deploying an AI agent to collect large amounts of data for model training for every new task. This process is costly, or even impossible, for many domains. In this study, we develop an infinite agent that learns to transfer knowledge memory from general foundation models (e.g. GPT4, DALLE) to novel domains or scenarios for scene understanding and generation in the physical or virtual world. The heart of our approach is an emerging mechanism, dubbed Augmented Reality with Knowledge Inference Interaction (ArK), which leverages knowledge-memory to generate scenes in unseen physical world and virtual reality environments. The knowledge interactive emergent ability (Figure 1) is demonstrated as the observation learns i) micro-action of cross-modality: in multi-modality models to collect a large amount of relevant knowledge memory data for each interaction task (e.g., unseen scene understanding) from the physical reality; and ii) macro-behavior of reality-agnostic: in mix-reality environments to improve interactions that tailor to different characterized roles, target variables, collaborative information, and so on. We validate the effectiveness of ArK on the scene generation and editing tasks. We show that our ArK approach, combined with large foundation models, significantly improves the quality of generated 2D/3D scenes, compared to baselines, demonstrating the potential benefit of incorporating ArK in generative AI for applications such as metaverse and gaming simulation.

2023-05-01

ArXiv (prépublication)