Publications

Generative Models for Decision Making
Bogdan Mazoure
Lisa Lee
Roberta Raileanu
Yilun Du
Walter Talbott
Katherine Metcalf
(Rex) Devon Hjelm
Alexander T Toshev
Generative Artificial Intelligence (AI) has made significant advancements in recent years, particularly with the development of large langua… (see more)ge and diffusion models. These generative models have demonstrated impressive capabilities in various tasks, such as text generation and image and audio synthesis. Concurrently, Reinforcement Learning (RL) has made significant strides in solving complex sequential decision-making problems with the help of external knowledge sources . However, there remains untapped potential in combining generative models with RL algorithms to tackle real-world challenges, particularly to improve sample efficiency of tabula rasa training by introducing priors from related domains such as visual question-answering, image captioning and image generation. This workshop aims to bring together researchers and practitioners from the fields of generative AI and reinforcement learning to explore the latest advances, methodologies, and applications. By fostering collaborations between these two domains, we intend to unlock new opportunities for addressing complex problems that lie at the intersection of both fields.
2024-03-08
ICLR.cc/2024/Workshop_Proposals (published)
openreview.net
openreview.net
Global AI Cultures
Rida Qadri
Fernando Diaz
Arjun Subramonian
Sunipa Dev
Georgina Emma Born
Mary L. Gray
Jessica Quaye
Rachel Bergmann
2024-03-08
ICLR.cc/2024/Workshop_Proposals (published)
openreview.net
openreview.net
Integrating Generative and Experimental Platforms or Biomolecular Design
Cheng-Hao Liu
Jarrid Rector-Brooks
Jason Yim
Soojung Yang
Sidney Lisanza
Francesca-Zhoufan Li
Pranam Chatterjee
Tommi Jaakkola
Regina Barzilay
David Baker
Frances H. Arnold
Yoshua Bengio
2024-03-08
ICLR.cc/2024/Workshop_Proposals (published)
openreview.net
openreview.net
Tackling Climate Change with Machine Learning: Fostering the Maturity of ML Applications for Climate Change
Shiva Madadkhani
Olivia Mendivil Ramos
Millie Chapman
Jesse Dunietz
Arthur Ouaknine
David Rolnick
Yoshua Bengio
2024-03-08
ICLR.cc/2024/Workshop_Proposals (published)
openreview.net
openreview.net
Machine learning and information theory concepts towards an AI Mathematician
Yoshua Bengio
Nikolay Malkin
The current state-of-the-art in artificial intelligence is impressive, especially in terms of mastery of language, but not so much in terms … (see more)of mathematical reasoning. What could be missing? Can we learn something useful about that gap from how the brains of mathematicians go about their craft? This essay builds on the idea that current deep learning mostly succeeds at system 1 abilities -- which correspond to our intuition and habitual behaviors -- but still lacks something important regarding system 2 abilities -- which include reasoning and robust uncertainty estimation. It takes an information-theoretical posture to ask questions about what constitutes an interesting mathematical statement, which could guide future work in crafting an AI mathematician. The focus is not on proving a given theorem but on discovering new and interesting conjectures. The central hypothesis is that a desirable body of theorems better summarizes the set of all provable statements, for example by having a small description length while at the same time being close (in terms of number of derivation steps) to many provable statements.
2024-03-07
ArXiv (preprint)
doi.org
arxiv.org
Personalized Negative Reservoir for Incremental Learning in Recommender Systems
Antonios Valkanas
Yuening Wang
Yingxue Zhang
Mark Coates
2024-03-06
ArXiv (preprint)
doi.org
arxiv.org
Stop Regressing: Training Value Functions via Classification for Scalable Deep RL
Jesse Farebrother
Jordi Orbay
Quan Ho Vuong
Adrien Ali Taiga
Yevgen Chebotar
Ted Xiao
A. Irpan
Sergey Levine
Pablo Samuel Castro
Aleksandra Faust
Aviral Kumar
Rishabh Agarwal
Value functions are a central component of deep reinforcement learning (RL). These functions, parameterized by neural networks, are trained … (see more)using a mean squared error regression objective to match bootstrapped target values. However, scaling value-based RL methods that use regression to large networks, such as high-capacity Transformers, has proven challenging. This difficulty is in stark contrast to supervised learning: by leveraging a cross-entropy classification loss, supervised methods have scaled reliably to massive networks. Observing this discrepancy, in this paper, we investigate whether the scalability of deep RL can also be improved simply by using classification in place of regression for training value functions. We demonstrate that value functions trained with categorical cross-entropy significantly improves performance and scalability in a variety of domains. These include: single-task RL on Atari 2600 games with SoftMoEs, multi-task RL on Atari with large-scale ResNets, robotic manipulation with Q-transformers, playing Chess without search, and a language-agent Wordle task with high-capacity Transformers, achieving state-of-the-art results on these domains. Through careful analysis, we show that the benefits of categorical cross-entropy primarily stem from its ability to mitigate issues inherent to value-based RL, such as noisy targets and non-stationarity. Overall, we argue that a simple shift to training value functions with categorical cross-entropy can yield substantial improvements in the scalability of deep RL at little-to-no cost.
2024-03-06
ArXiv (preprint)
doi.org
arxiv.org
Efficient Causal Graph Discovery Using Large Language Models
Thomas Jiralerspong
Xiaoyin Chen
Yash More
Vedant Shah
Yoshua Bengio
2024-03-05
ICLR.cc/2024/Workshop/AGI (poster)
doi.org
openreview.net
Explicit Knowledge Factorization Meets In-Context Learning: What Do We Gain?
Sarthak Mittal
Eric Elmoznino
Leo Gagnon
Sangnie Bhardwaj
Dhanya Sridhar
Guillaume Lajoie
2024-03-05
ICLR.cc/2024/Workshop/R2-FM (poster)
openreview.net
openreview.net
Optimisation of quantitative brain diffusion-relaxation MRI acquisition protocols with physics-informed machine learning.
Álvaro Planchuelo-Gómez
Maxime Descoteaux
Hugo Larochelle
Jana Hutter
Derek K. Jones
C. Tax
2024-03-05
Medical Image Analysis (published)
doi.org
Plant invasion in Mediterranean Europe: current hotspots and future scenarios
Luigi Cao Pinna
Laure Gallien
Laura J. Pollock
Irena Axmanová
Milan Chytrý
Marco Malavasi
Alicia T. R. Acosta
Juan Antonio Campos
Marta Carboni
The Mediterranean Basin has historically been subject to alien plant invasions that threaten its unique biodiversity. This seasonally dry an… (see more)d densely populated region is undergoing severe climatic and socioeconomic changes, and it is unclear whether these changes will worsen or mitigate plant invasions. Predictions are often biased, as species may not be in equilibrium in the invaded environment, depending on their invasion stage and ecological characteristics. To address future predictions uncertainty, we identified invasion hotspots across multiple biased modelling scenarios and ecological characteristics of successful invaders. We selected 92 alien plant species widespread in Mediterranean Europe and compiled data on their distribution in the Mediterranean and worldwide. We combined these data with environmental and propagule pressure variables to model global and regional species niches, and map their current and future habitat suitability. We identified invasion hotspots, examined their potential future shifts, and compared the results of different modelling strategies. Finally, we generalised our findings by using linear models to determine the traits and biogeographic features of invaders most likely to benefit from global change. Currently, invasion hotspots are found near ports and coastlines throughout Mediterranean Europe. However, many species occupy only a small portion of the environmental conditions to which they are preadapted, suggesting that their invasion is still an ongoing process. Future conditions will lead to declines in many currently widespread aliens, which will tend to move to higher elevations and latitudes. Our trait models indicate that future climates will generally favour species with conservative ecological strategies that can cope with reduced water availability, such as those with short stature and low specific leaf area. Taken together, our results suggest that in future environments, these conservative aliens will move farther from the introduction areas and upslope, threatening mountain ecosystems that have been spared from invasions so far.
2024-03-05
Ecography (published)
doi.org
Smoothness-Adaptive Sharpness-Aware Minimization for Finding Flatter Minima
Hiroki Naganuma
Junhyung Lyle Kim
Anastasios Kyrillidis
Ioannis Mitliagkas
The sharpness-aware minimization (SAM) procedure recently gained increasing attention due to its favorable generalization ability to unseen … (see more)data. SAM aims to find flatter (local) minima, utilizing a minimax objective. An immediate challenge in the application of SAM is the adjustment of two pivotal step sizes, which significantly influence its effectiveness. We introduce a novel, straightforward approach for adjusting step sizes that adapts to the smoothness of the objective function, thereby reducing the necessity for manual tuning. This method, termed Smoothness-Adaptive SAM (SA-SAM), not only simplifies the optimization process but also promotes the method's inherent tendency to converge towards flatter minima, enhancing performance in specific models.
2024-03-05
ICLR.cc/2024/Workshop/PML4LRS (poster)
openreview.net
openreview.net