Portrait de Antoine Clavaud n'est pas disponible

Antoine Clavaud

Alumni

Publications

Squeezing More from the Stream : Learning Representation Online for Streaming Reinforcement Learning
Nilaksh
Antoine Clavaud
Mathieu Reymond
Franccois Rivest
A. Chandar
AI Institute
Polytechnique Montr ´ eal
2026-02-09
arXiv (prépublication)
doi.org
arxiv.org
Squeezing More from the Stream : Learning Representation Online for Streaming Reinforcement Learning
Nilaksh
Antoine Clavaud
Mathieu Reymond
Francois Rivest
Sarath Chandar
In streaming Reinforcement Learning (RL), transitions are observed and discarded immediately after a single update. While this minimizes res… (voir plus)ource usage for on-device applications, it makes agents notoriously sample-inefficient, since value-based losses alone struggle to extract meaningful representations from transient data. We propose extending Self-Predictive Representations (SPR) to the streaming pipeline to maximize the utility of every observed frame. However, due to the highly correlated samples induced by the streaming regime, naively applying this auxiliary loss results in training instabilities. Thus, we introduce orthogonal gradient updates relative to the momentum target and resolve gradient conflicts arising from streaming-specific optimizers. Validated across the Atari, MinAtar, and Octax suites, our approach systematically outperforms existing streaming baselines. Latent-space analysis, including t-SNE visualizations and effective-rank measurements, confirms that our method learns significantly richer representations, bridging the performance gap caused by the absence of a replay buffer, while remaining efficient enough to train on just a few CPU cores.
2025-12-31
International Conference on Machine Learning (Accept (regular))
doi.org
openreview.net
CrystalGym: A New Benchmark for Materials Discovery Using Reinforcement Learning
Prashant Govindarajan
Mathieu Reymond
Antoine Clavaud
Mariano Phielipp
Santiago Miret
A. Chandar
*In silico* design and optimization of new materials primarily relies on high-accuracy atomic simulators that perform density functional the… (voir plus)ory (DFT) calculations. While recent works showcase the strong potential of machine learning to accelerate the material design process, they mostly consist of generative approaches that do not use direct DFT signals as feedback to improve training and generation mainly due to DFT's high computational cost. To aid the adoption of direct DFT signals in the materials design loop through online reinforcement learning (RL), we propose **CrystalGym**, an open-source RL environment for crystalline material discovery. Using CrystalGym, we benchmark value- and policy-based reinforcement learning algorithms for designing various crystals conditioned on target properties. Concretely, we optimize for challenging properties like the band gap, bulk modulus, and density, which are directly calculated from DFT in the environment. While none of the algorithms we benchmark solve all CrystalGym tasks, our extensive experiments and ablations show different sample efficiencies and ease of convergence to optimality for different algorithms and environment settings. Our goal is for CrystalGym to serve as a test bed for reinforcement learning researchers and material scientists to address these real-world design problems with practical applications. Furthermore, we introduce a novel class of challenges for reinforcement learning methods dealing with time-consuming reward signals, paving the way for future interdisciplinary research for machine learning motivated by real-world applications.
2025-03-02
AI4MAT @ International Conference on Learning Representations (spotlight)
doi.org
openreview.net
A Reinforcement Learning Pipeline for Band Gap-directed Crystal Generation
Prashant Govindarajan
Mathieu Reymond
Santiago Miret
Antoine Clavaud
Mariano Phielipp
Sarath Chandar
Property-driven AI-automated material discovery presents unique challenges owing to the complex nature of the chemical structural space and … (voir plus)computationally expensive simulations. For crystalline solids, the band gap is an important property for designing semiconductors and batteries. However, optimizing crystals for a target band gap is difficult and not well-explored. Reinforcement learning (RL) shows promise towards optimizing crystals, as it can freely explore the chemical space. However, it relies on regular band gap evaluations, which can only be accurately computed through expensive Density Functional Theory (DFT) simulations. In this study, we propose an active learning-inspired pipeline that combines RL and DFT simulations for optimizing crystal compositions given a target band gap. The pipeline includes an RL policy for predicting atom types and a band gap network that is fine-tuned with DFT data. Preliminary results indicate the need for furthering the state-of-the-art to address the inherent challenges of the problem.
2024-07-07
AI4Mat @ University of Natural Resources and Life Sciences, Vienna (poster)
openreview.net
openreview.net