Portrait of Kirill Neklyudov

Kirill Neklyudov

Core Academic Member
kirill.neklyudov@mila.quebec
Assistant Professor, Université de Montréal, Mathematics and Statistics
Research Topics
Deep Learning
Dynamical Systems
Generative Models
Molecular Modeling
Probabilistic Models
Website
Google Scholar
X

Current Students

Vincent Guan
Independent visiting researcher - University of British Columbia
Andres Guzman Cordero
PhD - Université de Montréal
Website
Github
Google Scholar
Majdi Hassan
PhD - Université de Montréal
Principal supervisor :
Dominique Beaini
Website
Github
Google Scholar
Tobias Höppe
Independent visiting researcher - Helmholtz Zentrum München
Github
Google Scholar
Simon Paiva
PhD - Université de Montréal
Jessie Richter-Powell
Independent visiting researcher - Université de Montréal
Google Scholar
Marta Skreta
Postdoctorate - Université de Montréal
Charlie Tan
Independent visiting researcher - University of Oxford
Chuanru Wang
PhD - Université de Montréal
Website

Publications

Efficient Evolutionary Search Over Chemical Space with Large Language Models
Haorui Wang
Marta Skreta
Cher Tian Ser
Wenhao Gao
Lingkai Kong
Felix Streith-Kalthoff
Chenru Duan
Yuchen Zhuang
Yue Yu
Yanqiao Zhu 0001
Yuanqi Du
Alan Aspuru-Guzik
Kirill Neklyudov
Chao Zhang
Molecular discovery, when formulated as an optimization problem, presents significant computational challenges because optimization objectiv… (see more)es can be non-differentiable. Evolutionary Algorithms (EAs), often used to optimize black-box objectives in molecular discovery, traverse chemical space by performing random mutations and crossovers, leading to a large number of expensive objective evaluations. In this work, we ameliorate this shortcoming by incorporating chemistry-aware Large Language Models (LLMs) into EAs. Namely, we redesign crossover and mutation operations in EAs using LLMs trained on large corpora of chemical information. We perform extensive empirical studies on both commercial and open-source models on multiple tasks involving property optimization, molecular rediscovery, and structure-based drug design, demonstrating that the joint usage of LLMs with EAs yields superior performance over all baseline models across single- and multi-objective settings. We demonstrate that our algorithm improves both the quality of the final solution and convergence speed, thereby reducing the number of required objective evaluations. Our code is available at http://github.com/zoom-wang112358/MOLLEO
2024-06-23
ArXiv (preprint)
doi.org
arxiv.org
Meta Flow Matching: Integrating Vector Fields on the Wasserstein Manifold
Lazar Atanackovic
Xi Zhang
Brandon Amos
Mathieu Blanchette
Leo J Lee
Yoshua Bengio
Alexander Tong
Kirill Neklyudov
Numerous biological and physical processes can be modeled as systems of interacting samples evolving continuously over time, e.g. the dynami… (see more)cs of communicating cells or physical particles. Flow-based models allow for learning these dynamics at the population level --- they model the evolution of the entire distribution of samples. However, current flow-based models are limited to a single initial population and a set of predefined conditions which describe different dynamics. We propose
2024-06-17
ICML.cc/2024/Workshop/GRaM (published)
openreview.net
openreview.net
Efficient Evolutionary Search Over Chemical Space with Large Language Models
Haorui Wang
Marta Skreta
Cher Tian Ser
Wenhao Gao
Lingkai Kong
Felix Streith-Kalthoff
Chenru Duan
Yuchen Zhuang
Yue Yu
Yanqiao Zhu 0001
Yuanqi Du
Alan Aspuru-Guzik
Kirill Neklyudov
Chao Zhang
Molecular discovery, when formulated as an optimization problem, presents significant computational challenges because optimization objectiv… (see more)es can be non-differentiable. Evolutionary Algorithms (EAs), often used to optimize black-box objectives in molecular discovery, traverse chemical space by performing random mutations and crossovers, leading to a large number of expensive objective evaluations. In this work, we ameliorate this shortcoming by incorporating chemistry-aware Large Language Models (LLMs) into EAs. Namely, we redesign crossover and mutation operations in EAs using LLMs trained on large corpora of chemical information. We perform extensive empirical studies on both commercial and open-source models on multiple tasks involving property optimization, molecular rediscovery, and structure-based drug design, demonstrating that the joint usage of LLMs with EAs yields superior performance over all baseline models across single- and multi-objective settings. We demonstrate that our algorithm improves both the quality of the final solution and convergence speed, thereby reducing the number of required objective evaluations. Our code is available at http://github.com/zoom-wang112358/MOLLEO
2024-06-01
arXiv (published)
doi.org
arxiv.org
Structured Inverse-Free Natural Gradient Descent: Memory-Efficient & Numerically-Stable KFAC
Wu Lin
Felix Dangel
Runa Eschenhagen
Kirill Neklyudov
Agustinus Kristiadi
Richard E. Turner
Alireza Makhzani
Second-order methods such as KFAC can be useful for neural net training. However, they are often memory-inefficient since their precondition… (see more)ing Kronecker factors are dense, and numerically unstable in low precision as they require matrix inversion or decomposition. These limitations render such methods unpopular for modern mixed-precision training. We address them by (i) formulating an inverse-free KFAC update and (ii) imposing structures in the Kronecker factors, resulting in structured inverse-free natural gradient descent (SINGD). On modern neural networks, we show that SINGD is memory-efficient and numerically robust, in contrast to KFAC, and often outperforms AdamW even in half precision. Our work closes a gap between first- and second-order methods in modern low-precision training.
2024-01-01
ICML (published)
proceedings.mlr.press
arxiv.org