Publications

On learning history-based policies for controlling Markov decision processes
Gandharv Patil
Aditya Mahajan
Doina Precup
Reinforcementlearning(RL)folkloresuggeststhathistory-basedfunctionapproximationmethods,suchas recurrent neural nets or history-based state a… (voir plus)bstraction, perform better than their memory-less counterparts, due to the fact that function approximation in Markov decision processes (MDP) can be viewed as inducing a Partially observable MDP. However, there has been little formal analysis of such history-based algorithms, as most existing frameworks focus exclusively on memory-less features. In this paper, we introduce a theoretical framework for studying the behaviour of RL algorithms that learn to control an MDP using history-based feature abstraction mappings. Furthermore, we use this framework to design a practical RL algorithm and we numerically evaluate its effectiveness on a set of continuous control tasks.
2024-05-01
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
Maximum entropy GFlowNets with soft Q-learning
Sobhan Mohammadpour
Emmanuel Bengio
Emma Frejinger
Pierre-Luc Bacon
Generative Flow Networks (GFNs) have emerged as a powerful tool for sampling discrete objects from unnormalized distributions, offering a sc… (voir plus)alable alternative to Markov Chain Monte Carlo (MCMC) methods. While GFNs draw inspiration from maximum entropy reinforcement learning (RL), the connection between the two has largely been unclear and seemingly applicable only in specific cases. This paper addresses the connection by constructing an appropriate reward function, thereby establishing an exact relationship between GFNs and maximum entropy RL. This construction allows us to introduce maximum entropy GFNs, which, in contrast to GFNs with uniform backward policy, achieve the maximum entropy attainable by GFNs without constraints on the state space.
2024-05-01
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
Proving Linear Mode Connectivity of Neural Networks via Optimal Transport
Damien Ferbach
Baptiste Goujaud
Gauthier Gidel
Aymeric Dieuleveut
The energy landscape of high-dimensional non-convex optimization problems is crucial to understanding the effectiveness of modern deep neura… (voir plus)l network architectures. Recent works have experimentally shown that two different solutions found after two runs of a stochastic training are often connected by very simple continuous paths (e.g., linear) modulo a permutation of the weights. In this paper, we provide a framework theoretically explaining this empirical observation. Based on convergence rates in Wasserstein distance of empirical measures, we show that, with high probability, two wide enough two-layer neural networks trained with stochastic gradient descent are linearly connected. Additionally, we express upper and lower bounds on the width of each layer of two deep neural networks with independent neuron weights to be linearly connected. Finally, we empirically demonstrate the validity of our approach by showing how the dimension of the support of the weight distribution of neurons, which dictates Wasserstein convergence rates is correlated with linear mode connectivity.
2024-05-01
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
Simulating Weighted Automata over Sequences and Trees with Transformers
Michael Rizvi
Maude Lizaire
Clara Lacroce
Guillaume Rabusseau
Transformers are ubiquitous models in the natural language processing (NLP) community and have shown impressive empirical successes in the p… (voir plus)ast few years. However, little is understood about how they reason and the limits of their computational capabilities. These models do not process data sequentially, and yet outperform sequential neural models such as RNNs. Recent work has shown that these models can compactly simulate the sequential reasoning abilities of deterministic finite automata (DFAs). This leads to the following question: can transformers simulate the reasoning of more complex finite state machines? In this work, we show that transformers can simulate weighted finite automata (WFAs), a class of models which subsumes DFAs, as well as weighted tree automata (WTA), a generalization of weighted automata to tree structured inputs. We prove these claims formally and provide upper bounds on the sizes of the transformer models needed as a function of the number of states the target automata. Empirically, we perform synthetic experiments showing that transformers are able to learn these compact solutions via standard gradient-based training.
2024-05-01
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
Simulation-Free Schrödinger Bridges via Score and Flow Matching
Alexander Tong
Nikolay Malkin
Kilian Fatras
Lazar Atanackovic
Yanlei Zhang
Guillaume Huguet
Guy Wolf
Yoshua Bengio
We present simulation-free score and flow matching ([SF]…
2024-05-01
Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
Stochastic Frank-Wolfe: Unified Analysis and Zoo of Special Cases
Ruslan Nazykov
Aleksandr Shestakov
Vladimir Solodkin
Aleksandr Beznosikov
Gauthier Gidel
Alexander Gasnikov
The Conditional Gradient (or Frank-Wolfe) method is one of the most well-known methods for solving constrained optimization problems appeari… (voir plus)ng in various machine learning tasks. The simplicity of iteration and applicability to many practical problems helped the method to gain popularity in the community. In recent years, the Frank-Wolfe algorithm received many different extensions, including stochastic modifications with variance reduction and coordinate sampling for training of huge models or distributed variants for big data problems. In this paper, we present a unified convergence analysis of the Stochastic Frank-Wolfe method that covers a large number of particular practical cases that may have completely different nature of stochasticity, intuitions and application areas. Our analysis is based on a key parametric assumption on the variance of the stochastic gradients. But unlike most works on unified analysis of other methods, such as SGD, we do not assume an unbiasedness of the real gradient estimation. We conduct analysis for convex and non-convex problems due to the popularity of both cases in machine learning. With this general theoretical framework, we not only cover rates of many known methods, but also develop numerous new methods. This shows the flexibility of our approach in developing new algorithms based on the Conditional Gradient approach. We also demonstrate the properties of the new methods through numerical experiments.
2024-05-01
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
On the Privacy of Selection Mechanisms with Gaussian Noise
Jonathan Lebensold
Doina Precup
Borja Balle
Report Noisy Max and Above Threshold are two classical differentially private (DP) selection mechanisms. Their output is obtained by adding … (voir plus)noise to a sequence of low-sensitivity queries and reporting the identity of the query whose (noisy) answer satisfies a certain condition. Pure DP guarantees for these mechanisms are easy to obtain when Laplace noise is added to the queries. On the other hand, when instantiated using Gaussian noise, standard analyses only yield approximate DP guarantees despite the fact that the outputs of these mechanisms lie in a discrete space. In this work, we revisit the analysis of Report Noisy Max and Above Threshold with Gaussian noise and show that, under the additional assumption that the underlying queries are bounded, it is possible to provide pure ex-ante DP bounds for Report Noisy Max and pure ex-post DP bounds for Above Threshold. The resulting bounds are tight and depend on closed-form expressions that can be numerically evaluated using standard methods. Empirically we find these lead to tighter privacy accounting in the high privacy, low data regime. Further, we propose a simple privacy filter for composing pure ex-post DP guarantees, and use it to derive a fully adaptive Gaussian Sparse Vector Technique mechanism. Finally, we provide experiments on mobility and energy consumption datasets demonstrating that our Sparse Vector Technique is practically competitive with previous approaches and requires less hyper-parameter tuning.
2024-05-01
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
Weight-Sharing Regularization
Mehran Shakerinava
Motahareh Sohrabi
Siamak Ravanbakhsh
Simon Lacoste-Julien
Weight-sharing is ubiquitous in deep learning. Motivated by this, we propose a "weight-sharing regularization" penalty on the weights …
2024-05-01
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
2851: Operational Ontology for Oncology (O3) - Multi-professional society standard supporting AI
Charles S. Mayo
Mary U. Feng
Kristy K. Brock
Randi Kudner
Peter Balter
Jeffrey Buchsbaum
Amanda Caissie
Emily Daugherty
Andre Dekker
Clifton D. Fuller
Julian Hong
David Hong
Sophia Kamran
Evangelia Katsoulakis
J. Kildea
Andra Krauze
Jon Kruse
Todd McNutt
Michelle Mierzwa
Amy Moreno … (voir 5 de plus)
Jatinder Palta
Richard Popple
Thomas Purdie
Susan Yom
Xiao Ying
2024-04-30
Radiotherapy and Oncology (publié)
doi.org
Body size interacts with the structure of the central nervous system: A multi-center in vivo neuroimaging study
René Labounek
Monica T. Bondy
Amy Paulson
Sandrine Bédard
Mihael Abramovic
Eva Alonso‐Ortiz
Nicole Atcheson
Laura Barlow
Robert Barry
Markus Barth
Marco Battiston
Christian Büchel
Matthew D. Budde
Virginie Callot
Anna Combes
Benjamin De Leener
Maxime Descoteaux
Paulo Loureiro de Sousa
Marek Dostál
Julien Doyon … (voir 73 de plus)
Adam Dvorak
Falk Eippert
Karla R. Epperson
Kevin Epperson
Patrick Freund
Jürgen Finsterbusch
Alexandru Foias
Michela Fratini
Issei Fukunaga
Claudia A. M. Gandini Wheeler‐Kingshott
Giancarlo Germani
Guillaume Gilbert
Federico Giove
Francesco Grussu
Akifumi Hagiwara
Pierre‐Gilles Henry
Tomáš Horák
Masaaki Hori
James M. Joers
Kouhei Kamiya
Haleh Karbasforoushan
Miloš Keřkovský
Ali Khatibi
Joo-Won Kim
Nawal Kinany
Hagen H. Kitzler
Shannon Kolind
Yazhuo Kong
Petr Kudlička
Paul Kuntke
Nyoman D. Kurniawan
Sławomir Kuśmia
Maria Marcella Laganà
Cornelia Laule
Christine Law
Tobias Leutritz
Yaou Liu
Sara Llufriú
Sean Mackey
Allan R. Martin
Eloy Martínez‐Heras
Loan Mattera
Kristin P. O’Grady
Nico Papinutto
Daniel S. Papp
Deborah Pareto
Todd B. Parrish
Anna Pichiecchio
Ferrán Prados
Àlex Rovira
Marc J. Ruitenberg
Rebecca S. Samson
Giovanni Savini
Maryam Seif
Alan C. Seifert
Alex K. Smith
Seth A. Smith
Zachary A. Smith
Elisabeth Solana
Yuichi Suzuki
George Tackley
Alexandra Tinnermann
Jan Valosek
Dimitri Van De Ville
Marios Yiannakas
Kenneth A. Weber
Nikolaus Weiskopf
Richard G. Wise
Patrik O. Wyss
Junqian Xu
Julien Cohen‐Adad
Christophe Lenglet
Igor Nestrašil
Clinical research emphasizes the implementation of rigorous and reproducible study designs that rely on between-group matching or controllin… (voir plus)g for sources of biological variation such as subject’s sex and age. However, corrections for body size (i.e. height and weight) are mostly lacking in clinical neuroimaging designs. This study investigates the importance of body size parameters in their relationship with spinal cord (SC) and brain magnetic resonance imaging (MRI) metrics. Data were derived from a cosmopolitan population of 267 healthy human adults (age 30.1±6.6 years old, 125 females). We show that body height correlated strongly or moderately with brain gray matter (GM) volume, cortical GM volume, total cerebellar volume, brainstem volume, and cross-sectional area (CSA) of cervical SC white matter (CSA-WM; 0.44≤r≤0.62). In comparison, age correlated weakly with cortical GM volume, precentral GM volume, and cortical thickness (−0.21≥r≥−0.27). Body weight correlated weakly with magnetization transfer ratio in the SC WM, dorsal columns, and lateral corticospinal tracts (−0.20≥r≥−0.23). Body weight further correlated weakly with the mean diffusivity derived from diffusion tensor imaging (DTI) in SC WM (r=−0.20) and dorsal columns (−0.21), but only in males. CSA-WM correlated strongly or moderately with brain volumes (0.39≤r≤0.64), and weakly with precentral gyrus thickness and DTI-based fractional anisotropy in SC dorsal columns and SC lateral corticospinal tracts (−0.22≥r≥−0.25). Linear mixture of sex and age explained 26±10% of data variance in brain volumetry and SC CSA. The amount of explained variance increased at 33±11% when body height was added into the mixture model. Age itself explained only 2±2% of such variance. In conclusion, body size is a significant biological variable. Along with sex and age, body size should therefore be included as a mandatory variable in the design of clinical neuroimaging studies examining SC and brain structure.
2024-04-30
bioRxiv (prépublication)
doi.org
Dynamic System Modeling Using a Multisource Transfer Learning-Based Modular Neural Network for Industrial Application
Haoshan Duan
Xi Meng
Jian Tang
JunFei Qiao
Establishing an accurate model of dynamic systems poses a challenge for complex industrial processes. Due to the ability to handle complex t… (voir plus)asks, modular neural networks (MNN) have been widely applied to industrial process modeling. However, the phenomenon of domain drift caused by operating conditions may lead to a cold start of the model, which affects the performance of MNN. For this reason, a multisource transfer learning-based MNN (MSTL-MNN) is proposed in this study. First, the knowledge-driven transfer learning process is performed with domain similarity evaluation, knowledge extraction, and fusion, aiming to form an initial subnetwork in the target domain. Then, the positive transfer process of effective knowledge can avoid the cold start problem of MNN. Second, during the data-driven fine-tuning process, a regularized self-organizing long short-term memory algorithm is designed to fine-tune the structure and parameters of the initial subnetwork, which can improve the prediction performance of MNN. Meanwhile, relevant theoretical analysis is given to ensure the feasibility of MSTL-MNN. Finally, the effectiveness of the proposed method is confirmed by two benchmark simulations and a real industrial dataset of a municipal solid waste incineration process. Experimental results demonstrate the merits of MSTL-MNN for industrial applications.
2024-04-30
IEEE Transactions on Industrial Informatics (publié)
doi.org
Fairness-aware data-driven-based model predictive controller: A study on thermal energy storage in a residential building
Ying Sun
Fariborz Haghighat
Benjamin C. M. Fung
2024-04-30
Journal of Energy Storage (publié)
doi.org