Publications

On learning history-based policies for controlling Markov decision processes

Reinforcementlearning(RL)folkloresuggeststhathistory-basedfunctionapproximationmethods,suchas recurrent neural nets or history-based state a… (see more)bstraction, perform better than their memory-less counterparts, due to the fact that function approximation in Markov decision processes (MDP) can be viewed as inducing a Partially observable MDP. However, there has been little formal analysis of such history-based algorithms, as most existing frameworks focus exclusively on memory-less features. In this paper, we introduce a theoretical framework for studying the behaviour of RL algorithms that learn to control an MDP using history-based feature abstraction mappings. Furthermore, we use this framework to design a practical RL algorithm and we numerically evaluate its effectiveness on a set of continuous control tasks.

2024-05-01

International Conference on Artificial Intelligence and Statistics (Accept (Poster))

doi.org

proceedings.mlr.press

Maximum entropy GFlowNets with soft Q-learning

Sobhan Mohammadpour

Emmanuel Bengio

Emma Frejinger

Pierre-Luc Bacon

Generative Flow Networks (GFNs) have emerged as a powerful tool for sampling discrete objects from unnormalized distributions, offering a sc… (see more)alable alternative to Markov Chain Monte Carlo (MCMC) methods. While GFNs draw inspiration from maximum entropy reinforcement learning (RL), the connection between the two has largely been unclear and seemingly applicable only in specific cases. This paper addresses the connection by constructing an appropriate reward function, thereby establishing an exact relationship between GFNs and maximum entropy RL. This construction allows us to introduce maximum entropy GFNs, which, in contrast to GFNs with uniform backward policy, achieve the maximum entropy attainable by GFNs without constraints on the state space.

2024-05-01

International Conference on Artificial Intelligence and Statistics (Accept (Poster))

doi.org

proceedings.mlr.press

Proving Linear Mode Connectivity of Neural Networks via Optimal Transport

Damien Ferbach

Baptiste Goujaud

Gauthier Gidel

Aymeric Dieuleveut

The energy landscape of high-dimensional non-convex optimization problems is crucial to understanding the effectiveness of modern deep neura… (see more)l network architectures. Recent works have experimentally shown that two different solutions found after two runs of a stochastic training are often connected by very simple continuous paths (e.g., linear) modulo a permutation of the weights. In this paper, we provide a framework theoretically explaining this empirical observation. Based on convergence rates in Wasserstein distance of empirical measures, we show that, with high probability, two wide enough two-layer neural networks trained with stochastic gradient descent are linearly connected. Additionally, we express upper and lower bounds on the width of each layer of two deep neural networks with independent neuron weights to be linearly connected. Finally, we empirically demonstrate the validity of our approach by showing how the dimension of the support of the weight distribution of neurons, which dictates Wasserstein convergence rates is correlated with linear mode connectivity.

2024-05-01

International Conference on Artificial Intelligence and Statistics (Accept (Poster))

doi.org

proceedings.mlr.press

Simulating Weighted Automata over Sequences and Trees with Transformers

Michael Rizvi

Maude Lizaire

Clara Lacroce

Guillaume Rabusseau

Transformers are ubiquitous models in the natural language processing (NLP) community and have shown impressive empirical successes in the p… (see more)ast few years. However, little is understood about how they reason and the limits of their computational capabilities. These models do not process data sequentially, and yet outperform sequential neural models such as RNNs. Recent work has shown that these models can compactly simulate the sequential reasoning abilities of deterministic finite automata (DFAs). This leads to the following question: can transformers simulate the reasoning of more complex finite state machines? In this work, we show that transformers can simulate weighted finite automata (WFAs), a class of models which subsumes DFAs, as well as weighted tree automata (WTA), a generalization of weighted automata to tree structured inputs. We prove these claims formally and provide upper bounds on the sizes of the transformer models needed as a function of the number of states the target automata. Empirically, we perform synthetic experiments showing that transformers are able to learn these compact solutions via standard gradient-based training.

2024-05-01

International Conference on Artificial Intelligence and Statistics (Accept (Poster))

doi.org

proceedings.mlr.press

Simulation-Free Schrödinger Bridges via Score and Flow Matching

Nikolay Malkin

Yanlei Zhang

We present simulation-free score and flow matching ([SF]…

2024-05-01

Artificial Intelligence and Statistics (Accept (Poster))

doi.org

proceedings.mlr.press

Stochastic Frank-Wolfe: Unified Analysis and Zoo of Special Cases

Ruslan Nazykov

Aleksandr Shestakov

Vladimir Solodkin

Aleksandr Beznosikov

Gauthier Gidel

Alexander Gasnikov

The Conditional Gradient (or Frank-Wolfe) method is one of the most well-known methods for solving constrained optimization problems appeari… (see more)ng in various machine learning tasks. The simplicity of iteration and applicability to many practical problems helped the method to gain popularity in the community. In recent years, the Frank-Wolfe algorithm received many different extensions, including stochastic modifications with variance reduction and coordinate sampling for training of huge models or distributed variants for big data problems. In this paper, we present a unified convergence analysis of the Stochastic Frank-Wolfe method that covers a large number of particular practical cases that may have completely different nature of stochasticity, intuitions and application areas. Our analysis is based on a key parametric assumption on the variance of the stochastic gradients. But unlike most works on unified analysis of other methods, such as SGD, we do not assume an unbiasedness of the real gradient estimation. We conduct analysis for convex and non-convex problems due to the popularity of both cases in machine learning. With this general theoretical framework, we not only cover rates of many known methods, but also develop numerous new methods. This shows the flexibility of our approach in developing new algorithms based on the Conditional Gradient approach. We also demonstrate the properties of the new methods through numerical experiments.

2024-05-01

International Conference on Artificial Intelligence and Statistics (Accept (Poster))

doi.org

proceedings.mlr.press

On the Privacy of Selection Mechanisms with Gaussian Noise

Jonathan Lebensold

Doina Precup

Borja Balle

Report Noisy Max and Above Threshold are two classical differentially private (DP) selection mechanisms. Their output is obtained by adding … (see more)noise to a sequence of low-sensitivity queries and reporting the identity of the query whose (noisy) answer satisfies a certain condition. Pure DP guarantees for these mechanisms are easy to obtain when Laplace noise is added to the queries. On the other hand, when instantiated using Gaussian noise, standard analyses only yield approximate DP guarantees despite the fact that the outputs of these mechanisms lie in a discrete space. In this work, we revisit the analysis of Report Noisy Max and Above Threshold with Gaussian noise and show that, under the additional assumption that the underlying queries are bounded, it is possible to provide pure ex-ante DP bounds for Report Noisy Max and pure ex-post DP bounds for Above Threshold. The resulting bounds are tight and depend on closed-form expressions that can be numerically evaluated using standard methods. Empirically we find these lead to tighter privacy accounting in the high privacy, low data regime. Further, we propose a simple privacy filter for composing pure ex-post DP guarantees, and use it to derive a fully adaptive Gaussian Sparse Vector Technique mechanism. Finally, we provide experiments on mobility and energy consumption datasets demonstrating that our Sparse Vector Technique is practically competitive with previous approaches and requires less hyper-parameter tuning.

2024-05-01

International Conference on Artificial Intelligence and Statistics (Accept (Poster))

doi.org

proceedings.mlr.press

Weight-Sharing Regularization

Weight-sharing is ubiquitous in deep learning. Motivated by this, we propose a "weight-sharing regularization" penalty on the weights …

2024-05-01

International Conference on Artificial Intelligence and Statistics (Accept (Poster))

doi.org

proceedings.mlr.press

2851: Operational Ontology for Oncology (O3) - Multi-professional society standard supporting AI

Charles S. Mayo

Mary U. Feng

Kristy K. Brock

Randi Kudner

Peter Balter

Jeffrey Buchsbaum

Amanda Caissie

Emily Daugherty

Andre Dekker

Clifton D. Fuller

Julian Hong

David Hong

Sophia Kamran

Evangelia Katsoulakis

J. Kildea

Andra Krauze

Jon Kruse

Todd McNutt

Michelle Mierzwa

Amy Moreno … (see 5 more)

Jatinder Palta

Richard Popple

Thomas Purdie

Susan Yom

Xiao Ying

2024-04-30

Radiotherapy and Oncology (published)

doi.org

Body size interacts with the structure of the central nervous system: A multi-center in vivo neuroimaging study

René Labounek

Monica T. Bondy

Amy Paulson

Sandrine Bédard

Mihael Abramovic

Eva Alonso‐Ortiz

Nicole Atcheson

Laura Barlow

Robert Barry

Markus Barth

Marco Battiston

Christian Büchel

Matthew D. Budde

Virginie Callot

Anna Combes

Benjamin De Leener

Maxime Descoteaux

Paulo Loureiro de Sousa

Marek Dostál

Julien Doyon … (see 73 more)

Adam Dvorak

Falk Eippert

Karla R. Epperson

Kevin Epperson

Patrick Freund

Jürgen Finsterbusch

Alexandru Foias

Michela Fratini

Issei Fukunaga

Claudia A. M. Gandini Wheeler‐Kingshott

Giancarlo Germani

Guillaume Gilbert

Federico Giove

Francesco Grussu

Akifumi Hagiwara

Pierre‐Gilles Henry

Tomáš Horák

Masaaki Hori

James M. Joers

Kouhei Kamiya

Haleh Karbasforoushan

Miloš Keřkovský

Ali Khatibi

Joo-Won Kim

Nawal Kinany

Hagen H. Kitzler

Shannon Kolind

Yazhuo Kong

Petr Kudlička

Paul Kuntke

Nyoman D. Kurniawan

Sławomir Kuśmia

Maria Marcella Laganà

Cornelia Laule

Christine Law

Tobias Leutritz

Yaou Liu

Sara Llufriú

Sean Mackey

Allan R. Martin

Eloy Martínez‐Heras

Loan Mattera

Kristin P. O’Grady

Nico Papinutto

Daniel S. Papp

Deborah Pareto

Todd B. Parrish

Anna Pichiecchio

Ferrán Prados

Àlex Rovira

Marc J. Ruitenberg

Rebecca S. Samson

Giovanni Savini

Maryam Seif

Alan C. Seifert

Alex K. Smith

Seth A. Smith

Zachary A. Smith

Elisabeth Solana

Yuichi Suzuki

George Tackley

Alexandra Tinnermann

Jan Valosek

Dimitri Van De Ville

Marios Yiannakas

Kenneth A. Weber

Nikolaus Weiskopf

Richard G. Wise

Patrik O. Wyss

Junqian Xu

Julien Cohen‐Adad

Christophe Lenglet

Igor Nestrašil

Clinical research emphasizes the implementation of rigorous and reproducible study designs that rely on between-group matching or controllin… (see more)g for sources of biological variation such as subject’s sex and age. However, corrections for body size (i.e. height and weight) are mostly lacking in clinical neuroimaging designs. This study investigates the importance of body size parameters in their relationship with spinal cord (SC) and brain magnetic resonance imaging (MRI) metrics. Data were derived from a cosmopolitan population of 267 healthy human adults (age 30.1±6.6 years old, 125 females). We show that body height correlated strongly or moderately with brain gray matter (GM) volume, cortical GM volume, total cerebellar volume, brainstem volume, and cross-sectional area (CSA) of cervical SC white matter (CSA-WM; 0.44≤r≤0.62). In comparison, age correlated weakly with cortical GM volume, precentral GM volume, and cortical thickness (−0.21≥r≥−0.27). Body weight correlated weakly with magnetization transfer ratio in the SC WM, dorsal columns, and lateral corticospinal tracts (−0.20≥r≥−0.23). Body weight further correlated weakly with the mean diffusivity derived from diffusion tensor imaging (DTI) in SC WM (r=−0.20) and dorsal columns (−0.21), but only in males. CSA-WM correlated strongly or moderately with brain volumes (0.39≤r≤0.64), and weakly with precentral gyrus thickness and DTI-based fractional anisotropy in SC dorsal columns and SC lateral corticospinal tracts (−0.22≥r≥−0.25). Linear mixture of sex and age explained 26±10% of data variance in brain volumetry and SC CSA. The amount of explained variance increased at 33±11% when body height was added into the mixture model. Age itself explained only 2±2% of such variance. In conclusion, body size is a significant biological variable. Along with sex and age, body size should therefore be included as a mandatory variable in the design of clinical neuroimaging studies examining SC and brain structure.

2024-04-30

bioRxiv (preprint)

doi.org

Dynamic System Modeling Using a Multisource Transfer Learning-Based Modular Neural Network for Industrial Application

Haoshan Duan

Xi Meng

Jian Tang

JunFei Qiao

Establishing an accurate model of dynamic systems poses a challenge for complex industrial processes. Due to the ability to handle complex t… (see more)asks, modular neural networks (MNN) have been widely applied to industrial process modeling. However, the phenomenon of domain drift caused by operating conditions may lead to a cold start of the model, which affects the performance of MNN. For this reason, a multisource transfer learning-based MNN (MSTL-MNN) is proposed in this study. First, the knowledge-driven transfer learning process is performed with domain similarity evaluation, knowledge extraction, and fusion, aiming to form an initial subnetwork in the target domain. Then, the positive transfer process of effective knowledge can avoid the cold start problem of MNN. Second, during the data-driven fine-tuning process, a regularized self-organizing long short-term memory algorithm is designed to fine-tune the structure and parameters of the initial subnetwork, which can improve the prediction performance of MNN. Meanwhile, relevant theoretical analysis is given to ensure the feasibility of MSTL-MNN. Finally, the effectiveness of the proposed method is confirmed by two benchmark simulations and a real industrial dataset of a municipal solid waste incineration process. Experimental results demonstrate the merits of MSTL-MNN for industrial applications.

2024-04-30

IEEE Transactions on Industrial Informatics (published)

doi.org

Fairness-aware data-driven-based model predictive controller: A study on thermal energy storage in a residential building

Ying Sun

Fariborz Haghighat

Benjamin C. M. Fung

2024-04-30

Journal of Energy Storage (published)

doi.org

Mila Ventures Launchpad

AI Policy Compass

Virtual Information Sessions

Publications

Mila Ventures Launchpad

AI Policy Compass

Virtual Information Sessions

Popular keywords:

Publications