Publications

Multi-phase black-hole feedback and a bright [CII] halo in a Lo-BAL quasar at $z\sim6.6$
Manuela Bischetti
Hyunseop 현섭 Choi 최
Fabrizio Fiore
Chiara Feruglio
Stefano Carniani
Valentina D'Odorico
Eduardo Banados
Huanqing Chen
Roberto Decarli
Simona Gallerani
J. Hlavacek-Larrondo
Samuel Lai
Karen M. Leighly
Chiara Mazzucchelli
Laurence Perreault-Levasseur
Roberta Tripodi
Fabian Walter
Feige Wang
Jinyi Yang
Maria Vittoria Zanchettin … (voir 1 de plus)
Yongda Zhu
2024-04-17
ArXiv (prépublication)
doi.org
arxiv.org
Multi-resolution Time-Series Transformer for Long-term Forecasting
Yitian Zhang
Liheng Ma
Soumyasundar Pal
Yingxue Zhang
Mark J. Coates
2024-04-17
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (publié)
doi.org
proceedings.mlr.press
Simulating Weighted Automata over Sequences and Trees with Transformers
Michael Rizvi
Maude Lizaire
Clara Lacroce
Guillaume Rabusseau
Transformers are ubiquitous models in the natural language processing (NLP) community and have shown impressive empirical successes in the p… (voir plus)ast few years. However, little is understood about how they reason and the limits of their computational capabilities. These models do not process data sequentially, and yet outperform sequential neural models such as RNNs. Recent work has shown that these models can compactly simulate the sequential reasoning abilities of deterministic finite automata (DFAs). This leads to the following question: can transformers simulate the reasoning of more complex finite state machines? In this work, we show that transformers can simulate weighted finite automata (WFAs), a class of models which subsumes DFAs, as well as weighted tree automata (WTA), a generalization of weighted automata to tree structured inputs. We prove these claims formally and provide upper bounds on the sizes of the transformer models needed as a function of the number of states the target automata. Empirically, we perform synthetic experiments showing that transformers are able to learn these compact solutions via standard gradient-based training.
2024-04-17
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (publié)
doi.org
proceedings.mlr.press
Simulation-Free Schrödinger Bridges via Score and Flow Matching
Alexander Tong
Nikolay Malkin
Kilian Fatras
Lazar Atanackovic
Yanlei Zhang
Guillaume Huguet
Guy Wolf
Yoshua Bengio
We present simulation-free score and flow matching ([SF]…
2024-04-17
International Conference on Artificial Intelligence and Statistics (publié)
doi.org
proceedings.mlr.press
Tackling the XAI Disagreement Problem with Regional Explanations
Gabriel Laberge
Yann Batiste Pequignot
Mario Marchand
Foutse Khomh
2024-04-17
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (publié)
proceedings.mlr.press
proceedings.mlr.press
On the Privacy of Selection Mechanisms with Gaussian Noise
Jonathan Lebensold
Doina Precup
Borja Balle
Report Noisy Max and Above Threshold are two classical differentially private (DP) selection mechanisms. Their output is obtained by adding … (voir plus)noise to a sequence of low-sensitivity queries and reporting the identity of the query whose (noisy) answer satisfies a certain condition. Pure DP guarantees for these mechanisms are easy to obtain when Laplace noise is added to the queries. On the other hand, when instantiated using Gaussian noise, standard analyses only yield approximate DP guarantees despite the fact that the outputs of these mechanisms lie in a discrete space. In this work, we revisit the analysis of Report Noisy Max and Above Threshold with Gaussian noise and show that, under the additional assumption that the underlying queries are bounded, it is possible to provide pure ex-ante DP bounds for Report Noisy Max and pure ex-post DP bounds for Above Threshold. The resulting bounds are tight and depend on closed-form expressions that can be numerically evaluated using standard methods. Empirically we find these lead to tighter privacy accounting in the high privacy, low data regime. Further, we propose a simple privacy filter for composing pure ex-post DP guarantees, and use it to derive a fully adaptive Gaussian Sparse Vector Technique mechanism. Finally, we provide experiments on mobility and energy consumption datasets demonstrating that our Sparse Vector Technique is practically competitive with previous approaches and requires less hyper-parameter tuning.
2024-04-17
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (publié)
doi.org
proceedings.mlr.press
Weight-Sharing Regularization
Mehran Shakerinava
Motahareh Sohrabi
Siamak Ravanbakhsh
Simon Lacoste-Julien
Weight-sharing is ubiquitous in deep learning. Motivated by this, we propose a "weight-sharing regularization" penalty on the weights …
2024-04-17
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (publié)
doi.org
proceedings.mlr.press
Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
Shreya Shankar
J.D. Zamfirescu-Pereira
Bjorn Hartmann
Aditya G Parameswaran
Ian Arawjo
2024-04-17
ArXiv (prépublication)
doi.org
arxiv.org
Asynchronous Algorithmic Alignment with Cocycles
Andrew Joseph Dudzik
Tamara von Glehn
Razvan Pascanu
Petar Veličković
State-of-the-art neural algorithmic reasoners make use of message passing in graph neural networks (GNNs). But typical GNNs blur the distinc… (voir plus)tion between the definition and invocation of the message function, forcing a node to send messages to its neighbours at every layer, synchronously. When applying GNNs to learn to execute dynamic programming algorithms, however, on most steps only a handful of the nodes would have meaningful updates to send. One, hence, runs the risk of inefficiencies by sending too much irrelevant data across the graph. But more importantly, many intermediate GNN steps have to learn the identity functions, which is a non-trivial learning problem. In this work, we explicitly separate the concepts of node state update and message function invocation. With this separation, we obtain a mathematical formulation that allows us to reason about asynchronous computation in both algorithms and neural networks. Our analysis yields several practical implementations of synchronous scalable GNN layers that are provably invariant under various forms of asynchrony.
2024-04-16
Proceedings of the Second Learning on Graphs Conference (publié)
doi.org
proceedings.mlr.press
Latent Space Representations of Neural Algorithmic Reasoners
Vladimir V. Mirjani'c
Razvan Pascanu
Petar Velivckovi'c University of Cambridge
Petar Veličković
Google Deepmind
Neural Algorithmic Reasoning (NAR) is a research area focused on designing neural architectures that can reliably capture classical computat… (voir plus)ion, usually by learning to execute algorithms. A typical approach is to rely on Graph Neural Network (GNN) architectures, which encode inputs in high-dimensional latent spaces that are repeatedly transformed during the execution of the algorithm. In this work we perform a detailed analysis of the structure of the latent space induced by the GNN when executing algorithms. We identify two possible failure modes: (i) loss of resolution, making it hard to distinguish similar values; (ii) inability to deal with values outside the range observed during training. We propose to solve the first issue by relying on a softmax aggregator, and propose to decay the latent space in order to deal with out-of-range values. We show that these changes lead to improvements on the majority of algorithms in the standard CLRS-30 benchmark when using the state-of-the-art Triplet-GMPNN processor. Our code is available at https://github.com/mirjanic/nar-latent-spaces
2024-04-16
Proceedings of the Second Learning on Graphs Conference (publié)
doi.org
proceedings.mlr.press
Government Interventions to Avert Future Catastrophic AI Risks
Yoshua Bengio
2024-04-14
Special Issue 5: Grappling With the Generative AI Revolution (publié)
doi.org
Improving microbial phylogeny with citizen science within a mass-market video game
Roman Sarrazin-Gendron
Parham Ghasemloo Gheidari
Alexander Butyaev
Timothy Keding
Eddie Cai
Jiayue Zheng
Renata Mutalova
Julien Mounthanyvong
Yuxue Zhu
Elena Nazarova
Chrisostomos Drogaris
Kornél Erhart
David Michael Joshua Mathieu Vincent Steven Dan Jonathan Bélanger Bouffard Davidson Falaise Fiset Hebert He
David Michael Joshua Mathieu Vincent Steven Dan Jonathan Seung Jonathan David Steve Ludger Bélanger
David Bélanger
Michael Bouffard
Joshua Davidson
Mathieu Falaise
Vincent Fiset
Steven Hébert … (voir 16 de plus)
Dan Hewitt
Jonathan Huot
Seung Kim
Jonathan Moreau-Genest
David Najjab
Steve Prince
Ludger Saintélien
Amélie Brouillette
Gabriel Richard
Randy Pitchford
Sébastien Caisse
Mathieu Blanchette
Daniel McDonald
Rob Knight
Attila Szantner
Jérôme Waldispühl
Citizen science video games are designed primarily for users already inclined to contribute to science, which severely limits their accessib… (voir plus)ility for an estimated community of 3 billion gamers worldwide. We created Borderlands Science (BLS), a citizen science activity that is seamlessly integrated within a popular commercial video game played by tens of millions of gamers. This integration is facilitated by a novel game-first design of citizen science games, in which the game design aspect has the highest priority, and a suitable task is then mapped to the game design. BLS crowdsources a multiple alignment task of 1 million 16S ribosomal RNA sequences obtained from human microbiome studies. Since its initial release on 7 April 2020, over 4 million players have solved more than 135 million science puzzles, a task unsolvable by a single individual. Leveraging these results, we show that our multiple sequence alignment simultaneously improves microbial phylogeny estimations and UniFrac effect sizes compared to state-of-the-art computational methods. This achievement demonstrates that hyper-gamified scientific tasks attract massive crowds of contributors and offers invaluable resources to the scientific community.
2024-04-14
Nature Biotechnology (publié)
doi.org