Publications

Convergence of Manifold Filter-Combine Networks

David R. Johnson

Joyce Chew

Edward De Brouwer

Deanna Needell

Michael Perlmutter

In order to better understand manifold neural networks (MNNs), we introduce Manifold Filter-Combine Networks (MFCNs). The filter-combine fra… (voir plus)mework parallels the popular aggregate-combine paradigm for graph neural networks (GNNs) and naturally suggests many interesting families of MNNs which can be interpreted as the manifold analog of various popular GNNs. We then propose a method for implementing MFCNs on high-dimensional point clouds that relies on approximating the manifold by a sparse graph. We prove that our method is consistent in the sense that it converges to a continuum limit as the number of data points tends to infinity.

2024-10-17

ArXiv (prépublication)

doi.org

arxiv.org

Assessment of the Climate Trace global powerplant CO2 emissions

Kevin R. Gurney

Bilal Aslam

Pawlok Dass

Lech Gawuc

Toby Dylan Hocking

Jarrett J Barber

Anna Kato

Accurate estimation of planetary greenhouse gas (GHG) emissions at the scale of individual emitting activities is a critical need for both s… (voir plus)cientific and policy applications. Powerplants represent the single largest and most concentrated form of global GHG emissions. Climate Trace, co-founded and promoted by former U.S. Vice President Al Gore, is a new effort using, in part, artificial intelligence (AI) approaches to estimate asset-scale GHG emissions. Climate Trace recently released a database of global powerplant CO2 emissions at the facility-scale that uses both AI and non-AI estimation approaches. However, no independent peer-reviewed assessment has been made of this important global emissions database. Here, we compare the Climate Trace powerplant CO2 emissions to an atmospherically calibrated, multi-constraint estimate of powerplant CO2 emissions in the United States. The 3.7% (65) of compared facilities that used an AI-based approach show a mean relative difference (MRD) of −1.1% (SD: 46.4%) in the year 2019. The 96.3% (1726) of the facilities that used a non-AI-based approach show a MRD of −50.0% (SD: 117.7%). Of the non-AI estimated facilities, 151 (8.7%) facilities agree to within ±20%. The large differences between Climate Trace and Vulcan-power emission estimates for these facilities is primarily caused by Climate Trace’ use of a national-mean power plant capacity factor (CF) which is a poor representation of the reported power plant CFs of individual US facilities and leads to very large errors at those same 1726 facilities.

2024-10-16

Environmental Research Letters (publié)

doi.org

A Simulation System Towards Solving Societal-Scale Manipulation

Maximilian Puelma Touzel

Sneheel Sarangi

Austin Welch

Gayatri K

Dan Zhao

Zachary Yang

Hao Yu

Ethan Kosak-Hine

Tom Gibbs

Andreea Musulan

Camille Thibault

Busra Tugce Gurbuz

Reihaneh Rabbany

Jean-François Godbout

Kellin Pelrine

The rise of AI-driven manipulation poses significant risks to societal trust and democratic processes. Yet, studying these effects in real-w… (voir plus)orld settings at scale is ethically and logistically impractical, highlighting a need for simulation tools that can model these dynamics in controlled settings to enable experimentation with possible defenses. We present a simulation environment designed to address this. We elaborate upon the Concordia framework that simulates offline, `real life' activity by adding online interactions to the simulation through social media with the integration of a Mastodon server. We improve simulation efficiency and information flow, and add a set of measurement tools, particularly longitudinal surveys. We demonstrate the simulator with a tailored example in which we track agents' political positions and show how partisan manipulation of agents can affect election results.

2024-10-16

ArXiv (prépublication)

doi.org

arxiv.org

Training Compute-Optimal Vision Transformers for Brain Encoding

Sana Ahmadi

Fraçois Paugam

Tristan Glatard

Lune P Bellec

The optimal training of a vision transformer for brain encoding depends on three factors: model size, data size, and computational resources… (voir plus). This study investigates these three pillars, focusing on the effects of data scaling, model scaling, and high-performance computing on brain encoding results. Using VideoGPT to extract efficient spatiotemporal features from videos and training a Ridge model to predict brain activity based on these features, we conducted benchmark experiments with varying data sizes (10k, 100k, 1M, 6M) and different model configurations of GPT-2, including hidden layer dimensions, number of layers, and number of attention heads. We also evaluated the effects of training models with 32-bit vs 16-bit floating point representations. Our results demonstrate that increasing the hidden layer dimensions significantly improves brain encoding performance, as evidenced by higher Pearson correlation coefficients across all subjects. In contrast, the number of attention heads does not have a significant effect on the encoding results. Additionally, increasing the number of layers shows some improvement in brain encoding correlations, but the trend is not as consistent as that observed with hidden layer dimensions. The data scaling results show that larger training datasets lead to improved brain encoding performance, with the highest Pearson correlation coefficients observed for the largest dataset size (6M). These findings highlight that the effects of data scaling are more significant compared to model scaling in enhancing brain encoding performance. Furthermore, we explored the impact of floating-point precision by comparing 32-bit and 16-bit representations. Training with 16-bit precision yielded the same brain encoding accuracy as 32-bit, while reducing training time by 1.17 times, demonstrating its efficiency for high-performance computing tasks.

2024-10-16

ArXiv (prépublication)

doi.org

arxiv.org

BlabberSeg: Real-Time Embedded Open-Vocabulary Aerial Segmentation

Haechan Mark Bong

Ricardo de Azambuja

Giovanni Beltrame

Real-time aerial image segmentation plays an important role in the environmental perception of Uncrewed Aerial Vehicles (UAVs). We introduce… (voir plus) BlabberSeg, an optimized Vision-Language Model built on CLIPSeg for on-board, real-time processing of aerial images by UAVs. BlabberSeg improves the efficiency of CLIPSeg by reusing prompt and model features, reducing computational overhead while achieving real-time open-vocabulary aerial segmentation. We validated BlabberSeg in a safe landing scenario using the Dynamic Open-Vocabulary Enhanced SafE-Landing with Intelligence (DOVESEI) framework, which uses visual servoing and open-vocabulary segmentation. BlabberSeg reduces computational costs significantly, with a speed increase of 927.41% (16.78 Hz) on a NVIDIA Jetson Orin AGX (64GB) compared with the original CLIPSeg (1.81Hz), achieving real-time aerial segmentation with negligible loss in accuracy (2.1% as the ratio of the correctly segmented area with respect to CLIPSeg). BlabberSeg's source code is open and available online.

2024-10-15

ArXiv (prépublication)

doi.org

arxiv.org

The Non-Local Model Merging Problem: Permutation Symmetries and Variance Collapse

Ekansh Sharma

Daniel M. Roy

Gintare Karolina Dziugaite

2024-10-15

ArXiv (prépublication)

doi.org

arxiv.org

WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines

Genta Indra Winata

Frederikus Hudi

Patrick Amadeus Irawan

David Anugraha

Rifki Afina Putri

Yutong Wang

Adam Nohejl

Ubaidillah Ariq Prathama

Nedjma OUSIDHOUM

Afifa Amriani

Anar Rzayev

Anirban Das

Ashmari Pramodya

Aulia Adila

Bryan Wilie

Candy Olivia Mawalim

Ching Lam Cheng

Daud Abolade

Emmanuele Chersoni

Enrico Santus … (voir 31 de plus)

Fariz Ikhwantri

Garry Kuwanto

Hanyang Zhao

Haryo Akbarianto Wibowo

Holy Lovenia

Jan Christian Blaise Cruz

Jan Wira Gotama Putra

Junho Myung

Lucky Susanto

Maria Angelica Riera Machin

Marina Zhukova

Michael Anugraha

Muhammad Farid Adilazuarda

Natasha Santosa

Peerat Limkonchotiwat

Raj Dabre

Rio Alexander Audino

Samuel Cahyawijaya

Shi-Xiong Zhang

Stephanie Yulia Salim

Yi Zhou

Yinxuan Gui

David Ifeoluwa Adelani

En-Shiun Annie Lee

Shogo Okada

Ayu Purwarianti

Alham Fikri Aji

Taro Watanabe

Derry Tanti Wijaya

Alice Oh

Chong-Wah Ngo

2024-10-15

ArXiv (prépublication)

doi.org

arxiv.org

Adversarial Bounding Boxes Generation (ABBG) Attack against Visual Object Trackers

Fatemeh Nourilenjan Nokabadi

Jean-François Lalonde

Christian Gagné

Adversarial perturbations aim to deceive neural networks into predicting inaccurate results. For visual object trackers, adversarial attacks… (voir plus) have been developed to generate perturbations by manipulating the outputs. However, transformer trackers predict a specific bounding box instead of an object candidate list, which limits the applicability of many existing attack scenarios. To address this issue, we present a novel white-box approach to attack visual object trackers with transformer backbones using only one bounding box. From the tracker predicted bounding box, we generate a list of adversarial bounding boxes and compute the adversarial loss for those bounding boxes. Experimental results demonstrate that our simple yet effective attack outperforms existing attacks against several robust transformer trackers, including TransT-M, ROMTrack, and MixFormer, on popular benchmark tracking datasets such as GOT-10k, UAV123, and VOT2022STS.

2024-10-14

NeurIPS.cc/2024/Workshop/AdvML-Frontiers (publié)

doi.org

openreview.net

Comparative evaluation of methodologies for estimating the effectiveness of non-pharmaceutical interventions in the context of COVID-19: a simulation study

Iris Ganser

Juliette Paireau

David L Buckeridge

Simon Cauchemez

Rodolphe Thiébaut

M. Prague

2024-10-14

medRxiv (prépublication)

doi.org

How different mental models of AI-based writing assistants impact writers’ interactions with them

Shalaleh Rismani

Su Lin Blodgett

A.R. Olteanu

Q. Vera Liao

AJung Moon

2024-10-14

Proceedings of the Third Workshop on Intelligent and Interactive Writing Assistants (publié)

doi.org

Learning to Forget using Hypernetworks

Jose Miguel Lara Rangel

Usman Anwar

Stefan Schoepf

Jack Foster

David M. Krueger

Machine unlearning is gaining increasing attention as a way to remove adversarial data poisoning attacks from already trained models and to … (voir plus)comply with privacy and AI regulations. The objective is to unlearn the effect of undesired data from a trained model while maintaining performance on the remaining data. This paper introduces HyperForget, a novel machine unlearning framework that leverages hypernetworks– neural networks that generate parameters for other networks– to dynamically sample models that lack knowledge of targeted data while preserving essential capabilities. Leveraging diffusion models, we implement two Diffusion HyperForget Networks and used them to sample unlearned models in Proof-of-Concept experiments. The unlearned models obtained zero accuracy on the forget set, while preserving good accuracy on the retain sets, highlighting the potential of HyperForget for dynamic targeted data removal and a promising direction for developing adaptive machine unlearning algorithms.

2024-10-14

NeurIPS.cc/2024/Workshop/AdvML-Frontiers (publié)

openreview.net