Publications

ChatGPT vs LLaMA: Impact, Reliability, and Challenges in Stack Overflow Discussions
Leuson Da Silva
Jordan Samhi
Since its release in November 2022, ChatGPT has shaken up Stack Overflow, the premier platform for developers' queries on programming and so… (voir plus)ftware development. Demonstrating an ability to generate instant, human-like responses to technical questions, ChatGPT has ignited debates within the developer community about the evolving role of human-driven platforms in the age of generative AI. Two months after ChatGPT's release, Meta released its answer with its own Large Language Model (LLM) called LLaMA: the race was on. We conducted an empirical study analyzing questions from Stack Overflow and using these LLMs to address them. This way, we aim to (ii) measure user engagement evolution with Stack Overflow over time; (ii) quantify the reliability of LLMs' answers and their potential to replace Stack Overflow in the long term; (iii) identify and understand why LLMs fails; and (iv) compare LLMs together. Our empirical results are unequivocal: ChatGPT and LLaMA challenge human expertise, yet do not outperform it for some domains, while a significant decline in user posting activity has been observed. Furthermore, we also discuss the impact of our findings regarding the usage and development of new LLMs.
Computing Power and the Governance of Artificial Intelligence
Girish Sastry
Lennart Heim
Haydn Belfield
Markus Anderljung
Miles Brundage
Julian Hazell
Cullen C. O'keefe
Gillian K. Hadfield
Richard Ngo
Konstantin Pilz
George Gor
Emma Bluemke
Sarah Shoker
Janet Egan
Robert F. Trager
Shahar Avin
Adrian Weller
Diane Coyle
Computing power, or"compute,"is crucial for the development and deployment of artificial intelligence (AI) capabilities. As a result, govern… (voir plus)ments and companies have started to leverage compute as a means to govern AI. For example, governments are investing in domestic compute capacity, controlling the flow of compute to competing countries, and subsidizing compute access to certain sectors. However, these efforts only scratch the surface of how compute can be used to govern AI development and deployment. Relative to other key inputs to AI (data and algorithms), AI-relevant compute is a particularly effective point of intervention: it is detectable, excludable, and quantifiable, and is produced via an extremely concentrated supply chain. These characteristics, alongside the singular importance of compute for cutting-edge AI models, suggest that governing compute can contribute to achieving common policy objectives, such as ensuring the safety and beneficial use of AI. More precisely, policymakers could use compute to facilitate regulatory visibility of AI, allocate resources to promote beneficial outcomes, and enforce restrictions against irresponsible or malicious AI development and usage. However, while compute-based policies and technologies have the potential to assist in these areas, there is significant variation in their readiness for implementation. Some ideas are currently being piloted, while others are hindered by the need for fundamental research. Furthermore, naive or poorly scoped approaches to compute governance carry significant risks in areas like privacy, economic impacts, and centralization of power. We end by suggesting guardrails to minimize these risks from compute governance.
Mixtures of Experts Unlock Parameter Scaling for Deep RL
Johan Samir Obando Ceron
Ghada Sokar
Timon Willi
Clare Lyle
Jesse Farebrother
Jakob Nicolaus Foerster
Model approximation in MDPs with unbounded per-step cost
Berk Bozkurt
Ashutosh Nayyar
Yi Ouyang
We consider the problem of designing a control policy for an infinite-horizon discounted cost Markov decision process …
A neuronal least-action principle for real-time learning in cortical circuits
Walter Senn
Dominik Dold
Akos F. Kungl
Benjamin Ellenberger
Jakob Jordan
João Sacramento
Mihai A. Petrovici
One of the most fundamental laws of physics is the principle of least action. Motivated by its predictive power, we introduce a neuronal lea… (voir plus)st-action principle for cortical processing of sensory streams to produce appropriate behavioural outputs in real time. The principle postulates that the voltage dynamics of cortical pyramidal neurons prospectively minimize the local somato-dendritic mismatch error within individual neurons. For motor output neurons, it implies minimizing an instantaneous behavioural error. For deep network neurons, it implies a prospective firing to overcome integration delays and correct for possible output errors right in time. The neuron-specific errors are extracted in the apical dendrites of pyramidal neurons through a cortical microcircuit that tries to explain away the feedback from the periphery, and correct the trajectory on the fly. Any motor output is in a moving equilibrium with the sensory inputs and the motor feedback during the whole sensory-motor trajectory. Ongoing synaptic plasticity reduces the somato-dendritic mismatch error within each cortical neuron and performs gradient descent on the output cost at any moment in time. The neuronal least-action principle offers an axiomatic framework to derive local neuronal and synaptic dynamics for global real-time computation and learning in the brain and in physical substrates in general.
Regional Adaptive Metropolis Light Transport
Hisanari Otsu
Killian Herveau
Johannes Hanika
Carsten Dachsbacher
The design of the proposal distributions, and most notably the kernel parameters, are crucial for the performance of Markov chain Monte Carl… (voir plus)o (MCMC) rendering. A poor selection of parameters can increase the correlation of the Markov chain and result in bad rendering performance. We approach this problem by a novel path perturbation strategy for online-learning of state-dependent kernel parameters. We base our approach on the theoretical framework of regional adaptive MCMC which enables the adaptation of parameters depending on the region of the state space which contains the current sample, and on information collected from previous samples. For this, we define a partitioning of the path space on a low-dimensional canonical space to capture the characteristics of paths, with a focus on path segments closer to the sensor. Fast convergence is achieved by adaptive refinement of the partitions. Exemplarily, we present two novel regional adaptive path perturbation techniques akin to lens and multi-chain perturbations. Our approach can easily be used on top of existing path space MLT methods to improve rendering efficiency, while being agnostic to the initial choice of kernel parameters.
Antagonistic AI
Alice Cai
Elena L. Glassman
An enhanced wideband tracking method for characteristic modes
Chao Huang
Chenjiang Guo
Xia Ma
Yi Yuan
An enhanced wideband tracking method for characteristic modes (CMs) is investigated in this paper. The method consists of three stages, and … (voir plus)its core tracking stage (CTS) is based on a classical eigenvector correlation-based algorithm. To decrease the tracking time and eliminate the crossing avoidance (CRA), we append a commonly used eigenvalue filter (EF) as the preprocessing stage and a novel postprocessing stage to the CTS. The proposed postprocessing stage can identify all CRA mode pairs by analyzing their trajectory and correlation characteristics. Subsequently, it can predict corresponding CRA frequencies and correct problematic qualities rapidly. Considering potential variations in eigenvector numbers at consecutive frequency samples caused by the EF, a new execution condition for the adaptive frequency adjustment in the CTS is introduced. Finally, CMs of a conductor plate and a fractal structure are investigated to demonstrate the performance of the proposed method, and the obtained results are discussed.
Leveraging Function Space Aggregation for Federated Learning at Scale
Nikita Dhawan
Nicole Elyse Mitchell
Zachary Charles
Zachary Garrett
The federated learning paradigm has motivated the development of methods for aggregating multiple client updates into a global server model,… (voir plus) without sharing client data. Many federated learning algorithms, including the canonical Federated Averaging (FedAvg), take a direct (possibly weighted) average of the client parameter updates, motivated by results in distributed optimization. In this work, we adopt a function space perspective and propose a new algorithm, FedFish, that aggregates local approximations to the functions learned by clients, using an estimate based on their Fisher information. We evaluate FedFish on realistic, large-scale cross-device benchmarks. While the performance of FedAvg can suffer as client models drift further apart, we demonstrate that FedFish is more robust to longer local training. Our evaluation across several settings in image and language benchmarks shows that FedFish outperforms FedAvg as local training epochs increase. Further, FedFish results in global networks that are more amenable to efficient personalization via local fine-tuning on the same or shifted data distributions. For instance, federated pretraining on the C4 dataset, followed by few-shot personalization on Stack Overflow, results in a 7% improvement in next-token prediction by FedFish over FedAvg.
Metrics reloaded: Pitfalls and recommendations for image analysis validation
Lena Maier-Hein
Annika Reinke
Evangelia Christodoulou
Ben Glocker
PATRICK GODAU
Fabian Isensee
Jens Kleesiek
Michal Kozubek
Mauricio Reyes
MICHAEL A. RIEGLER
Manuel Wiesenfarth
Michael Baumgartner
Matthias Eisenmann
DOREEN HECKMANN-NÖTZEL
A. EMRE KAVUR
TIM RÄDSCH
Minu Dietlinde Tizabi
LAURA ACION
Michela Antonelli
Spyridon Bakas
Peter Bankhead
Allison Benis
M. Jorge Cardoso
Veronika Cheplygina
BETH A. CIMINI
Gary S. Collins
Keyvan Farahani
Bram van Ginneken
Daniel A. Hashimoto
Michael M. Hoffman
Merel Huisman
Pierre Jannin
CHARLES E. KAHN
ALEXANDROS KARARGYRIS
Alan Karthikesalingam
H. Kenngott
Annette Kopp-Schneider
Anna Kreshuk
Tahsin Kurc
Bennett Landman
GEERT LITJENS
Amin Madani
Klaus Maier-Hein
Anne L. Martel
Peter Mattson
ERIK MEIJERING
Bjoern Menze
David Moher
KAREL G.M. MOONS
Henning Müller
Felix Nickel
Brennan Nichyporuk
Jens Petersen
NASIR RAJPOOT
Nicola Rieke
Julio Saez-Rodriguez
Clarisa S'anchez Guti'errez
SHRAVYA SHETTY
M. Smeden
Carole H. Sudre
Ronald M. Summers
Abdel Aziz Taha
Sotirios A. Tsaftaris
B. Calster
Gael Varoquaux
PAUL F. JÄGER
Nearest Neighbour Score Estimators for Diffusion Generative Models
Matthew Niedoba
Dylan Green
Saeid Naderiparizi
Vasileios Lioutas
Jonathan Wilder Lavington
Xiaoxuan Liang
Yunpeng Liu
Ke Zhang
Setareh Dabiri
Adam Ścibior
Berend Zwartsenberg
Understanding metric-related pitfalls in image analysis validation
Annika Reinke
Minu Dietlinde Tizabi
Michael Baumgartner
Matthias Eisenmann
DOREEN HECKMANN-NÖTZEL
A. EMRE KAVUR
TIM RÄDSCH
Carole H. Sudre
LAURA ACION
Michela Antonelli
Spyridon Bakas
Allison Benis
Arriel Benis
Matthew Blaschko
FLORIAN BUETTNER
Florian Buttner
M. Jorge Cardoso
Veronika Cheplygina
JIANXU CHEN … (voir 62 de plus)
Evangelia Christodoulou
BETH A. CIMINI
Keyvan Farahani
LUCIANA FERRER
Gary S. Collins
Adrian Galdran
Bram van Ginneken
Ben Glocker
PATRICK GODAU
Daniel A. Hashimoto
Michael M. Hoffman
Robert Cary Haase
Merel Huisman
Fabian Isensee
Pierre Jannin
CHARLES E. KAHN
DAGMAR KAINMUELLER
BERNHARD KAINZ
ALEXANDROS KARARGYRIS
Jens Kleesiek
Florian Kofler
THIJS KOOI
Annette Kopp-Schneider
Alan Karthikesalingam
H. Kenngott
Michal Kozubek
Anna Kreshuk
Tahsin Kurc
BENNETT A. LANDMAN
GEERT LITJENS
Amin Madani
Klaus Maier-Hein
Anne L. Martel
ERIK MEIJERING
Bjoern Menze
KAREL G.M. MOONS
Henning Müller
Brennan Nichyporuk
Peter Mattson
Felix Nickel
Jens Petersen
SUSANNE M. RAFELSKI
NASIR RAJPOOT
Mauricio Reyes
MICHAEL A. RIEGLER
Nicola Rieke
Julio Saez-Rodriguez
Clara I. Sánchez
SHRAVYA SHETTY
Ronald M. Summers
Abdel Aziz Taha
ALEKSEI TIULPIN
Sotirios A. Tsaftaris
Ben Van Calster
Gael Varoquaux
ZIV R. YANIV
M. Smeden
PAUL F. JÄGER
Lena Maier-Hein
B. Calster
Manuel Wiesenfarth
Ziv Rafael Yaniv