Publications

Criticality of resting-state EEG predicts perturbational complexity and level of consciousness during anesthesia
Charlotte Maschke
Jordan O’Byrne
Michele Angelo Colombo
Melanie Boly
Olivia Gosseries
Steven Laureys
Mario Rosanova
Karim Jerbi
Stefanie Blain-Moraes
Consciousness has been proposed to be supported by electrophysiological patterns poised at criticality, a dynamical regime which exhibits ad… (see more)aptive computational properties, maximally complex patterns and divergent sensitivity to perturbation. Here, we investigated dynamical properties of the resting-state electroencephalogram of healthy subjects undergoing general anesthesia with propofol, xenon or ketamine. We then studied the relation of these dynamic properties with the perturbational complexity index (PCI), which has shown remarkably high sensitivity in detecting consciousness independent of behavior. All participants were unresponsive under anesthesia, while consciousness was retained only during ketamine anesthesia (in the form of vivid dreams)., enabling an experimental dissociation between unresponsiveness and unconsciousness. We estimated (i) avalanche criticality, (ii) chaoticity, and (iii) criticality-related measures, and found that states of unconsciousness were characterized by a distancing from both the edge of activity propagation and the edge of chaos. We were then able to predict individual subjects’ PCI (i.e., PCImax) with a mean absolute error below 7%. Our results establish a firm link between the PCI and criticality and provide further evidence for the role of criticality in the emergence of consciousness. 2 Significance Statement Complexity has long been of interest in consciousness science and had a fundamental impact on many of today’s theories of consciousness. The perturbational complexity index (PCI) uses the complexity of the brain’s response to cortical perturbations to quantify the presence of consciousness. We propose criticality as a unifying framework underlying maximal complexity and sensitivity to perturbation in the conscious brain. We demonstrate that criticality measures derived from resting-state electroencephalography can distinguish conscious from unconscious states, using propofol, xenon and ketamine anesthesia, and from these measures we were able to predict the PCI with a mean error below 7%. Our results support the hypothesis that critical brain dynamics are implicated in the emergence of consciousness and may provide new directions for the assessment of consciousness.
2023-10-31
bioRxiv (preprint)
doi.org
Deep PDE Solvers for Subgrid Modelling and Out-of-Distribution Generalization
Patrick Chatain
Adam M. Oberman
2023-10-31
NeurIPS.cc/2023/Workshop/DLDE (poster)
openreview.net
openreview.net
Generative Learning of Continuous Data by Tensor Networks
Alex Meiburg
Jing Chen
Jacob Miller
Raphaelle Tihon
Guillaume Rabusseau
Alejandro Perdomo-ortiz
2023-10-31
ArXiv (preprint)
doi.org
arxiv.org
Physics-Informed Transformer Networks
Fabricio Dos Santos
F. Dos
Tara Akhound-Sadegh
Siamak Ravanbakhsh
Physics-informed neural networks (PINNs) have been recognized as a viable alternative to conventional numerical solvers for Partial Differen… (see more)tial Equations (PDEs). The main appeal of PINNs is that since they directly enforce the PDE equation, one does not require access to costly ground truth solutions for training the model. However, a key challenge is their limited generalization across varied initial conditions. Addressing this, our study presents a novel Physics-Informed Transformer (PIT) model for learning the solution operator for PDEs. Using the attention mechanism, PIT learns to leverage the relationships between its initial condition and query points, resulting in a significant improvement in generalization. Moreover, in contrast to existing physics-informed networks, our model is invariant to the discretization of the input domain, providing great flexibility in problem specification and training. We validated our proposed method on the 1D Burgers’ and the 2D Heat equations, demonstrating notable improvement over standard PINN models for operator learning with negligible computational overhead.
2023-10-31
NeurIPS.cc/2023/Workshop/DLDE (poster)
openreview.net
openreview.net
Root phosphatase activity is coordinated with the root conservation gradient across a phosphorus gradient in a lowland tropical forest
Xavier Guilbeault-Mayers
Étienne Laliberté
Soil phosphorus (P) is a growth-limiting nutrient in tropical ecosystems, driving diverse P-acquisition strategies among plants. Particularl… (see more)y, mining for inorganic P through phosphomonoesterase (PME) activity is essential, given the substantial proportion of organic P in soils. Yet the relationship between PME activity and other P-acquisition root traits remains unclear. We measured root PME activity and commonly-measured root traits, including root diameter, specific root length (SRL), root tissue density (RTD), and nitrogen concentration ([N]) in 18 co-occurring trees across soils with varying P availability to better understand trees response to P supply. Root [N] and RTD were inversely related, and that axis was related to soil P supply. Indeed, both traits correlated positively and negatively to PME activity, which responded strongly to P supply. Conversely, root diameter was inversely related to SRL, but this axis was not related to P supply. Suggesting that limiting similarity influenced variation along the diameter-SRL axis, explaining high local trait diversity. Meanwhile, environmental filtering tended to impact trait values along the root [N]-RTD axis. Overall, P availability indicator traits like PME activity and root hairs only tended to be associated with these axes, highlighting limitations of these axes in describing convergent adaptations at local sites.
2023-10-31
bioRxiv (preprint)
doi.org
Causal Fair Metric: Bridging Causality, Individual Fairness, and Adversarial Robustness
Ahmad-reza Ehyaei
Golnoosh Farnadi
Samira Samadi
Despite the essential need for comprehensive considerations in responsible AI, factors like robustness, fairness, and causality are often st… (see more)udied in isolation. Adversarial perturbation, used to identify vulnerabilities in models, and individual fairness, aiming for equitable treatment of similar individuals, despite initial differences, both depend on metrics to generate comparable input data instances. Previous attempts to define such joint metrics often lack general assumptions about data or structural causal models and were unable to reflect counterfactual proximity. To address this, our paper introduces a causal fair metric formulated based on causal structures encompassing sensitive attributes and protected causal perturbation. To enhance the practicality of our metric, we propose metric learning as a method for metric estimation and deployment in real-world problems in the absence of structural causal models. We also demonstrate the application of our novel metric in classifiers. Empirical evaluation of real-world and synthetic datasets illustrates the effectiveness of our proposed metric in achieving an accurate classifier with fairness, resilience to adversarial perturbations, and a nuanced understanding of causal relationships.
2023-10-30
ArXiv (preprint)
doi.org
arxiv.org
Characterizing Manipulation from AI Systems
Micah Carroll
Alan Chan
Henry Ashton
David Scott Krueger
Manipulation is a concern in many domains, such as social media, advertising, and chatbots. As AI systems mediate more of our digital intera… (see more)ctions, it is important to understand the degree to which AI systems might manipulate humans without the intent of the system designers. Our work clarifies challenges in defining and measuring this kind of manipulation from AI systems. Firstly, we build upon prior literature on manipulation and characterize the space of possible notions of manipulation, which we find to depend upon the concepts of incentives, intent, covertness, and harm. We review proposals on how to operationalize each concept and we outline challenges in including each concept in a definition of manipulation. Second, we discuss the connections between manipulation and related concepts, such as deception and coercion. We then analyze how our characterization of manipulation applies to recommender systems and language models, and give a brief overview of the regulation of manipulation in other domains. While some progress has been made in defining and measuring manipulation from AI systems, many gaps remain. In the absence of a consensus definition and reliable tools for measurement, we cannot rule out the possibility that AI systems learn to manipulate humans without the intent of the system designers. Manipulation could pose a significant threat to human autonomy and precautionary actions to mitigate it are likely warranted.
2023-10-30
Equity and Access in Algorithms, Mechanisms, and Optimization (published)
doi.org
arxiv.org
FETA: Fairness Enforced Verifying, Training, and Predicting Algorithms for Neural Networks
Kiarash Mohammadi
Aishwarya Sivaraman
Golnoosh Farnadi
2023-10-30
Equity and Access in Algorithms, Mechanisms, and Optimization (published)
doi.org
arxiv.org
TorchProbe: Fuzzing Dynamic Deep Learning Compilers
Qidong Su
Chuqin Geng
Gennady G. Pekhimenko
Xujie Si
Static and dynamic computational graphs represent two distinct approaches to constructing deep learning frameworks. The former prioritizes c… (see more)ompiler-based optimizations, while the latter focuses on programmability and user-friendliness. The recent release of PyTorch 2.0, which supports compiling arbitrary deep learning programs in Python, signifies a new direction in the evolution of deep learning infrastructure to incorporate compiler techniques in a more dynamic manner and support more dynamic language features like dynamic control flows and closures. Given PyTorch's seamless integration with Python, its compiler aims to support arbitrary deep learning code written in Python. However, the inherent dynamism of Python poses challenges to the completeness and robustness of the compiler. While recent research has introduced fuzzing to test deep learning compilers, there is still a lack of comprehensive analysis on how to test dynamic features. To address this issue, we propose several code transformations to generate test cases involving dynamic features. These transformations preserve the program's semantics, ensuring that any discrepancy between the transformed and original programs indicates the presence of a bug. Through our approach, we have successfully identified twenty previously unknown bugs in the PyTorch compiler and its underlying tensor compiler Triton.
2023-10-30
ArXiv (preprint)
doi.org
arxiv.org
Unraveling the Interconnected Axes of Heterogeneity in Machine Learning for Democratic and Inclusive Advancements
Maryam Molamohammadi
Afaf Taïk
Nicolas Le Roux
Golnoosh Farnadi
2023-10-30
Equity and Access in Algorithms, Mechanisms, and Optimization (published)
doi.org
arxiv.org
Proving Linear Mode Connectivity of Neural Networks via Optimal Transport
Damien Ferbach
Baptiste Goujaud
Gauthier Gidel
Aymeric Dieuleveut
The energy landscape of high-dimensional non-convex optimization problems is crucial to understanding the effectiveness of modern deep neura… (see more)l network architectures. Recent works have experimentally shown that two different solutions found after two runs of a stochastic training are often connected by very simple continuous paths (e.g., linear) modulo a permutation of the weights. In this paper, we provide a framework theoretically explaining this empirical observation. Based on convergence rates in Wasserstein distance of empirical measures, we show that, with high probability, two wide enough two-layer neural networks trained with stochastic gradient descent are linearly connected. Additionally, we express upper and lower bounds on the width of each layer of two deep neural networks with independent neuron weights to be linearly connected. Finally, we empirically demonstrate the validity of our approach by showing how the dimension of the support of the weight distribution of neurons, which dictates Wasserstein convergence rates is correlated with linear mode connectivity.
2023-10-29
ArXiv (preprint)
doi.org
arxiv.org
A Case Study of Instruction Tuning with Mixture of Parameter-Efficient Experts
Oleksiy Ostapenko
Lucas Caccia
Zhan Su
Nicolas Le Roux
Laurent Charlin
Alessandro Sordoni
We study the applicability of mixture of parameter-efficient experts (MoPEs) for instruction-tuning large decoder-only language models. Rece… (see more)nt literature indicates that MoPEs might enhance performance in specific multi-task instruction-following datasets. In this paper, we extend such previous results and study applicability of MoPEs in settings previously overlooked: a) with open-domain instruction-following datasets; b) with recent decoder-only models and c) with downstream out-of-distribution test sets. We build on top of LLaMA1-13B/-7B and LLaMA2-13B. We study different variants of learned routing, namely per-example routing ([PE]), and a more expensive per-token ([PT]) routing. Overall, we are unable to substantiate strong performance gains observed in related studies in our setting. We observe occasional enhancements of LLAMA2 fine-tuned on Open Platypus dataset in 0-shot SNI evaluation and TruthfulQA evaluation after fine-tuning on a subset of Flan. We shed some light on the inner workings of MoPEs by comparing different routing strategies. We find that [PE] routing tends to collapse at downstream evaluation time reducing the importance of router's application. We plan to publicly release our code.
2023-10-28
NeurIPS.cc/2023/Workshop/Instruction (published)
openreview.net
openreview.net