Publications

Generative Learning of Continuous Data by Tensor Networks

Alex Meiburg

Jian Hua Chen

Jacob Miller

Raphaelle Tihon

Guillaume Rabusseau

Alejandro Perdomo-ortiz

2023-10-30

ArXiv (preprint)

Physics-Informed Transformer Networks

Fabricio Dos Santos

F. Dos

Tara Akhound-Sadegh

Siamak Ravanbakhsh

Physics-informed neural networks (PINNs) have been recognized as a viable alternative to conventional numerical solvers for Partial Differen… (see more)tial Equations (PDEs). The main appeal of PINNs is that since they directly enforce the PDE equation, one does not require access to costly ground truth solutions for training the model. However, a key challenge is their limited generalization across varied initial conditions. Addressing this, our study presents a novel Physics-Informed Transformer (PIT) model for learning the solution operator for PDEs. Using the attention mechanism, PIT learns to leverage the relationships between its initial condition and query points, resulting in a significant improvement in generalization. Moreover, in contrast to existing physics-informed networks, our model is invariant to the discretization of the input domain, providing great flexibility in problem specification and training. We validated our proposed method on the 1D Burgers’ and the 2D Heat equations, demonstrating notable improvement over standard PINN models for operator learning with negligible computational overhead.

2023-10-30

NeurIPS.cc/2023/Workshop/DLDE (poster)

Root phosphatase activity is coordinated with the root conservation gradient across a phosphorus gradient in a lowland tropical forest

Xavier Guilbeault-Mayers

Étienne Laliberté

Soil phosphorus (P) is a growth-limiting nutrient in tropical ecosystems, driving diverse P-acquisition strategies among plants. Particularl… (see more)y, mining for inorganic P through phosphomonoesterase (PME) activity is essential, given the substantial proportion of organic P in soils. Yet the relationship between PME activity and other P-acquisition root traits remains unclear. We measured root PME activity and commonly-measured root traits, including root diameter, specific root length (SRL), root tissue density (RTD), and nitrogen concentration ([N]) in 18 co-occurring trees across soils with varying P availability to better understand trees response to P supply. Root [N] and RTD were inversely related, and that axis was related to soil P supply. Indeed, both traits correlated positively and negatively to PME activity, which responded strongly to P supply. Conversely, root diameter was inversely related to SRL, but this axis was not related to P supply. Suggesting that limiting similarity influenced variation along the diameter-SRL axis, explaining high local trait diversity. Meanwhile, environmental filtering tended to impact trait values along the root [N]-RTD axis. Overall, P availability indicator traits like PME activity and root hairs only tended to be associated with these axes, highlighting limitations of these axes in describing convergent adaptations at local sites.

2023-10-30

bioRxiv (preprint)

Causal Fair Metric: Bridging Causality, Individual Fairness, and Adversarial Robustness

Ahmad-reza Ehyaei

Golnoosh Farnadi

Samira Samadi

Despite the essential need for comprehensive considerations in responsible AI, factors like robustness, fairness, and causality are often st… (see more)udied in isolation. Adversarial perturbation, used to identify vulnerabilities in models, and individual fairness, aiming for equitable treatment of similar individuals, despite initial differences, both depend on metrics to generate comparable input data instances. Previous attempts to define such joint metrics often lack general assumptions about data or structural causal models and were unable to reflect counterfactual proximity. To address this, our paper introduces a causal fair metric formulated based on causal structures encompassing sensitive attributes and protected causal perturbation. To enhance the practicality of our metric, we propose metric learning as a method for metric estimation and deployment in real-world problems in the absence of structural causal models. We also demonstrate the application of our novel metric in classifiers. Empirical evaluation of real-world and synthetic datasets illustrates the effectiveness of our proposed metric in achieving an accurate classifier with fairness, resilience to adversarial perturbations, and a nuanced understanding of causal relationships.

2023-10-29

ArXiv (preprint)

FETA: Fairness Enforced Verifying, Training, and Predicting Algorithms for Neural Networks

Kiarash Mohammadi

Aishwarya Sivaraman

Golnoosh Farnadi

2023-10-29

Equity and Access in Algorithms, Mechanisms, and Optimization (published)

TorchProbe: Fuzzing Dynamic Deep Learning Compilers

Qidong Su

Chuqin Geng

Gennady G. Pekhimenko

Xujie Si

Static and dynamic computational graphs represent two distinct approaches to constructing deep learning frameworks. The former prioritizes c… (see more)ompiler-based optimizations, while the latter focuses on programmability and user-friendliness. The recent release of PyTorch 2.0, which supports compiling arbitrary deep learning programs in Python, signifies a new direction in the evolution of deep learning infrastructure to incorporate compiler techniques in a more dynamic manner and support more dynamic language features like dynamic control flows and closures. Given PyTorch's seamless integration with Python, its compiler aims to support arbitrary deep learning code written in Python. However, the inherent dynamism of Python poses challenges to the completeness and robustness of the compiler. While recent research has introduced fuzzing to test deep learning compilers, there is still a lack of comprehensive analysis on how to test dynamic features. To address this issue, we propose several code transformations to generate test cases involving dynamic features. These transformations preserve the program's semantics, ensuring that any discrepancy between the transformed and original programs indicates the presence of a bug. Through our approach, we have successfully identified twenty previously unknown bugs in the PyTorch compiler and its underlying tensor compiler Triton.

2023-10-29

ArXiv (preprint)

Typology of ICU-Healthcare Providers Who Delayed or Declined COVID-19 Vaccination

Elie Azoulay

Frédéric Pochard

Guillaume Dumas

Nancy Kentish-Barnes

OBJECTIVES: To assess COVID-19 vaccination rates in ICU-healthcare providers (HCPs) in France and to identify the typology of those who dela… (see more)yed or declined vaccination. DESIGN: Cross-sectional study. SETTING: Twenty-one ICUs in France. SUBJECTS: Members of the nursing and medical staff and other allied professionals. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Six hundred ninety-six of the 950 respondents (73.3%) had undergone a full vaccination schedule. Other HCPs either declined vaccination (n = 112) or delayed vaccination until it became mandatory (n = 142). Factors independently associated with full vaccination were age older than 50 years (odds ratio, 0.25 [95% CI, 0.12–0.51]), more than 5 years of ICU experience (0.66 [0.47–0.93]), increasing working time during the surge (0.94 [0.88–1.00]), and spending time with the family (0.92 [0.85–0.99]). Conversely, being a nurse (1.94 [1.25–2.99]) or a nurse assistant (2.77 [1.62–4.73]), and feeling not supported by hospital and ICU directors (1.49 [1.01–2.20]) was independently associated with not being vaccinated. CONCLUSIONS: These results are important to take into account to better implement vaccination strategies in HCPs for existing or future pandemics.

2023-10-29

Critical Care Medicine (published)

Characterizing Manipulation from Al Systems

MICAH CARROLL

Alan Chan

Henry Ashton

David Krueger

Manipulation is a common concern in many domains, such as social media, advertising, and chatbots. As AI systems mediate more of our interac… (see more)tions with the world, it is important to understand the degree to which AI systems might manipulate humans without the intent of the system designers. Our work clarifies challenges in defining and measuring manipulation in the context of AI systems. Firstly, we build upon prior literature on manipulation from other fields and characterize the space of possible notions of manipulation, which we find to depend upon the concepts of incentives, intent, harm, and covertness. We review proposals on how to operationalize each factor. Second, we propose a definition of manipulation based on our characterization: a system is manipulative if it acts as if it were pursuing an incentive to change a human (or another agent) intentionally and covertly. Third, we discuss the connections between manipulation and related concepts, such as deception and coercion. Finally, we contextualize our operationalization of manipulation in some applications. Our overall assessment is that while some progress has been made in defining and measuring manipulation from AI systems, many gaps remain. In the absence of a consensus definition and reliable tools for measurement, we cannot rule out the possibility that AI systems learn to manipulate humans without the intent of the system designers. We argue that such manipulation poses a significant threat to human autonomy, suggesting that precautionary actions to mitigate it are warranted.

2023-10-28

Equity and Access in Algorithms, Mechanisms, and Optimization (published)

Unraveling the Interconnected Axes of Heterogeneity in Machine Learning for Democratic and Inclusive Advancements

The growing utilization of machine learning (ML) in decision-making processes raises questions about its benefits to society. In this study,… (see more) we identify and analyze three axes of heterogeneity that significantly influence the trajectory of ML products. These axes are i) values, culture and regulations, ii) data composition, and iii) resource and infrastructure capacity. We demonstrate how these axes are interdependent and mutually influence one another, emphasizing the need to consider and address them jointly. Unfortunately, the current research landscape falls short in this regard, often failing to adopt a holistic approach. We examine the prevalent practices and methodologies that skew these axes in favor of a selected few, resulting in power concentration, homogenized control, and increased dependency. We discuss how this fragmented study of the three axes poses a significant challenge, leading to an impractical solution space that lacks reflection of real-world scenarios. Addressing these issues is crucial to ensure a more comprehensive understanding of the interconnected nature of society and to foster the democratic and inclusive development of ML systems that are more aligned with real-world complexities and its diverse requirements.

2023-10-28

Equity and Access in Algorithms, Mechanisms, and Optimization (published)

A Case Study of Instruction Tuning with Mixture of Parameter-Efficient Experts

O. Ostapenko

Lucas Caccia

2023-10-27

NeurIPS.cc/2023/Workshop/Instruction (published)

Detecting Backdoors with Meta-Models

Lauro Langosco

Neel Alex

William Baker

David John Quarel

Herbie Bradley

David M. Krueger

It is widely known that it is possible to implant backdoors into neural networks, by which an attacker can choose an input to produce a part… (see more)icular undesirable output (e.g.\ misclassify an image). We propose to use \emph{meta-models}, neural networks that take another network's parameters as input, to detect backdoors directly from model weights. To this end we present a meta-model architecture and train it on a dataset of approx.\ 4000 clean and backdoored CNNs trained on CIFAR-10. Our approach is simple and scalable, and is able to detect the presence of a backdoor with

2023-10-27

NeurIPS.cc/2023/Workshop/BUGS (poster)