A density estimation perspective on learning from pairwise human preferences
Vincent Dumoulin
Daniel D. Johnson
Yann Dauphin
Learning from human feedback (LHF) -- and in particular learning from pairwise preferences -- has recently become a crucial ingredient in tr… (voir plus)aining large language models (LLMs), and has been the subject of much research. Most recent works frame it as a reinforcement learning problem, where a reward function is learned from pairwise preference data and the LLM is treated as a policy which is adapted to maximize the rewards, often under additional regularization constraints. We propose an alternative interpretation which centers on the generative process for pairwise preferences and treats LHF as a density estimation problem. We provide theoretical and empirical results showing that for a family of generative processes defined via preference behavior distribution equations, training a reward function on pairwise preferences effectively models an annotator's implicit preference distribution. Finally, we discuss and present findings on"annotator misspecification"-- failure cases where wrong modeling assumptions are made about annotator behavior, resulting in poorly-adapted models -- suggesting that approaches that learn from pairwise human preferences could have trouble learning from a population of annotators with diverse viewpoints.
RAMEN Unveils Clinical Variable Networks for COVID-19 Severity and Long COVID Using Absorbing Random Walks and Genetic Algorithms
Yiwei Xiong
Jingtao Wang
Xiaoxiao Shang
Tingting Chen
Douglas D. Fraser
Gregory Fonseca
Simon Rousseau
The COVID-19 pandemic has significantly altered global socioeconomic structures and individual lives. Understanding the disease mechanisms a… (voir plus)nd facilitating diagnosis requires comprehending the complex interplay among clinical factors like demographics, symptoms, comorbidities, treatments, lab results, complications, and other metrics, and their relation to outcomes such as disease severity and long term outcomes (e.g., post-COVID-19 condition/long COVID). Conventional correlational methods struggle with indirect and directional connections among these factors, while standard graphical methods like Bayesian networks are computationally demanding for extensive clinical variables. In response, we introduced RAMEN, a methodology that integrates Genetic Algorithms with random walks for efficient Bayesian network inference, designed to map the intricate relationships among clinical variables. Applying RAMEN to the Biobanque québécoise de la COVID-19 (BQC19) dataset, we identified critical markers for long COVID and varying disease severity. The Bayesian Network, corroborated by existing literature and supported through multi-omics analyses, highlights significant clinical variables linked to COVID-19 outcomes. RAMEN’s ability to accurately map these connections contributes substantially to developing early and effective diagnostics for severe COVID-19 and long COVID.
On the Societal Impact of Open Foundation Models
Sayash Kapoor
Rishi Bommasani
Kevin Klyman
Shayne Longpre
Ashwin Ramaswami
Peter Cihon
Aspen Hopkins
Kevin Bankston
Stella Biderman
Miranda Bogen
Rumman Chowdhury
Alex Engler
Peter Henderson
Yacine Jernite
Seth Lazar
Stefano Maffulli
Alondra Nelson
Aviya Skowron
Dawn Song … (voir 5 de plus)
Victor Storchan
Daniel Zhang
Daniel E. Ho
Percy Liang
Arvind Narayanan
Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, w… (voir plus)e focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.
On the Societal Impact of Open Foundation Models
Sayash Kapoor
Rishi Bommasani
Kevin Klyman
Shayne Longpre
Ashwin Ramaswami
Peter Cihon
Aspen Hopkins
Kevin Bankston
Stella Biderman
Miranda Bogen
Rumman Chowdhury
Alex Engler
Peter Henderson
Yacine Jernite
Seth Lazar
Stefano Maffulli
Alondra Nelson
Aviya Skowron
Dawn Song … (voir 5 de plus)
Victor Storchan
Daniel Zhang
Daniel E. Ho
Percy Liang
Arvind Narayanan
Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, w… (voir plus)e focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.
On the Societal Impact of Open Foundation Models
Sayash Kapoor
Rishi Bommasani
Kevin Klyman
Shayne Longpre
Ashwin Ramaswami
Peter Cihon
Aspen Hopkins
Kevin Bankston
Stella Biderman
Miranda Bogen
Rumman Chowdhury
Alex Engler
Peter Henderson
Yacine Jernite
Seth Lazar
Stefano Maffulli
Alondra Nelson
Aviya Skowron
Dawn Song … (voir 5 de plus)
Victor Storchan
Daniel Zhang
Daniel E. Ho
Percy Liang
Arvind Narayanan
Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, w… (voir plus)e focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.
Effective Latent Differential Equation Models via Attention and Multiple Shooting
Germán Abrevaya
Mahta Ramezanian-Panahi
Jean-Christophe Gagnon-Audet
Pablo Polosecki
Silvina Ponce Dawson
Guillermo Cecchi
Correction to: Multi-agent reinforcement learning for fast-timescale demand response of residential loads
Vincent Mai
Philippe Maisonneuve
Tianyu Zhang
Hadi Nekoei
Distinctive whole-brain cell types predict tissue damage patterns in thirteen neurodegenerative conditions
Veronika Pak
Quadri Adewale
Mahsa Dadar
Yashar Zeighami
Yasser Iturria-Medina
For over a century, brain research narrative has mainly centered on neuron cells. Accordingly, most neurodegenerative studies focus on neuro… (voir plus)nal dysfunction and their selective vulnerability, while we lack comprehensive analyses of other major cell types’ contribution. By unifying spatial gene expression, structural MRI, and cell deconvolution, here we describe how the human brain distribution of canonical cell types extensively predicts tissue damage in thirteen neurodegenerative conditions, including early-and late-onset Alzheimer’s disease, Parkinson’s disease, dementia with Lewy bodies, amyotrophic lateral sclerosis, mutations in presenilin-1, and three clinical variants of frontotemporal lobar degeneration (behavioural variant, semantic and non-fluent primary progressive aphasia) along with associated 3-repeat and 4-repeat tauopathies and TDP43 proteinopathies types A and C. We reconstructed comprehensive whole-brain reference maps of cellular abundance for six major cell types and identified characteristic axes of spatial overlapping with atrophy. Our results support the strong mediating role of non-neuronal cells, primarily microglia and astrocytes, in spatial vulnerability to tissue loss in neurodegeneration, with distinct and shared across-disorders pathomechanisms. These observations provide critical insights into the multicellular pathophysiology underlying spatiotemporal advance in neurodegeneration. Notably, they also emphasize the need to exceed the current neuro-centric view of brain diseases, supporting the imperative for cell-specific therapeutic targets in neurodegeneration.
Intra-Host Evolution Analyses in an Immunosuppressed Patient Supports SARS-CoV-2 Viral Reservoir Hypothesis
Dominique Fournelle
Fatima Mostefai
Elsa Brunet-Ratnasingham
Raphael Poujol
Jean-Christophe Grenier
José Héctor Gálvez
Amélie Pagliuzza
Inès Levade
Sandrine Moreira
Mehdi Benlarbi
Guillaume Beaudoin-Bussières
Gabrielle Gendron-Lepage
Catherine Bourassa
Alexandra Tauzin
Simon Grandjean Lapierre
Nicolas Chomont
Andrés Finzi
Daniel E. Kaufmann
Morgan Craig
Intra-Host Evolution Analyses in an Immunosuppressed Patient Supports SARS-CoV-2 Viral Reservoir Hypothesis
Dominique Fournelle
Fatima Mostefai
Elsa Brunet-Ratnasingham
Raphael Poujol
Jean-Christophe Grenier
José Héctor Gálvez
Amélie Pagliuzza
Inès Levade
Sandrine Moreira
Mehdi Benlarbi
Guillaume Beaudoin-Bussières
Gabrielle Gendron-Lepage
Catherine Bourassa
Alexandra Tauzin
Simon Grandjean Lapierre
Nicolas Chomont
Andrés Finzi
Daniel E. Kaufmann
Morgan Craig
Throughout the SARS-CoV-2 pandemic, several variants of concern (VOCs) have been identified, many of which share recurrent mutations in the … (voir plus)spike glycoprotein’s receptor-binding domain (RBD). This region coincides with known epitopes and can therefore have an impact on immune escape. Protracted infections in immunosuppressed patients have been hypothesized to lead to an enrichment of such mutations and therefore drive evolution towards VOCs. Here, we present the case of an immunosuppressed patient that developed distinct populations with immune escape mutations throughout the course of their infection. Notably, by investigating the co-occurrence of substitutions on individual sequencing reads in the RBD, we found quasispecies harboring mutations that confer resistance to known monoclonal antibodies (mAbs) such as S:E484K and S:E484A. These mutations were acquired without the patient being treated with mAbs nor convalescent sera and without them developing a detectable immune response to the virus. We also provide additional evidence for a viral reservoir based on intra-host phylogenetics, which led to a viral substrain that evolved elsewhere in the patient’s body, colonizing their upper respiratory tract (URT). The presence of SARS-CoV-2 viral reservoirs can shed light on protracted infections interspersed with periods where the virus is undetectable, and potential explanations for long-COVID cases.
Intra-Host Evolution Analyses in an Immunosuppressed Patient Supports SARS-CoV-2 Viral Reservoir Hypothesis
Dominique Fournelle
Fatima Mostefai
Elsa Brunet-Ratnasingham
Raphael Poujol
Jean-Christophe Grenier
José Héctor Gálvez
Amélie Pagliuzza
Inès Levade
Sandrine Moreira
Mehdi Benlarbi
Guillaume Beaudoin-Bussières
Gabrielle Gendron-Lepage
Catherine Bourassa
Alexandra Tauzin
Simon Grandjean Lapierre
Nicolas Chomont
Andrés Finzi
Daniel E. Kaufmann
Morgan Craig
Throughout the SARS-CoV-2 pandemic, several variants of concern (VOCs) have been identified, many of which share recurrent mutations in the … (voir plus)spike glycoprotein’s receptor-binding domain (RBD). This region coincides with known epitopes and can therefore have an impact on immune escape. Protracted infections in immunosuppressed patients have been hypothesized to lead to an enrichment of such mutations and therefore drive evolution towards VOCs. Here, we present the case of an immunosuppressed patient that developed distinct populations with immune escape mutations throughout the course of their infection. Notably, by investigating the co-occurrence of substitutions on individual sequencing reads in the RBD, we found quasispecies harboring mutations that confer resistance to known monoclonal antibodies (mAbs) such as S:E484K and S:E484A. These mutations were acquired without the patient being treated with mAbs nor convalescent sera and without them developing a detectable immune response to the virus. We also provide additional evidence for a viral reservoir based on intra-host phylogenetics, which led to a viral substrain that evolved elsewhere in the patient’s body, colonizing their upper respiratory tract (URT). The presence of SARS-CoV-2 viral reservoirs can shed light on protracted infections interspersed with periods where the virus is undetectable, and potential explanations for long-COVID cases.
Molar Pregnancy in a Quadruplet Conception Following IVF: A Case Report
Madhuri A Mehendale
Meenal Shailesh Sarmalkar
Prerna Kailashchand Gupta
Agraj S Doshi