Martin Weiss

Can LLMs Faithfully Enact Superforecaster Personas?

Andrew Robert Williams

Evan Jiang

Nasim Rahaman

Superforecasters are people that make forecasts that are statistically better than the average person; they offer an interesting testbed for… (see more) persona modelling. In this preliminary work, we investigate whether large language models (LLMs) can faithfully model superforecaster-like personas making similar forecasts and generating similar rationales behind the forecasts. When we compare LLM-generated responses to the ground truth responses of superforecasters, we find that few-shot prompting generates numerically closer forecasts than instruction-prompting baselines. We also measure the similarity of generated and real rationales, finding that the rationales generated based on examples are judged more similar to those of superforecasters than instruction-prompting baselines. However, an analysis of the reasoning patterns in the rationales shows significant differences between human and LLM forecasters, pointing to a gap between stylistic imitation and deep reasoning similarity. These preliminary results raise several interesting questions about how getting superforecaster-like behaviour from LLMs actually works, and open new avenues to explore for improving the forecasting behaviour of LLMs.

2026-08-02

Social_Sim @ Conference on Language Modeling (poster)

openreview.net

Lacuna: A Research Map for Machine Learning

Martin Weiss

Miles Q. Li

Alejandro H. Artiles

Yacine Mkhinini

Chris Pal

Hugo Larochelle

Nasim Rahaman

Lacuna is a research map for machine learning that uses LLMs to turn papers and scholarly metadata into markdown summaries, concept elements… (see more), research directions, and research proposals. Each item keeps links to the primary source records and papers that support it. We release the map with web, markdown, and MCP interfaces. Across LitSearch, Multi-XScience-CS/ML, and ScholarQA-CS-ML, Lacuna outperforms OpenScholar with the strongest gains on LitSearch retrieval (Recall@10 0.538 vs. 0.424 for OpenScholar v3). We also evaluate Lacuna Deep Research, a multi-stage report agent over the map, on 25 ReportBench-ML survey tasks: Lacuna Deep Research reaches 0.052 citation F1, 0.339 citation precision, 99 expert-reference hits, and 7.82/10 RACE report quality, while GPT-Researcher reaches 0.039 F1, 0.290 precision, 72 hits, and 5.24/10 RACE.

2026-06-23

arXiv (preprint)

doi.org

arxiv.org

Lacuna: A Research Map for Machine Learning Problem Formulation

Martin Weiss

Alejandro Hernandez

Yacine Mkhinini

Miles Q. Li

Christopher Pal

Hugo Larochelle

Nasim Rahaman

Research problem formulation is the cognitive task of turning a vague scientific idea into a testable hypothesis. \textbf{Lacuna} is a resea… (see more)rch map that supports this process for machine learning, using LLMs to turn papers and scholarly metadata into markdown summaries, concept elements, research directions, author and venue pages, and research proposals. Each item keeps links to the primary source records and papers that support it. We release the map with web, markdown, and MCP interfaces, plus scripts for reproducing the agent run. On a theorem-proving seed idea, the agent reaches a research proposal in 85.5 seconds and 7 tool calls, vs.\ 289.2 seconds and 27 tool calls for a sequential-PDF baseline. On ScholarQA-CS-ML, an ML/AI subset of the OpenScholar benchmark, Lacuna-GPT-4o scores 0.694 under the ScholarQABench rubric judge, compared with 0.672 for the OpenScholar-GPT-4o baseline on the same questions.

2026-05-31

AI4Research @ International Conference on Machine Learning (poster)

openreview.net

SelvaBox: A high‑resolution dataset for tropical tree crown detection

Hugo Baudchon

Arthur Ouaknine

Martin Weiss

Mélisande Teng

Thomas Walla

Antoine Caron-Guay

Christopher Pal

Étienne Laliberté

Detecting individual tree crowns in tropical forests is essential to study these complex and crucial ecosystems impacted by human interventi… (see more)ons and climate change. However, tropical crowns vary widely in size, structure, and pattern and are largely overlapping and intertwined, requiring advanced remote sensing methods applied to high-resolution imagery. Despite growing interest in tropical tree crown detection, annotated datasets remain scarce, hindering robust model development. We introduce SelvaBox, the largest open‑access dataset for tropical tree crown detection in high-resolution drone imagery. It spans three countries and contains more than

2025-12-31

International Conference on Learning Representations (Accept (Poster))

openreview.net

DeLLMphi: A Multi-Turn Method for Multi-Agent Forecasting

Andrew Robert Williams

Martin Weiss

Victoria Feere

Nasim Rahaman

Hugo Larochelle

The Delphi method is a structured forecasting process that engages experts in iterative prediction and reflection. Each round, experts submi… (see more)t forecasts to a mediator, receive an aggregated and synthesized response highlighting key arguments, and update their forecasts based on collective insight. However, Delphi panels are labour intensive, slow and hard to reproduce, requiring diverse knowledgeable participants to engage periodically across weeks or months. To address these constraints, we propose **DeLLMphi**, a forecasting method that replaces human experts and mediators with LLMs. We show (i) that providing example superforecaster reasoning traces and predictions helps to elicit more accurate forecasts from LLM experts, (ii) that the mediator plays the crucial role of surfacing different lines of reasoning and points of disagreement, and (iii) that multiple rounds and experts lead to better forecasts, showing that multi-turn interaction is key to DeLLMphi.

2025-11-03

NeurIPS.cc/2025/Workshop/MTI-LLM (poster)

openreview.net

Redesigning Information Markets in the Era of Language Models

Martin Weiss

Nasim Rahaman

Manuel Wuthrich

Yoshua Bengio

Li Erran Li

Bernhard Schölkopf

Christopher Pal

2024-07-09

colmweb.org/COLM/2024/Conference (accepted)

openreview.net

Language Models Can Reduce Asymmetry in Information Markets

Nasim Rahaman

Martin Weiss

Manuel Wuthrich

Yoshua Bengio

Erran L. Li

Christopher Pal

Bernhard Schölkopf

2024-03-20

ArXiv (preprint)

doi.org

arxiv.org

Proactive Contact Tracing

Prateek Gupta

Tegan Maharaj

Martin Weiss

Nasim Rahaman

Hannah Alsdurf

Nanor Minoyan

Soren Harnois-Leblanc

Joanna Merckx

Andrew Williams

Victor Schmidt

Pierre-Luc St-Charles

Akshay Patel

Yang Zhang

David L. Buckeridge

Christopher Pal

Bernhard Schölkopf

Yoshua Bengio

The COVID-19 pandemic has spurred an unprecedented demand for interventions that can reduce disease spread without excessively restricting d… (see more)aily activity, given negative impacts on mental health and economic outcomes. Digital contact tracing (DCT) apps have emerged as a component of the epidemic management toolkit. Existing DCT apps typically recommend quarantine to all digitally-recorded contacts of test-confirmed cases. Over-reliance on testing may, however, impede the effectiveness of such apps, since by the time cases are confirmed through testing, onward transmissions are likely to have occurred. Furthermore, most cases are infectious over a short period; only a subset of their contacts are likely to become infected. These apps do not fully utilize data sources to base their predictions of transmission risk during an encounter, leading to recommendations of quarantine to many uninfected people and associated slowdowns in economic activity. This phenomenon, commonly termed as “pingdemic,” may additionally contribute to reduced compliance to public health measures. In this work, we propose a novel DCT framework, Proactive Contact Tracing (PCT), which uses multiple sources of information (e.g. self-reported symptoms, received messages from contacts) to estimate app users’ infectiousness histories and provide behavioral recommendations. PCT methods are by design proactive, predicting spread before it occurs. We present an interpretable instance of this framework, the Rule-based PCT algorithm, designed via a multi-disciplinary collaboration among epidemiologists, computer scientists, and behavior experts. Finally, we develop an agent-based model that allows us to compare different DCT methods and evaluate their performance in negotiating the trade-off between epidemic control and restricting population mobility. Performing extensive sensitivity analysis across user behavior, public health policy, and virological parameters, we compare Rule-based PCT to i) binary contact tracing (BCT), which exclusively relies on test results and recommends a fixed-duration quarantine, and ii) household quarantine (HQ). Our results suggest that both BCT and Rule-based PCT improve upon HQ, however, Rule-based PCT is more efficient at controlling spread of disease than BCT across a range of scenarios. In terms of cost-effectiveness, we show that Rule-based PCT pareto-dominates BCT, as demonstrated by a decrease in Disability Adjusted Life Years, as well as Temporary Productivity Loss. Overall, we find that Rule-based PCT outperforms existing approaches across a varying range of parameters. By leveraging anonymized infectiousness estimates received from digitally-recorded contacts, PCT is able to notify potentially infected users earlier than BCT methods and prevent onward transmissions. Our results suggest that PCT-based applications could be a useful tool in managing future epidemics.

2023-03-12

PLOS Digital Health (published)

doi.org

Visual Question Answering From Another Perspective: CLEVR Mental Rotation Tests

Christopher Pal

Different types of mental rotation tests have been used extensively in psychology to understand human visual reasoning and perception. Under… (see more)standing what an object or visual scene would look like from another viewpoint is a challenging problem that is made even harder if it must be performed from a single image. We explore a controlled setting whereby questions are posed about the properties of a scene if that scene was observed from another viewpoint. To do this we have created a new version of the CLEVR dataset that we call CLEVR Mental Rotation Tests (CLEVR-MRT). Using CLEVR-MRT we examine standard methods, show how they fall short, then explore novel neural architectures that involve inferring volumetric representations of a scene. These volumes can be manipulated via camera-conditioned transformations to answer the question. We examine the efficacy of different model variants through rigorous ablations and demonstrate the efficacy of volumetric representations.

2022-11-29

Pattern Recognition (unknown)

doi.org

openreview.net

Neural Attentive Circuits

Nasim Rahaman

Martin Weiss

Francesco Locatello

Chris Pal

Yoshua Bengio

Bernhard Schölkopf

Li Erran Li

Nicolas Ballas

Recent work has seen the development of general purpose neural architectures that can be trained to perform tasks across diverse data modali… (see more)ties. General purpose models typically make few assumptions about the underlying data-structure and are known to perform well in the large-data regime. At the same time, there has been growing interest in modular neural architectures that represent the data using sparsely interacting modules. These models can be more robust out-of-distribution, computationally efficient, and capable of sample-efficient adaptation to new data. However, they tend to make domain-specific assumptions about the data, and present challenges in how module behavior (i.e., parameterization) and connectivity (i.e., their layout) can be jointly learned. In this work, we introduce a general purpose, yet modular neural architecture called Neural Attentive Circuits (NACs) that jointly learns the parameterization and a sparse connectivity of neural modules without using domain knowledge. NACs are best understood as the combination of two systems that are jointly trained end-to-end: one that determines the module configuration and the other that executes it on an input. We demonstrate qualitatively that NACs learn diverse and meaningful module configurations on the NLVR2 dataset without additional supervision. Quantitatively, we show that by incorporating modularity in this way, NACs improve upon a strong non-modular baseline in terms of low-shot adaptation on CIFAR and CUBs dataset by about 10%, and OOD robustness on Tiny ImageNet-R by about 2.5%. Further, we find that NACs can achieve an 8x speedup at inference time while losing less than 3% performance. Finally, we find NACs to yield competitive results on diverse data modalities spanning point-cloud classification, symbolic processing and text-classification from ASCII bytes, thereby confirming its general purpose nature.

2022-11-28

Conference on Neural Information Processing Systems (Accept)

doi.org

openreview.net

A General-Purpose Neural Architecture for Geospatial Systems

Nasim Rahaman

Martin Weiss

Frederik Träuble

Francesco Locatello

Alexandre Lacoste

Yoshua Bengio

Chris Pal

Li Erran Li

Bernhard Schölkopf

2022-11-01

OpenReview (unknown)

doi.org

openreview.net

Latent Variable Sequential Set Transformers for Joint Multi-agent Motion Prediction

Jim Aldon D’Souza

Samira E. Kahou

Felix Heide

Christopher Pal

Robust multi-agent trajectory prediction is essential for the safe control of robotic systems. A major challenge is to efficiently learn a r… (see more)epresentation that approximates the true joint distribution of contextual, social, and temporal information to enable planning. We propose Latent Variable Sequential Set Transformers which are encoder-decoder architectures that generate scene-consistent multi-agent trajectories. We refer to these architectures as "AutoBots". The encoder is a stack of interleaved temporal and social multi-head self-attention (MHSA) modules which alternately perform equivariant processing across the temporal and social dimensions. The decoder employs learnable seed parameters in combination with temporal and social MHSA modules allowing it to perform inference over the entire future scene in a single forward pass efficiently. AutoBots can produce either the trajectory of one ego-agent or a distribution over the future trajectories for all agents in the scene. For the single-agent prediction case, our model achieves top results on the global nuScenes vehicle motion prediction leaderboard, and produces strong results on the Argoverse vehicle prediction challenge. In the multi-agent setting, we evaluate on the synthetic partition of TrajNet++ dataset to showcase the model's socially-consistent predictions. We also demonstrate our model on general sequences of sets and provide illustrative experiments modelling the sequential structure of the multiple strokes that make up symbols in the Omniglot data. A distinguishing feature of AutoBots is that all models are trainable on a single desktop GPU (1080 Ti) in under 48h.

2022-04-24

International Conference on Learning Representations (Accept (Spotlight))

doi.org

openreview.net

Mila Ventures Launchpad

AI Policy Compass

Virtual Information Sessions

Martin Weiss

Publications

Mila Ventures Launchpad

AI Policy Compass

Virtual Information Sessions

Popular keywords:

Martin Weiss

Publications