Publications

Mock Deep Testing: Toward Separate Development of Data and Models for Deep Learning
Ruchira Manke
Mohammad Wardat
Foutse Khomh
Hridesh Rajan
While deep learning (DL) has permeated, and become an integral component of many critical software systems, today software engineering resea… (see more)rch hasn't explored how to separately test data and models that are integral for DL approaches to work effectively. The main challenge in independently testing these components arises from the tight dependency between data and models. This research explores this gap, introducing our methodology of mock deep testing for unit testing of DL applications. To enable unit testing, we introduce a design paradigm that decomposes the workflow into distinct, manageable components, minimizes sequential dependencies, and modularizes key stages of the DL. For unit testing these components, we propose modeling their dependencies using mocks. This modular approach facilitates independent development and testing of the components, ensuring comprehensive quality assurance throughout the development process. We have developed KUnit, a framework for enabling mock deep testing for the Keras library. We empirically evaluated KUnit to determine the effectiveness of mocks. Our assessment of 50 DL programs obtained from Stack Overflow and GitHub shows that mocks effectively identified 10 issues in the data preparation stage and 53 issues in the model design stage. We also conducted a user study with 36 participants using KUnit to perceive the effectiveness of our approach. Participants using KUnit successfully resolved 25 issues in the data preparation stage and 38 issues in the model design stage. Our findings highlight that mock objects provide a lightweight emulation of the dependencies for unit testing, facilitating early bug detection. Lastly, to evaluate the usability of KUnit, we conducted a post-study survey. The results reveal that KUnit is helpful to DL application developers, enabling them to independently test each component effectively in different stages.
2025-05-05
2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE) (published)
doi.org
arxiv.org
Kernel-Level Event-Based Performance Anomaly Detection in Software Systems under Varying Load Conditions
Anthonia Njoku
Heng Li
Foutse Khomh
2025-05-04
Companion of the 16th ACM/SPEC International Conference on Performance Engineering (published)
doi.org
The Search for Squawk: Agile Modeling in Bioacoustics
Vincent Dumoulin
Otilia Stretcu
Jenny Hamer
Lauren Harrell
Rob Laber
Hugo Larochelle
Bart Van Merriënboer
Amanda K. Navine
Patrick Hart
Ben Williams
Timothy A. C. Lamont
Tries B. Rasak
Mars Coral Restoration Team
Sheryn Brodie
Brendan Doohan
Philip Eichinski
Paul Roe
Lin Schwarzkopf
Tom Denton
2025-05-04
ArXiv (preprint)
arxiv.org
arxiv.org
Ant Colony Sampling with GFlowNets for Combinatorial Optimization
Minsu Kim
Sanghyeok Choi
Jiwoo Son
Hyeonah Kim
Jinkyoo Park
Yoshua Bengio
We present the Generative Flow Ant Colony Sampler (GFACS), a novel meta-heuristic method that hierarchically combines amortized inference an… (see more)d parallel stochastic search. Our method first leverages Generative Flow Networks (GFlowNets) to amortize a \emph{multi-modal} prior distribution over combinatorial solution space that encompasses both high-reward and diversified solutions. This prior is iteratively updated via parallel stochastic search in the spirit of Ant Colony Optimization (ACO), leading to the posterior distribution that generates near-optimal solutions. Extensive experiments across seven combinatorial optimization problems demonstrate GFACS's promising performances.
2025-05-02
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
Fair Resource Allocation in Weakly Coupled Markov Decision Processes
Xiaohui Tu
Yossiri Adulyasak
Nima Akbarzadeh
Erick Delage
We consider fair resource allocation in sequential decision-making environments modeled as weakly coupled Markov decision processes, where r… (see more)esource constraints couple the action spaces of
2025-05-02
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
Q-learning for Quantile MDPs: A Decomposition, Performance, and Convergence Analysis
Jia Lin Hau
Erick Delage
Esther Derman
Mohammad Ghavamzadeh
Marek Petrik
In Markov decision processes (MDPs), quantile risk measures such as Value-at-Risk are a standard metric for modeling RL agents' preferences … (see more)for certain outcomes. This paper proposes a new Q-learning algorithm for quantile optimization in MDPs with strong convergence and performance guarantees. The algorithm leverages a new, simple dynamic program (DP) decomposition for quantile MDPs. Compared with prior work, our DP decomposition requires neither known transition probabilities nor solving complex saddle point equations and serves as a suitable foundation for other model-free RL algorithms. Our numerical results in tabular domains show that our Q-learning algorithm converges to its DP variant and outperforms earlier algorithms.
2025-05-02
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
On the Identifiability of Causal Abstractions
Xiusi Li
Sékou-Oumar Kaba
Siamak Ravanbakhsh
Causal representation learning (CRL) enhances machine learning models' robustness and generalizability by learning structural causal models … (see more)associated with data-generating processes. We focus on a family of CRL methods that uses contrastive data pairs in the observable space, generated before and after a random, unknown intervention, to identify the latent causal model. (Brehmer et al., 2022) showed that this is indeed possible, given that all latent variables can be intervened on individually. However, this is a highly restrictive assumption in many systems. In this work, we instead assume interventions on arbitrary subsets of latent variables, which is more realistic. We introduce a theoretical framework that calculates the degree to which we can identify a causal model, given a set of possible interventions, up to an abstraction that describes the system at a higher level of granularity.
2025-05-02
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
"It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Models
Angel Hsing-Chi Hwang
Q. Vera Liao
Su Lin Blodgett
A.R. Olteanu
Adam Trischler
2025-05-01
Proceedings of the ACM on Human-Computer Interaction (published)
doi.org
arxiv.org
Alberta Wells Dataset: Pinpointing Oil and Gas Wells from Satellite Imagery
Pratinav Seth
Michelle Lin
Brefo Dwamena Yaw
Jade Boutot
Mary Kang
David Rolnick
Millions of abandoned oil and gas wells are scattered across the world, leaching methane into the atmosphere and toxic compounds into the gr… (see more)oundwater. Many of these locations are unknown, preventing the wells from being plugged and their polluting effects averted. Remote sensing is a relatively unexplored tool for pinpointing abandoned wells at scale. We introduce the first large-scale benchmark dataset for this problem, leveraging medium-resolution multi-spectral satellite imagery from Planet Labs. Our curated dataset comprises over 213,000 wells (abandoned, suspended, and active) from Alberta, a region with especially high well density, sourced from the Alberta Energy Regulator and verified by domain experts. We evaluate baseline algorithms for well detection and segmentation, showing the promise of computer vision approaches but also significant room for improvement.
2025-04-30
International Conference on Machine Learning (poster)
doi.org
proceedings.mlr.press
Burst firing optimizes invariant coding of natural communication signals by electrosensory neural populations
Michael G. Metzen
Amin Akhshi
Pouya Bashivan
Anmar Khadra
Maurice J. Chacron
Accurate perception of objects within the environment independent of context is essential for the survival of an organism. While neurons tha… (see more)t respond in an invariant manner to different stimulus waveforms resulting from identitypreserving transformations of objects are thought to provide a neural correlate of context-independent perception, how such responses emerge in the brain remains poorly understood. Here, we demonstrate that burst firing in neural populations can give rise to an invariant representation of highly heterogeneous natural communication stimuli. Multi-unit recordings from central sensory neural populations showed that considering burst spike trains led to invariant representations at the population but not the single neuron level. Computational modeling further revealed that optimal invariance is achieved at burst firing levels seen experimentally. Taken together, our results demonstrate an important function for burst firing toward establishing invariant representations of sensory input in neural populations.
2025-04-30
iScience (published)
doi.org
Context is Key: A Benchmark for Forecasting with Essential Textual Information
Arjun Ashok
Andrew R. Williams
Étienne Marcotte
Valentina Zantedeschi
Jithendaraa Subramanian
Roland Riachi
James Requeima
Alexandre Lacoste
Irina Rish
Nicolas Chapados
Alexandre Drouin
Forecasting is a critical task in decision-making across numerous domains. While historical numerical data provide a start, they fail to con… (see more)vey the complete context for reliable and accurate predictions. Human forecasters frequently rely on additional information, such as background knowledge and constraints, which can efficiently be communicated through natural language. However, in spite of recent progress with LLM-based forecasters, their ability to effectively integrate this textual information remains an open question. To address this, we introduce "Context is Key" (CiK), a time-series forecasting benchmark that pairs numerical data with diverse types of carefully crafted textual context, requiring models to integrate both modalities; crucially, every task in CiK requires understanding textual context to be solved successfully. We evaluate a range of approaches, including statistical models, time series foundation models, and LLM-based forecasters, and propose a simple yet effective LLM prompting method that outperforms all other tested methods on our benchmark. Our experiments highlight the importance of incorporating contextual information, demonstrate surprising performance when using LLM-based forecasting models, and also reveal some of their critical shortcomings. This benchmark aims to advance multimodal forecasting by promoting models that are both accurate and accessible to decision-makers with varied technical expertise. The benchmark can be visualized at https://servicenow.github.io/context-is-key-forecasting/v0/.
2025-04-30
International Conference on Machine Learning (poster)
doi.org
proceedings.mlr.press
Diminished social memory and hippocampal correlates of social interactions in chronic social defeat stress susceptibility
Amanda Larosa
Tian Rui Zhang
Alice S. Wong
Cyrus Y.H. Fung
Y. H. Fung Cyrus
Xiong Ling Yun (Jenny) Long
Prabhjeet Singh
Benjamin C. M. Fung
Tak Pan Wong
The susceptibility to chronic stress has been associated with depression, a mood disorder which highly implicates the hippocampus. Hippocamp… (see more)al contribution to stress susceptibility has been supported by findings in mice following chronic social defeat stress (CSDS). However, little is known of the role of hippocampal activity in determining the development of stress susceptibility. We used the UCLA miniscope to longitudinally measure the activity of dorsal CA1 hippocampal neurons across CSDS. Apart from examining the representation of social information by these neurons, we also compared social memory in mice that were susceptible or resilient to CSDS. We observed more stable dCA1 correlates of social interaction and social memory in CSDS resilience. Such changes were absent in CSDS susceptible mice and accompanied by greater social memory impairments. CSDS susceptibility may be supported by hippocampal social cognitive processes, reflected in diminished hippocampal representations of social information and a greater impairment in social memory.
2025-04-30
Biological Psychiatry Global Open Science (published)
doi.org