Ghait Boukachab

Learning Decision Trees as Amortized Structure Inference

Building predictive models for tabular data presents fundamental challenges, notably in scaling consistently, i.e., more resources translati… (see more)ng to better performance, and generalizing systematically beyond the training data distribution. Designing decision tree models remains especially challenging given the intractably large search space, and most existing methods rely on greedy heuristics, while deep learning inductive biases expect a temporal or spatial structure not naturally present in tabular data. We propose a hybrid amortized structure inference approach to learn predictive decision tree ensembles given data, formulating decision tree construction as a sequential planning problem. We train a deep reinforcement learning (GFlowNet) policy to solve this problem, yielding a generative model that samples decision trees from the Bayesian posterior. We show that our approach, DT-GFN, outperforms state-of-the-art decision tree and deep learning methods on standard classification benchmarks derived from real-world data, robustness to distribution shifts, and anomaly detection, all while yielding interpretable models with shorter description lengths. Samples from the trained DT-GFN model can be ensembled to construct a random forest, and we further show that the performance of scales consistently in ensemble size, yielding ensembles of predictors that continue to generalize systematically.

2025-03-10

ArXiv (preprint)

doi.org

arxiv.org

Learning Decision Trees as Amortized Structure Inference

Mohammed Mahfoud

Ghait Boukachab

Michał Koziarski

Alex Hernandez-Garcia

Stefan Bauer

Yoshua Bengio

Nikolay Malkin

2025-03-10

ArXiv (preprint)

arxiv.org

Learning Decision Trees as Amortized Structure Inference

Mohammed Mahfoud

Ghait Boukachab

Michał Koziarski

Alex Hernandez-Garcia

Stefan Bauer

Yoshua Bengio

Nikolay Malkin

2025-03-05

ICLR.cc/2025/Workshop/FPI (poster)

doi.org

openreview.net

Deep learning based vessel arrivals monitoring via autoregressive statistical control charts

Sara El Mekkaoui

Ghait Boukachab

Loubna Benabbou

Abdelaziz Berrado

2024-07-01

WMU Journal of Maritime Affairs (JoMA) (published)

doi.org

Deep learning based vessel arrivals monitoring via autoregressive statistical control charts

Sara El Mekkaoui

Ghait Boukachab

Loubna Benabbou

Abdelaziz Berrado

2024-07-01

WMU Journal of Maritime Affairs (JoMA) (published)

doi.org

Improving *day-ahead* Solar Irradiance Time Series Forecasting by Leveraging Spatio-Temporal Context

Solar power harbors immense potential in mitigating climate change by substantially reducing CO…

openreview.net

What if We Enrich day-ahead Solar Irradiance Time Series Forecasting with Spatio-Temporal Context?

The global integration of solar power into the electrical grid could have a crucial impact on climate change mitigation, yet poses a challen… (see more)ge due to solar irradiance variability. We present a deep learning architecture which uses spatio-temporal context from satellite data for highly accurate day-ahead time-series forecasting, in particular Global Horizontal Irradiance (GHI). We provide a multi-quantile variant which outputs a prediction interval for each time-step, serving as a measure of forecasting uncertainty. In addition, we suggest a testing scheme that separates easy and difficult scenarios, which appears useful to evaluate model performance in varying cloud conditions. Our approach exhibits robust performance in solar irradiance forecasting, including zero-shot generalization tests at unobserved solar stations, and holds great promise in promoting the effective use of solar power and the resulting reduction of CO

2023-07-28

ICML.cc/2023/Workshop/SynS_and_ML (published)

openreview.net