Portrait de Mathieu Bazinet n'est pas disponible

Mathieu Bazinet

Doctorat - Université Laval
Superviseur⋅e principal⋅e
Pascal Germain
Sujets de recherche
Apprentissage de représentations
Théorie de l'apprentissage automatique
GitHub

Publications

Generalization Bounds via Meta-Learned Model Representations: PAC-Bayes and Sample Compression Hypernetworks
Benjamin Leblanc
Mathieu Bazinet
Nathaniel D'Amours
Alexandre Drouin
Pascal Germain
Both PAC-Bayesian and Sample Compress learning frameworks have been shown instrumental for deriving tight (non-vacuous) generalization bound… (voir plus)s for neural networks. We leverage these results in a meta-learning scheme, relying on a hypernetwork that outputs the parameters of a downstream predictor from a dataset input. The originality of our approach lies in the investigated hypernetwork architectures that encode the dataset before decoding the parameters: (1) a PAC-Bayesian encoder that expresses a posterior distribution over a latent space, (2) a Sample Compress encoder that selects a small sample of the dataset input along with a message from a discrete set, and (3) a hybrid between both approaches motivated by a new Sample Compress theorem handling continuous messages. The latter theorem exploits the pivotal information transiting at the encoder-decoder junction in order to compute generalization guarantees for each downstream predictor obtained by our meta-learning scheme.
2025-05-01
ICML.cc/2025/Conference (poster)
openreview.net
openreview.net
Sample Compression for Continual Learning
Jacob Comeau
Mathieu Bazinet
Pascal Germain
Cem Subakan
2025-03-13
ArXiv (prépublication)
arxiv.org
arxiv.org
Sample Compression for Continual Learning
Jacob Comeau
Mathieu Bazinet
Pascal Germain
Cem Subakan
2025-03-13
ArXiv (prépublication)
doi.org
arxiv.org
Sample Compression for Self Certified Continual Learning
Jacob Comeau
Mathieu Bazinet
Pascal Germain
Cem Subakan
Continual learning algorithms aim to learn from a sequence of tasks, making the training distribution non-stationary. The majority of existi… (voir plus)ng continual learning approaches in the literature rely on heuristics and do not provide learning guarantees. In this paper, we present a new method called Continual Pick-to-Learn (CoP2L), which is able to retain the most representative samples for each task in an efficient way. CoP2L combines the Pick-to-Learn algorithm (rooted in the sample compression theory) and the experience replay continual learning scheme. This allows us to provide non-vacuous upper bounds on the generalization loss of the learned predictors, numerically computable after each task. We empirically evaluate our approach on several standard continual learning benchmarks across Class-Incremental, Task-Incremental, and Domain-Incremental settings. Our results show that CoP2L is highly competitive across all setups, often outperforming existing baselines, and significantly mitigating catastrophic forgetting compared to vanilla experience replay in the Class-Incremental setting. It is possible to leverage the bounds provided by CoP2L in practical scenarios to certify the predictor reliability on previously learned tasks, in order to improve the trustworthiness of the continual learning algorithm.
2025-03-13
ArXiv (prépublication)
arxiv.org
arxiv.org
Sample Compression for Self Certified Continual Learning
Jacob Comeau
Mathieu Bazinet
Pascal Germain
Cem Subakan
Continual learning algorithms aim to learn from a sequence of tasks, making the training distribution non-stationary. The majority of existi… (voir plus)ng continual learning approaches in the literature rely on heuristics and do not provide learning guarantees. In this paper, we present a new method called Continual Pick-to-Learn (CoP2L), which is able to retain the most representative samples for each task in an efficient way. CoP2L combines the Pick-to-Learn algorithm (rooted in the sample compression theory) and the experience replay continual learning scheme. This allows us to provide non-vacuous upper bounds on the generalization loss of the learned predictors, numerically computable after each task. We empirically evaluate our approach on several standard continual learning benchmarks across Class-Incremental, Task-Incremental, and Domain-Incremental settings. Our results show that CoP2L is highly competitive across all setups, often outperforming existing baselines, and significantly mitigating catastrophic forgetting compared to vanilla experience replay in the Class-Incremental setting. It is possible to leverage the bounds provided by CoP2L in practical scenarios to certify the predictor reliability on previously learned tasks, in order to improve the trustworthiness of the continual learning algorithm.
2025-03-13
ArXiv (prépublication)
arxiv.org
arxiv.org
Sample compression unleashed : New generalization bounds for real valued losses
Mathieu Bazinet
Valentina Zantedeschi
Pascal Germain
2025-01-22
aistats.org/AISTATS/2025/Conference (poster)
proceedings.mlr.press
openreview.net
Sample Compression Hypernetworks: From Generalization Bounds to Meta-Learning
Benjamin Leblanc
Mathieu Bazinet
Nathaniel D'Amours
Alexandre Drouin
Pascal Germain
Reconstruction functions are pivotal in sample compression theory, a framework for deriving tight generalization bounds. From a small sample… (voir plus) of the training set (the compression set) and an optional stream of information (the message), they recover a predictor previously learned from the whole training set. While usually fixed, we propose to learn reconstruction functions. To facilitate the optimization and increase the expressiveness of the message, we derive a new sample compression generalization bound for real-valued messages. From this theoretical analysis, we then present a new hypernetwork architecture that outputs predictors with tight generalization guarantees when trained using an original meta-learning framework. The results of promising preliminary experiments are then reported.
2024-10-09
NeurIPS.cc/2024/Workshop/Compression (publié)
openreview.net
openreview.net