Portrait de Mattie Tesfaldet

Mattie Tesfaldet

mattie.tesfaldet@mila.quebec
Collaborateur·rice de recherche - McGill
Superviseur⋅e principal⋅e
Derek Nowrouzezahrai
Co-supervisor
Chris Pal
Sujets de recherche
Apprentissage profond
Modèles génératifs
Systèmes dynamiques
Vision par ordinateur
Site web
Google Scholar
Bluesky
GitHub

Publications

Generative Point Tracking with Flow Matching
Mattie Tesfaldet
Adam W. Harley
Konstantinos G. Derpanis
D. Nowrouzezahrai
Christopher Pal
Tracking a point through a video can be a challenging task due to uncertainty arising from visual obfuscations, such as appearance changes a… (voir plus)nd occlusions. Although current state-of-the-art discriminative models excel in regressing long-term point trajectory estimates -- even through occlusions -- they are limited to regressing to a mean (or mode) in the presence of uncertainty, and fail to capture multi-modality. To overcome this limitation, we introduce Generative Point Tracker (GenPT), a generative framework for modelling multi-modal trajectories. GenPT is trained with a novel flow matching formulation that combines the iterative refinement of discriminative trackers, a window-dependent prior for cross-window consistency, and a variance schedule tuned specifically for point coordinates. We show how our model's generative capabilities can be leveraged to improve point trajectory estimates by utilizing a best-first search strategy on generated samples during inference, guided by the model's own confidence of its predictions. Empirically, we evaluate GenPT against the current state of the art on the standard PointOdyssey, Dynamic Replica, and TAP-Vid benchmarks. Further, we introduce a TAP-Vid variant with additional occlusions to assess occluded point tracking performance and highlight our model's ability to capture multi-modality. GenPT is capable of capturing the multi-modality in point trajectories, which translates to state-of-the-art tracking accuracy on occluded points, while maintaining competitive tracking accuracy on visible points compared to extant discriminative point trackers.
2025-10-22
ArXiv (prépublication)
doi.org
arxiv.org
Attention-based Neural Cellular Automata
Mattie Tesfaldet
Derek Nowrouzezahrai
Christopher Pal
Recent extensions of Cellular Automata (CA) have incorporated key ideas from modern deep learning, dramatically extending their capabilities… (voir plus) and catalyzing a new family of Neural Cellular Automata (NCA) techniques. Inspired by Transformer-based architectures, our work presents a new class of
2022-11-28
Conference on Neural Information Processing Systems (Accept)
doi.org
openreview.net