Portrait of Fırat Öncel is unavailable

Fırat Öncel

firat.oncel@mila.quebec
PhD - Concordia University
Supervisor
Mirco Ravanelli
Co-supervisor
Laurent Charlin
Research Topics
Audio Processing
Computer Vision
Deep Learning
Large Language Models (LLM)
Machine Learning For Speech and Audio
Recommender Systems
Representation Learning
Website
Google Scholar
GitHub

Publications

Time-Incremental Continued Pretraining of LLMs: Knowledge Updates Without Catastrophic Forgetting
Fırat Öncel
Salman Hussain Ali
Mirco Ravanelli
Cem Subakan
Çağatay Yıldız
Large language models (LLMs) drift out of date the moment their pretraining ends, yet retraining from scratch is prohibitively expensive. Co… (see more)ntinued pretraining (CPT) is the natural remedy, but it is typically evaluated through a continual learning lens that assumes disjoint data streams. This is a poor fit for time-incremental updates on web-scale crawls, where successive snapshots share substantial URL overlap by design. We study time-incremental CPT in this realistic regime: continued pretraining on FineWeb-Edu dumps drawn strictly from after each model's knowledge cutoff, evaluated across six open-weight models spanning three families (OLMo2, Llama-3.1/3.2, Gemma-3-1B) and four parameter scales (1B-3B-7B-8B). We organize our findings around four practical questions. (i) Is knowledge acquired? Yes, but heterogeneously, and without catastrophic forgetting: five of six models also improve on pre-cutoff factual recall, and the gains track pretraining saturation (driven primarily by token budget per parameter). (ii) What does it cost? Almost nothing: the macro-average across a thirteen-task suite stays within 0.01 of the base for every model. (iii) What is the recipe? Data quality dominates quantity (a curated 6B-token slice matches a broader 40B one); the optima for knowledge acquisition and general capability are separated by roughly an order of magnitude in learning rate; and LoRA at sufficient rank matches full CPT. (iv) Does it survive deployment? CPT gains transfer through SFT, while DPO's effect is family-dependent. Together, these results paint a more optimistic picture of time-incremental CPT than the prior continual learning literature suggests.
2026-09-19
arXiv (preprint)
doi.org
arxiv.org
Controllable and Content-Based Recommendations
Fırat Öncel
Jihoon Jeong
Emiliano Penaloza
Mirco Ravanelli
Laurent Charlin
Cem Subakan
Traditional recommendation systems rely on latent (dense) representations, making them difficult to interpret and control. We propose the Co… (see more)ntrollable and Content-Based Recommendations (CCBR) framework, which builds its recommendations from textual user profile representations. CCBR plugs into collaborative filtering models and introduces controllability via text bottlenecks. We show that CCBR enables text-based and multimodal interventions, allowing users to steer the model towards the directions they prefer. Different from existing controllable recommendation systems, CCBR infers the text summaries directly from item contents (images, audio or video). Across image-, audio-, and video-based datasets, we demonstrate that the proposed framework obtains competitive model performance with standard (latent-representation) models while providing controllable model summaries via text. The model also outperforms TEARS, a recent baseline for controllable recommendation systems. Through systematic interventions, we demonstrate the efficacy of the user steering mechanism.
2026-07-22
arXiv (preprint)
doi.org
arxiv.org
Audio Prototypical Network For Controllable Music Recommendation
Fırat Öncel
Emiliano Penaloza
Haolun Wu
Shubham Gupta
Mirco Ravanelli
Laurent Charlin
Cem Subakan
Traditional recommendation systems represent user preferences in dense representations obtained through black-box encoder models. While thes… (see more)e models often provide strong recommendation performance, they lack interpretability for users, leaving users unable to understand or control the system's modeling of their preferences. This limitation is especially challenging in music recommendation, where user preferences are highly personal and often evolve based on nuanced qualities like mood, genre, tempo, or instrumentation. In this paper, we propose an audio prototypical network for controllable music recommendation. This network expresses user preferences in terms of prototypes representative of semantically meaningful features pertaining to musical qualities. We show that the model obtains competitive recommendation performance compared to popular baseline models while also providing interpretable and controllable user profiles.
2024-12-31
MLSP (published)
doi.org
openreview.net
Adaptation Odyssey in LLMs: Why Does Additional Pretraining Sometimes Fail to Improve?
Fırat Öncel
Matthias Bethge
Beyza Ermis
Mirco Ravanaelli
Yusuf Cem Sübakan
cCaugatay Yildiz
2023-12-31
EMNLP (published)
doi.org
arxiv.org