Éric Granger

Alessandro Lameiras Koerich

Simon L Bacon

Oya Celiktutan

Soufiane Belharbi

Muhammad Osama Zeeshan

Muhammad Haseeb Aslam

Chunchang Shao

Guanyu Hu

The 11th Affective & Behavior Analysis in-the-Wild (ABAW) Workshop and Competition, held in conjunction with ECCV 2026, continues to advance… (voir plus) research on the modelling, analysis and understanding of human affect and behavior in unconstrained real-world environments. The workshop maintains its dual structure, comprising both a competition and a paper track. The ABAW Competition features two challenging benchmarks targeting key problems in affective and behavioral understanding: Multi-Task Learning (MTL), requiring the joint estimation of valence-arousal, facial expressions and action units from a static version of the Aff-Wild2 database, and Ambivalence/Hesitancy (AH) Video Recognition, focusing on the recognition of complex behavioral states from multimodal videos using the BAH dataset. In parallel, the paper track presents recent advances spanning multimodal AI and deep learning, affective computing, behavior understanding, datasets and benchmarks, human-centered AI, human-robot interaction and social robotics, robustness, explainability and real-world applications. Overall, the 11th ABAW Workshop and Competition continues to serve as a leading forum for benchmarking, collaboration and innovation, fostering the development of the next generation of multimodal, trustworthy and human-centered AI systems.

2026-09-02

ABAW @ European Conference on Computer Vision (publié)

openreview.net

From Affect to Complex Behavior: Advancing Multimodal Human-Centered AI at the 10th ABAW Workshop & Competition

Dimitrios Kollias

Panagiotis Tzirakis

Alan Cowen

Stefanos Zafeiriou

Irene Kotsia

Simon Bacon

Jens Madsen

Soufiane Belharbi

Muhammad Haseeb Aslam

Chunchang Shao

Guanyu Hu

The 10th Affective & Behavior Analysis in-the-Wild (ABAW) Workshop and Competition, held at CVPR 2026, continues to advance research on … (voir plus)modelling, analysis, understanding of human affect and behavior in real-world, unconstrained environments. The workshop maintains its dual structure, comprising both a competition and a paper track. The ABAW Competition introduces a diverse set of challenges targeting key aspects of affective and behavioral understanding, including continuous affect (valence-arousal) estimation, discrete affect (expression and action unit) recognition, as well as more complex behavior analysis tasks, such as emotional mimicry intensity estimation, ambivalence/hesitancy recognition and fine-grained violence detection. These challenges are built upon large-scale in-the-wild datasets, providing comprehensive benchmarks for state-of-the-art approaches. In parallel, the paper track presents a wide range of contributions spanning pose, motion & behavior estimation, affect modelling & multimodal learning, benchmarks, datasets & evaluation protocols, fairness, robustness & deployment. Overall, the 10th ABAW Workshop and Competition continues to serve as a key platform for benchmarking, collaboration and innovation, shaping the development of next-generation multimodal, human-centered AI systems.

2026-05-23

arXiv (prépublication)

Ambivalence/Hesitancy Recognition in Videos for Personalized Digital Health Interventions

Manuela González‐González

Soufiane Belharbi

Muhammad Zeeshan

Masoumeh Sharafi

Muhammad Haseeb Aslam

Lorenzo Sia

Nicolas Richet

Alessandro L. Koerich

Simon L Bacon

Using behavioural science, health interventions focus on behaviour change by providing a framework to help patients acquire and maintain hea… (voir plus)lthy habits that improve medical outcomes. In-person interventions are costly and difficult to scale, especially in resource-limited regions. Digital health interventions offer a cost-effective approach, potentially supporting independent living and self-management. Automating such interventions, especially through machine learning, has gained considerable attention recently. Ambivalence and hesitancy (A/H) play a primary role for individuals to delay, avoid, or abandon health interventions. A/H are subtle and conflicting emotions that place a person in a state between positive and negative evaluations of a behaviour, or between acceptance and refusal to engage in it. They manifest as affective inconsistency across modalities or within a modality, such as language, facial, vocal expressions, and body language. While experts can be trained to recognize A/H, integrating them into digital health interventions is costly and less effective. Automatic A/H recognition is therefore critical for the personalization and cost-effectiveness of digital health interventions. Here, we explore the application of deep learning models for A/H recognition in videos, a multi-modal task by nature. In particular, this paper covers three learning setups: supervised learning, unsupervised domain adaptation for personalization, and zero-shot inference via large language models (LLMs). Our experiments are conducted on the unique and recently published BAH video dataset for A/H recognition. Our results show limited performance, suggesting that more adapted multi-modal models are required for accurate A/H recognition. Better methods for modeling spatio-temporal and multimodal fusion are necessary to leverage conflicts within/across modalities.

2026-04-12

arXiv (prépublication)

CLIP-AUTT: Test-Time Personalization with Action Unit Prompting for Fine-Grained Video Emotion Recognition

Muhammad Zeeshan

Masoumeh Sharafi

Benoit Savary

Alessandro L. Koerich

Personalization in emotion recognition (ER) is essential for an accurate interpretation of subtle and subject-specific expressive patterns. … (voir plus)Recent advances in vision-language models (VLMs) such as CLIP demonstrate strong potential for leveraging joint image-text representations in ER. However, CLIP-based methods either depend on CLIP's contrastive pretraining or on LLMs to generate descriptive text prompts, which are noisy, computationally expensive, and fail to capture fine-grained expressions, leading to degraded performance. In this work, we leverage Action Units (AUs) as structured textual prompts within CLIP to model fine-grained facial expressions. AUs encode the subtle muscle activations underlying expressions, providing localized and interpretable semantic cues for more robust ER. We introduce CLIP-AU, a lightweight AU-guided temporal learning method that integrates interpretable AU semantics into CLIP. It learns generic, subject-agnostic representations by aligning AU prompts with facial dynamics, enabling fine-grained ER without CLIP fine-tuning or LLM-generated text supervision. Although CLIP-AU models fine-grained AU semantics, it does not adapt to subject-specific variability in subtle expressions. To address this limitation, we propose CLIP-AUTT, a video-based test-time personalization method that dynamically adapts AU prompts to videos from unseen subjects. By combining entropy-guided temporal window selection with prompt tuning, CLIP-AUTT enables subject-specific adaptation while preserving temporal consistency. Our extensive experiments on three challenging video-based subtle ER datasets, BioVid, StressID, and BAH, indicate that CLIP-AU and CLIP-AUTT outperform state-of-the-art CLIP-based FER and TTA methods, achieving robust and personalized subtle ER. Our code is publicly available at: https://github.com/osamazeeshan/CLIP-AUTT.

2026-03-29

arXiv (prépublication)

Test-Time Adaptation via Cache Personalization for Facial Expression Recognition in Videos

Masoumeh Sharafi

Muhammad Zeeshan

Soufiane Belharbi

Alessandro L. Koerich

Facial expression recognition (FER) in videos requires model personalization to capture the considerable variations across subjects. Vision-… (voir plus)language models (VLMs) offer strong transfer to downstream tasks through image-text alignment, but their performance can still degrade under inter-subject distribution shifts. Personalizing models using test-time adaptation (TTA) methods can mitigate this challenge. However, most state-of-the-art TTA methods rely on unsupervised parameter optimization, introducing computational overhead that is impractical in many real-world applications. This paper introduces TTA through Cache Personalization (TTA-CaP), a cache-based TTA method that enables cost-effective (gradient-free) personalization of VLMs for video FER. Prior cache-based TTA methods rely solely on dynamic memories that store test samples, which can accumulate errors and drift due to noisy pseudo-labels. TTA-CaP leverages three coordinated caches: a personalized source cache that stores source-domain prototypes, a positive target cache that accumulates reliable subject-specific samples, and a negative target cache that stores low-confidence cases as negative samples to reduce the impact of noisy pseudo-labels. Cache updates and replacement are controlled by a tri-gate mechanism based on temporal stability, confidence, and consistency with the personalized cache. Finally, TTA-CaP refines predictions through fusion of embeddings, yielding refined representations that support temporally stable video-level predictions. Our experiments on three challenging video FER datasets, BioVid, StressID, and BAH, indicate that TTA-CaP can outperform state-of-the-art TTA methods under subject-specific and environmental shifts, while maintaining low computational and memory overhead for real-world deployment.

2026-03-21

arXiv (prépublication)

Leveraging Diversity for Privileged Multi-Teacher Knowledge Distillation for Facial Expression Recognition

Muhammad Haseeb Aslam

Alessandro L. Koerich

2025-12-31

SSRN Electronic Journal (accepté)

Learning Task-Agnostic Representations through Multi-Teacher Distillation

Jackie CK Cheung

Ismail Ben Ayed

Mohammadhadi Shateri

Pablo Piantanida

Casting complex inputs into tractable representations is a critical step across various fields. Diverse embedding models emerge from differe… (voir plus)nces in architectures, loss functions, input modalities and datasets, each capturing unique aspects of the input. Multi-teacher distillation leverages this diversity to enrich representations but often remains tailored to specific tasks. In this paper, we introduce a task-agnostic framework based on a ``majority vote" objective function. We demonstrate that this function is bounded by the mutual information between student and teachers' embeddings, leading to a task-agnostic distillation loss that eliminates dependence on task-specific labels or prior knowledge. Our evaluations across text, vision models, and molecular modeling show that our method effectively leverages teacher diversity, resulting in representations enabling better performance for a wide range of downstream tasks such as classification, clustering, or regression. Additionally, we train and release state-of-the-art embedding models, enhancing downstream performance in various modalities.

2025-12-30

OpenReview (publié)

openreview.net

LT-Soups: Bridging Head and Tail Classes via Subsampled Model Soups

Masih Aminbeidokhti

Subhankar Roy

Elisa Ricci

Real-world datasets typically exhibit long-tailed (LT) distributions, where a few head classes dominate and many tail classes are severely u… (voir plus)nderrepresented. While recent work shows that parameter-efficient fine-tuning (PEFT) methods like LoRA and AdaptFormer preserve tail-class performance on foundation models such as CLIP, we find that they do so at the cost of head-class accuracy. We identify the head-tail ratio, the proportion of head to tail classes, as a crucial but overlooked factor influencing this trade-off. Through controlled experiments on CIFAR100 with varying imbalance ratio (

2025-11-10

ArXiv (prépublication)

High-Rate Mixout: Revisiting Mixout for Robust Domain Generalization

Masih Aminbeidokhti

Heitor Rapela Medeiros

Srikanth Muralidharan

2025-10-07

ArXiv (prépublication)

Revisiting Mixout: An Overlooked Path to Robust Finetuning

Masih Aminbeidokhti

Heitor Rapela Medeiros

Finetuning vision foundation models often improves in-domain accuracy but comes at the cost of robustness under distribution shift. We revis… (voir plus)it Mixout, a stochastic regularizer that intermittently replaces finetuned weights with their pretrained reference, through the lens of a single-run, weight-sharing implicit ensemble. This perspective reveals three key levers that govern robustness: the \emph{masking anchor}, \emph{resampling frequency}, and \emph{mask sparsity}. Guided by this analysis, we introduce GMixout, which (i) replaces the fixed anchor with an exponential moving-average snapshot that adapts during training, and (ii) regulates masking period via an explicit resampling-frequency hyperparameter. Our sparse-kernel implementation updates only a small fraction of parameters with no inference-time overhead, enabling training on consumer-grade GPUs. Experiments on benchmarks covering covariate shift, corruption, and class imbalance, ImageNet / ImageNet-LT, DomainNet, iWildCam, and CIFAR100-C, GMixout consistently improves in-domain accuracy beyond zero-shot performance while surpassing both Model Soups and strong parameter-efficient finetuning baselines under distribution shift.

2025-10-07

ArXiv (prépublication)

VLOD-TTA: Test-Time Adaptation of Vision-Language Object Detectors

Atif Belal

Heitor Rapela Medeiros

2025-09-30

ArXiv (prépublication)