Alfa: Attentive Low-Rank Filter Adaptation for Structure-Aware Cross-Domain Personalized Gaze Estimation

2026-03-09

AI Summary

This work addresses the performance degradation of pre-trained gaze estimation models caused by user-specific anatomical variations, such as eyelid shape and facial geometry, by proposing a label-free test-time personalization method. The approach formulates personalization as an attention-based reweighting of semantic patterns within pre-trained filters, leveraging singular value decomposition to extract dominant spatial components and integrating a low-rank attention mechanism to enable parameter-efficient fine-tuning (PEFT). With only a few unlabeled samples, the method accurately adapts to individual differences. Evaluated on four cross-dataset gaze estimation benchmarks, it achieves the lowest average angular error, significantly outperforming existing test-time adaptation techniques and LoRA variants. Furthermore, the method demonstrates strong generalization capability beyond vision tasks, showing effectiveness in non-visual domains such as diffusion language models.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Pre-trained gaze models learn to identify useful patterns commonly found across users, but subtle user-specific variations (i.e., eyelid shape or facial structure) can degrade model performance. Test-time personalization (TTP) adapts pre-trained models to these user-specific domain shifts using only a few unlabeled samples. Efficient fine-tuning is critical in performing this domain adaptation: data and computation resources can be limited-especially for on-device customization. While popular parameter-efficient fine-tuning (PEFT) methods address adaptation costs by updating only a small set of weights, they may not be taking full advantage of structures encoded in pre-trained filters. To more effectively leverage existing structures learned during pre-training, we reframe personalization as a process to reweight existing features rather than learning entirely new ones. We present Attentive Low-Rank Filter Adaptation (Alfa) to adapt gaze models by reweighting semantic patterns in pre-trained filters. With Alfa, singular value decomposition (SVD) extracts dominant spatial components that capture eye and facial characteristics across users. Via an attention mechanism, we need only a few unlabeled samples to adjust and reweight pre-trained structures, selectively amplifying those relevant to a target user. Alfa achieves the lowest average gaze errors across four cross-dataset gaze benchmarks, outperforming existing TTP methods and low-rank adaptation (LoRA)-based variants. We also show that Alfa's attentive low-rank methods can be applied to applications beyond vision, such as diffusion-based language models.

Problem

• gaze estimation
• test-time personalization
• domain adaptation
• parameter-efficient fine-tuning
• cross-domain

Innovation

• Attentive Low-Rank Adaptation
• Test-Time Personalization
• Filter Reweighting
• Singular Value Decomposition
• Parameter-Efficient Fine-Tuning