PainFormer: a Vision Foundation Model for Automatic Pain Assessment

2025-05-02

AI Summary

To address the clinical challenge of automatic, continuous, and robust pain intensity assessment, this paper introduces Pain-VFM—the first vision foundation model specifically designed for pain evaluation. Pain-VFM is pre-trained via multi-task learning on 14 diverse datasets comprising 10.9 million samples, supporting multimodal inputs including RGB, synthetic thermal imaging, depth videos, and physiological signals (ECG, EMG, GSR, fNIRS). It features a novel pain-specialized Vision Transformer architecture and an Embedding-Mixer module for cross-modal fusion, enabling general-purpose representation learning and plug-and-play task adaptation. Evaluated on BioVid and AI4Pain benchmarks, Pain-VFM outperforms all 73 baseline methods across both unimodal and multimodal settings, achieving state-of-the-art performance. Its design significantly enhances generalization across heterogeneous clinical populations and robustness under real-world deployment conditions.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Pain is a manifold condition that impacts a significant percentage of the population. Accurate and reliable pain evaluation for the people suffering is crucial to developing effective and advanced pain management protocols. Automatic pain assessment systems provide continuous monitoring and support decision-making processes, ultimately aiming to alleviate distress and prevent functionality decline. This study introduces PainFormer, a vision foundation model based on multi-task learning principles trained simultaneously on 14 tasks/datasets with a total of 10.9 million samples. Functioning as an embedding extractor for various input modalities, the foundation model provides feature representations to the Embedding-Mixer, a transformer-based module that performs the final pain assessment. Extensive experiments employing behavioral modalities-including RGB, synthetic thermal, and estimated depth videos-and physiological modalities such as ECG, EMG, GSR, and fNIRS revealed that PainFormer effectively extracts high-quality embeddings from diverse input modalities. The proposed framework is evaluated on two pain datasets, BioVid and AI4Pain, and directly compared to 73 different methodologies documented in the literature. Experiments conducted in unimodal and multimodal settings demonstrate state-of-the-art performances across modalities and pave the way toward general-purpose models for automatic pain assessment.

Problem

• Develops automatic pain assessment for effective management
• Integrates multi-modal data for comprehensive pain evaluation
• Achieves state-of-the-art performance across diverse datasets

Innovation

• Multi-task learning with 14 tasks/datasets
• Transformer-based Embedding-Mixer for assessment
• Handles diverse behavioral and physiological inputs