Fairness-Aware Fine-Tuning of Vision-Language Models for Medical Glaucoma Diagnosis

2025-12-03

AI Summary

To address fairness disparities in medical vision-language models—specifically, accuracy variations across demographic groups in glaucoma diagnosis—this paper proposes a fairness-aware low-rank fine-tuning framework. Methodologically, we introduce a differentiable MaxAccGap loss function to enable end-to-end optimization for accuracy parity across subgroups. We further propose two novel parameter-efficient architectures: GR-LoRA and Hybrid-LoRA, which integrate inverse-frequency gradient weighting with low-rank adaptation, requiring only 0.24% trainable parameters. Evaluated on a dataset of 10,000 fundus images, GR-LoRA reduces diagnostic accuracy disparity by 69% while achieving an overall accuracy of 53.15%. These results demonstrate substantial improvements in model fairness and deployment feasibility, offering a lightweight, equitable AI solution for primary-care glaucoma screening.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Hardware-aware ML
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Vision-language models achieve expert-level performance on medical imaging tasks but exhibit significant diagnostic accuracy disparities across demographic groups. We introduce fairness-aware Low-Rank Adaptation for medical VLMs, combining parameter efficiency with explicit fairness optimization. Our key algorithmic contribution is a differentiable MaxAccGap loss that enables end-to-end optimization of accuracy parity across demographic groups. We propose three methods: FR-LoRA integrates MaxAccGap regularization into the training objective, GR-LoRA applies inverse frequency weighting to balance gradient contributions, and Hybrid-LoRA combines both mechanisms.Evaluated on 10,000 glaucoma fundus images, GR-LoRA reduces diagnostic accuracy disparities by 69% while maintaining 53.15% overall accuracy. Ablation studies reveal that strong regularization strength achieves optimal fairness with minimal accuracy trade-off, and race-specific optimization yields 60% disparity reduction. Our approach requires only 0.24% trainable parameters, enabling practical deployment of fair medical AI in resource-constrained healthcare settings.

Problem

• Address diagnostic accuracy disparities across demographic groups in medical imaging.
• Optimize fairness and accuracy simultaneously in vision-language models for glaucoma diagnosis.
• Enable practical deployment of fair medical AI with minimal trainable parameters.

Innovation

• Fairness-aware Low-Rank Adaptation for medical vision-language models
• Differentiable MaxAccGap loss optimizes accuracy parity across groups
• Requires only 0.24% trainable parameters for resource-efficient deployment