Towards Group Fairness with Multiple Sensitive Attributes in Federated Foundation Models

2025-06-23

AI Summary

This work addresses the challenge of group fairness across multiple sensitive attributes (e.g., gender, race, age) in Federated Foundation Models (FFMs). To this end, we propose the first interpretable fair modeling paradigm that integrates causal discovery and causal inference. Our method constructs a causal graph among sensitive attributes to quantify latent dependencies and confounding effects, and jointly optimizes multi-dimensional fairness constraints within the federated learning framework. Innovatively, we embed structural causal models directly into the FFM training pipeline, enabling coordinated optimization of fairness objectives across clients and attributes. Experiments on sensitive domains—particularly healthcare—demonstrate significant improvements in subgroup-level predictive fairness (average gain of 23.6%) while providing human-interpretable causal attribution paths. This work establishes both theoretical foundations and a practical blueprint for building trustworthy, fair, and explainable federated foundation models.

Technology Category

• Machine Learning: Causal Learning
• Computer Vision: Bias, Fairness & Privacy
• Reasoning under Uncertainty: Causality

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

The deep integration of foundation models (FM) with federated learning (FL) enhances personalization and scalability for diverse downstream tasks, making it crucial in sensitive domains like healthcare. Achieving group fairness has become an increasingly prominent issue in the era of federated foundation models (FFMs), since biases in sensitive attributes might lead to inequitable treatment for under-represented demographic groups. Existing studies mostly focus on achieving fairness with respect to a single sensitive attribute. This renders them unable to provide clear interpretability of dependencies among multiple sensitive attributes which is required to achieve group fairness. Our paper takes the first attempt towards a causal analysis of the relationship between group fairness across various sensitive attributes in the FFM. We extend the FFM structure to trade off multiple sensitive attributes simultaneously and quantify the causal effect behind the group fairness through causal discovery and inference. Extensive experiments validate its effectiveness, offering insights into interpretability towards building trustworthy and fair FFM systems.

Problem

• Achieving group fairness in federated foundation models with multiple sensitive attributes
• Analyzing causal relationships among multiple sensitive attributes for fairness
• Extending FFM structure to balance and quantify fairness trade-offs

Innovation

• Causal analysis of group fairness in FFMs
• Extend FFM structure for multiple sensitive attributes
• Quantify causal effect via discovery and inference