SHAN: Object-Level Privacy Detection via Inference on Scene Heterogeneous Graph

2024-03-14

arXiv.org

AI Summary

Existing privacy detection methods are largely limited to image-level classification, suffering from low localization accuracy, poor cross-scene generalization, weak interpretability, and lack of fine-grained annotations. To address these limitations, this paper introduces the first object-level privacy detection paradigm. We propose a Scene Heterogeneous Graph (SHG) to explicitly model semantic relationships among objects and their contextual surroundings, and design the Scene Heterogeneous Graph Attention Network (SHAN), which jointly integrates graph self-attention, multimodal scene reasoning, and weakly supervised label modeling. To support this line of research, we release OPD-1K and OPD-5K—the first two benchmark datasets for object-level privacy detection. Extensive experiments demonstrate that our method significantly outperforms existing baselines in precise object localization, cross-scene generalization, and decision interpretability, establishing a novel paradigm and foundational infrastructure for fine-grained identification of privacy-sensitive content.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Bias, Fairness & Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

With the rise of social platforms, protecting privacy has become an important issue. Privacy object detection aims to accurately locate private objects in images. It is the foundation of safeguarding individuals' privacy rights and ensuring responsible data handling practices in the digital age. Since privacy of object is not shift-invariant, the essence of the privacy object detection task is inferring object privacy based on scene information. However, privacy object detection has long been studied as a subproblem of common object detection tasks. Therefore, existing methods suffer from serious deficiencies in accuracy, generalization, and interpretability. Moreover, creating large-scale privacy datasets is difficult due to legal constraints and existing privacy datasets lack label granularity. The granularity of existing privacy detection methods remains limited to the image level. To address the above two issues, we introduce two benchmark datasets for object-level privacy detection and propose SHAN, Scene Heterogeneous graph Attention Network, a model constructs a scene heterogeneous graph from an image and utilizes self-attention mechanisms for scene inference to obtain object privacy. Through experiments, we demonstrated that SHAN performs excellently in privacy object detection tasks, with all metrics surpassing those of the baseline model.

Problem

• Detecting private objects in images using scene context inference
• Addressing accuracy and generalization issues in privacy object detection
• Creating object-level privacy detection benchmarks and models

Innovation

• Constructs scene heterogeneous graph from images
• Uses self-attention mechanisms for scene inference
• Performs object-level privacy detection via graph analysis