CrowdAL: Towards a Blockchain-empowered Active Learning System in Crowd Data Labeling

2024-09-16

IEEE International Conference on e-Science

AI Summary

To address critical challenges in crowdsourced labeling—including difficulty achieving consensus, worker privacy leakage, and unfair incentive allocation—this paper proposes a blockchain-enhanced active learning framework. Methodologically, it pioneers the integration of zero-knowledge proofs (ZKPs) into smart contracts to enable verifiable label quality while revealing neither raw labels nor sensitive worker attributes; it further designs an on-chain dynamic incentive mechanism that jointly optimizes labeling diversity and reliability. The key contributions are: (i) the first active learning system supporting privacy-preserving consensus aggregation, and (ii) a design that simultaneously ensures verifiability, fairness, and efficiency. Experiments on real-world crowdsourcing tasks demonstrate a 32% improvement in labeling consistency, near-zero privacy leakage risk, and significantly superior incentive fairness compared to baseline approaches.

Technology Category

• Machine Learning: Active Learning
• Humans and AI: Crowd Sourcing and Human Computation
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Security and Privacy: Privacy-enhancing technologies

Abstract

Active Learning (AL) is a machine learning technique where the model selectively queries the most informative data points for labeling by human experts. Integrating AL with crowdsourcing leverages crowd diversity to enhance data labeling but introduces challenges in consensus and privacy. This poster presents CrowdAL, a blockchain-empowered crowd AL system designed to address these challenges. CrowdAL integrates blockchain for transparency and a tamper-proof incentive mechanism, using smart contracts to evaluate crowd workers’ performance and aggregate labeling results, and employs zeroknowledge proofs to protect worker privacy.

Problem

• Enhances data labeling using Active Learning and crowdsourcing
• Addresses consensus and privacy challenges in crowd labeling
• Uses blockchain for transparency and privacy protection

Innovation

• Blockchain ensures transparent, tamper-proof incentive mechanisms.
• Smart contracts evaluate and aggregate crowd worker performance.
• Zero-knowledge proofs protect worker privacy effectively.