L2CU: Learning to Complement Unseen Users

2026-01-03

IEEE Access

AI Summary

Existing human-in-the-loop learning methods often rely on a single global user model, which struggles to generalize to unseen users and fails to account for individual differences, thereby limiting performance. This work proposes L2CU, a novel framework that enables personalized human-AI complementary learning for previously unseen users. L2CU constructs a library of representative user profiles by clustering annotation patterns from known users, then matches new users to their nearest profile and leverages the corresponding profile-specific complementary model for collaboration. By departing from the conventional global modeling paradigm, L2CU supports model-agnostic collaborative enhancement and significantly improves human-AI joint classification accuracy across multiple benchmarks, including CIFAR-10N, CIFAR-10H, Fashion-MNIST-H, Chaoyang, and AgNews.

Technology Category

• Humans and AI: Human-in-the-loop Machine Learning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Machine Learning: Multimodal Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Recent research highlights the potential of machine learning models to learn to complement (L2C) human strengths; however, generalizing this capability to unseen users remains a significant challenge. Existing L2C methods oversimplify interaction between human and AI by relying on a single, global user model that neglects individual user variability, leading to suboptimal cooperative performance. Addressing this, we introduce L2CU, a novel L2C framework for human-AI cooperative classification with unseen users. Given sparse and noisy user annotations, L2CU identifies representative annotator profiles capturing distinct labeling patterns. By matching unseen users to these profiles, L2CU leverages profile-specific models to complement the user and achieve superior joint accuracy. We evaluate L2CU on datasets (CIFAR-10N, CIFAR-10H, Fashion-MNIST-H, Chaoyang and AgNews), demonstrating its effectiveness as a model-agnostic solution for improving human-AI cooperative classification.

Problem

• human-AI collaboration
• unseen users
• user variability
• cooperative classification
• learning to complement

Innovation

• Learning to Complement
• Unseen Users
• Annotator Profiles
• Human-AI Collaboration
• Model-Agnostic