Multi-Round Human-AI Collaboration with User-Specified Requirements

2026-02-19

AI Summary

This work addresses the central challenge in high-stakes, multi-turn human-AI collaboration: balancing the avoidance of counterfactual harm—defined as not undermining human strengths—with achieving complementarity by compensating for human error-prone behaviors. The paper proposes the first user-centered collaborative framework that enables users to formalize these dual objectives as executable constraints through customizable rules. It introduces an online algorithm that dynamically satisfies these constraints during interaction without requiring explicit modeling of human behavior or assumptions about its distribution. Theoretical analysis provides finite-sample guarantees, and experiments on medical diagnosis and visual reasoning tasks demonstrate that the method reliably controls constraint violation rates while predictably improving human decision accuracy through adjustable constraint strength.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

As humans increasingly rely on multiround conversational AI for high stakes decisions, principled frameworks are needed to ensure such interactions reliably improve decision quality. We adopt a human centric view governed by two principles: counterfactual harm, ensuring the AI does not undermine human strengths, and complementarity, ensuring it adds value where the human is prone to err. We formalize these concepts via user defined rules, allowing users to specify exactly what harm and complementarity mean for their specific task. We then introduce an online, distribution free algorithm with finite sample guarantees that enforces the user-specified constraints over the collaboration dynamics. We evaluate our framework across two interactive settings: LLM simulated collaboration on a medical diagnostic task and a human crowdsourcing study on a pictorial reasoning task. We show that our online procedure maintains prescribed counterfactual harm and complementarity violation rates even under nonstationary interaction dynamics. Moreover, tightening or loosening these constraints produces predictable shifts in downstream human accuracy, confirming that the two principles serve as practical levers for steering multi-round collaboration toward better decision quality without the need to model or constrain human behavior.

Problem

• counterfactual harm
• complementarity
• human-AI collaboration
• multi-round interaction
• decision quality

Innovation

• counterfactual harm
• complementarity
• user-specified constraints
• online algorithm
• human-AI collaboration