Align AI to Dynamic Human-AI Workflows

2026-07-15

AI Summary

Current AI alignment approaches rely on static human preferences, which struggle to capture the dynamic, context-dependent nature of human–AI collaboration. This work proposes a paradigm shift toward “interactive complementarity,” emphasizing that preferences emerge dynamically through the co-evolution of human and AI behaviors. Introducing a trajectory-level perspective on dynamic alignment, the study integrates machine learning with social science theories and insights from interdisciplinary workshops to construct a framework for modeling human–AI interaction dynamics. This framework reveals novel forms of asymmetry and coordination challenges inherent in such systems. By establishing an alignment agenda tailored to dynamic human–AI workflows, the research lays a theoretical foundation for developing AI systems capable of interactive adaptation.

Abstract

Current alignment approaches typically focus on emulating human behavior using static representations of human preferences, failing to capture the dynamic, context-dependent nature of real-world human-AI interactions. In this paper, we argue for a shift from static and emulative to interactive and complementary alignment, where preferences emerge through interaction and alignment is defined not by satisfying preferences alone. We first formalize this gap by contrasting existing alignment with a trajectory-level view in which human and model behavior co-evolve over time. Because these interaction dynamics have not been adequately captured within existing ML formulations, we ground this perspective in insights from an interdisciplinary workshop. We draw on lessons from social-science accounts of human-human collaboration and then argue that human-AI systems amplify these dynamics, introducing new asymmetries that make reasoning about uncertainty harder and introduce new coordination challenges. Based on these lessons and new challenges, we conclude by outlining a research agenda for developing AI systems that align with humans in interaction, requiring an interdisciplinary synthesis of machine learning and the social and decision sciences.

Problem

• AI alignment
• dynamic interaction
• human-AI collaboration
• preference modeling
• coordination challenges

Innovation

• interactive alignment
• complementary AI
• trajectory-level dynamics
• human-AI collaboration
• preference co-evolution