A Personalized Exercise Assistant using Reinforcement Learning (PEARL): Results from a four-arm Randomized-controlled Trial

2025-08-12

AI Summary

Insufficient global physical activity poses a major public health challenge, yet existing mHealth interventions struggle to simultaneously achieve personalization, theory-driven design, and scalability. This study introduces the first reinforcement learning (RL)-guided, behaviorally informed just-in-time adaptive intervention (JITAI) that dynamically optimizes both the content and timing of health prompts. In a large-scale, four-arm randomized controlled trial (N=155), we provide the first empirical evidence of long-term efficacy for RL-driven digital interventions: the RL group exhibited a statistically significant increase of 296 daily steps over controls at one month (p<0.01), with sustained effects at two months—and outperformed both random and fixed-strategy arms. Our key innovation lies in embedding 155 empirically validated behavioral principles into the RL framework, implemented via Fitbit and analyzed using generalized estimating equations—enabling theoretically grounded, data-driven, and scalable personalized nudges.

Technology Category

• Machine Learning: Reinforcement Learning
• Intelligent Robots: Behavior Learning & Control
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Consistent physical inactivity poses a major global health challenge. Mobile health (mHealth) interventions, particularly Just-in-Time Adaptive Interventions (JITAIs), offer a promising avenue for scalable, personalized physical activity (PA) promotion. However, developing and evaluating such interventions at scale, while integrating robust behavioral science, presents methodological hurdles. The PEARL study was the first large-scale, four-arm randomized controlled trial to assess a reinforcement learning (RL) algorithm, informed by health behavior change theory, to personalize the content and timing of PA nudges via a Fitbit app. We enrolled and randomized 13,463 Fitbit users into four study arms: control, random, fixed, and RL. The control arm received no nudges. The other three arms received nudges from a bank of 155 nudges based on behavioral science principles. The random arm received nudges selected at random. The fixed arm received nudges based on a pre-set logic from survey responses about PA barriers. The RL group received nudges selected by an adaptive RL algorithm. We included 7,711 participants in primary analyses (mean age 42.1, 86.3% female, baseline steps 5,618.2). We observed an increase in PA for the RL group compared to all other groups from baseline to 1 and 2 months. The RL group had significantly increased average daily step count at 1 month compared to all other groups: control (+296 steps, p=0.0002), random (+218 steps, p=0.005), and fixed (+238 steps, p=0.002). At 2 months, the RL group sustained a significant increase compared to the control group (+210 steps, p=0.0122). Generalized estimating equation models also revealed a sustained increase in daily steps in the RL group vs. control (+208 steps, p=0.002). These findings demonstrate the potential of a scalable, behaviorally-informed RL approach to personalize digital health interventions for PA.

Problem

• Addressing global physical inactivity via personalized mobile health interventions
• Developing scalable reinforcement learning for adaptive exercise nudges
• Evaluating behavioral theory-based algorithms for activity promotion

Innovation

• Reinforcement learning personalizes exercise nudges
• Behavioral theory informs adaptive intervention timing
• Large-scale trial validates RL for health promotion