Precision Physical Activity Prescription via Reinforcement Learning for Functional Actions

2026-05-18

AI Summary

This study addresses the lack of personalized approaches in current physical activity recommendations for optimizing daily step-count patterns based on individual health metrics. Leveraging longitudinal step-count data and repeatedly measured health biomarkers from the All of Us Research Program, this work proposes a novel offline reinforcement learning framework that treats the entire daily step-count distribution as a functional action. It introduces, for the first time, a functional action space into offline reinforcement learning and develops a new algorithm capable of effectively handling continuous function-valued actions. In simulation experiments, the proposed method outperforms existing continuous-action reinforcement learning approaches. Policies learned from real-world data not only recommend higher average daily steps and more stable activity patterns but also yield precise, individualized prescriptions tailored to subgroups defined by glucose levels, BMI, blood pressure, age, and sex.

Abstract

Physical activity (PA) plays an important role in maintaining and improving health. Daily steps have been a key PA measure that is easily accessible with common wearable devices. However, methods are lacking to recommend a personalized optimal distribution of daily steps over a period of time for the best of certain health biomarkers. In this paper, we fill this void based on the data from the All of Us Research Program which includes months of step counts as well as repeated measurements of key health biomarkers. We develop a new offline reinforcement learning (RL) algorithm to learn personalized and optimal PA distributions associated with cardiometabolic risk, where the action is a function representing the daily step distribution over a period of time. Simulation studies demonstrate the advantage of the proposed approach over existing continuous-action RL methods. The learned optimal policy from the All of Us data generally suggests people take more daily steps and also follow a more consistent pattern of PA over time while offering tailored recommendations for subgroups in blood glucose level, body mass index, blood pressure, age, and sex.

Problem

• Physical Activity Prescription
• Personalized Recommendation
• Daily Step Distribution
• Health Biomarkers
• Cardiometabolic Risk

Innovation

• offline reinforcement learning
• personalized physical activity prescription
• step distribution optimization
• cardiometabolic risk
• wearable data