Motion-Aware Reinforcement Learning For Object Localization

2026-06-19

AI Summary

This work addresses the challenges of overshooting and instability in bounding box refinement for object detection, particularly the lack of motion consistency in temporal scenarios. To this end, the authors propose MARLNet, a proximal policy optimization (PPO)-based reinforcement learning framework for refinement. MARLNet incorporates a constant-velocity motion prior into its state representation and introduces an action smoothness penalty in the reward function, effectively decoupling the conventional IoU-based reward from physical deviation penalties to prevent reward collapse. The study also reveals an inherent representational ceiling in refinement strategies relying on cropped features. Experimental results demonstrate that MARLNet improves detection success rates by 0.011 and 0.007 (at IoU ≥ 0.5) on Pascal VOC 2012 and VisDrone 2019, respectively, while achieving stable training and significantly suppressing overshoot artifacts.

Abstract

We present MARLNet (Motion-Aware Reinforcement Learning Network), a PPO-based bounding-box refinement agent that incorporates a constant-velocity motion prior into the observation state and an action smoothness penalty into the reward function. The agent operates on 268-dimensional observations encoding the current proposal, a kinematic prediction, the previous action, and a 256-dimensional EfficientNet-B0 crop feature, and learns a five-dimensional policy controlling coordinate adjustments and a binary termination trigger. Evaluated on Pascal VOC 2012 and VisDrone 2019, MARLNet trains stably across all regularization strengths tested and achieves consistent gains in detection success rate at $\text{IoU} \geq 0.5$: up to $+0.011$ on VOC ($λ_\text{phys}{=}0.10$), where the motion prior prevents the overshooting that causes plain PPO to regress on this metric, and $+0.007$ on VisDrone ($λ_\text{phys}{=}0.70$), where unconstrained PPO achieves a larger gain ($+0.025$) owing to the weaker base detector. Through reward design ablations and training dynamics analysis, we identify a reward interference in which combining a constant-velocity deviation penalty with an absolute IoU term causes trigger collapse, and show that replacing it with the action smoothness penalty resolves this failure. We further characterize a representational ceiling facing crop-feature refinement agents that share a backbone with their base detector, confirmed through a global-plus-local observation ablation. Project page: https://prithviraj97.github.io/marl-net

Problem

• object localization
• reinforcement learning
• bounding-box refinement
• motion prior
• reward design

Innovation

• motion-aware reinforcement learning
• bounding-box refinement
• action smoothness penalty
• constant-velocity prior
• trigger collapse