CCLSTM: Coupled Convolutional Long-Short Term Memory Network for Occupancy Flow Forecasting

2025-06-06

AI Summary

Accurate future state prediction of dynamic agents in autonomous driving heavily relies on high-quality vectorized inputs and computationally intensive Transformer architectures. Method: This paper proposes an end-to-end lightweight motion modeling approach based on Occupancy Flow Fields (OFF), eliminating both vectorization preprocessing and self-attention mechanisms. It introduces a novel, purely convolutional coupled LSTM architecture that jointly models spatiotemporal dependencies via trainable convolutional recurrence. The method adopts a direct occupancy flow field encoding–decoding paradigm, drastically reducing computational overhead and deployment cost. Contribution/Results: Evaluated on the 2024 Waymo Occupancy Flow Prediction Challenge, our method achieves state-of-the-art (SOTA) performance, ranking first across all official metrics.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Learning to Search
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Predicting future states of dynamic agents is a fundamental task in autonomous driving. An expressive representation for this purpose is Occupancy Flow Fields, which provide a scalable and unified format for modeling motion, spatial extent, and multi-modal future distributions. While recent methods have achieved strong results using this representation, they often depend on high-quality vectorized inputs, which are unavailable or difficult to generate in practice, and the use of transformer-based architectures, which are computationally intensive and costly to deploy. To address these issues, we propose extbf{Coupled Convolutional LSTM (CCLSTM)}, a lightweight, end-to-end trainable architecture based solely on convolutional operations. Without relying on vectorized inputs or self-attention mechanisms, CCLSTM effectively captures temporal dynamics and spatial occupancy-flow correlations using a compact recurrent convolutional structure. Despite its simplicity, CCLSTM achieves state-of-the-art performance on occupancy flow metrics and, as of this submission, ranks (1^{ ext{st}}) in all metrics on the 2024 Waymo Occupancy and Flow Prediction Challenge leaderboard.

Problem

• Predict future states of dynamic agents in autonomous driving
• Overcome reliance on high-quality vectorized inputs in occupancy flow prediction
• Reduce computational cost of transformer-based architectures for flow forecasting

Innovation

• Lightweight convolutional LSTM for occupancy flow
• End-to-end trainable without vectorized inputs
• Compact recurrent structure captures spatiotemporal dynamics