Arachne: Orchestrating Cascades for Efficient Text-to-Video Model Training

2026-07-02

AI Summary

This work addresses the challenges of load imbalance and low hardware utilization in text-to-video model training, which stem from data heterogeneity and static parallelization strategies. To overcome these limitations, the authors propose a fine-grained cascade-based task decomposition mechanism that integrates dynamic orchestration with spatiotemporal co-optimization to efficiently schedule and synchronize compute units across distributed clusters. This approach transcends conventional bucketing and static parallelism constraints, enabling training efficiency that scales favorably with system size. Experimental results demonstrate that the method reduces iteration time by up to 65% compared to state-of-the-art frameworks, with performance gains consistently increasing as training scale expands.

Abstract

The rising demand for AI-generated videos is fueled by advances in large-scale Text-to-Video (T2V) models, trained on extensive datasets of video clips spanning diverse resolutions and durations. To address this data heterogeneity, current training methods often use a bucketing strategy that groups samples into discrete buckets for efficiency. However, this approach struggles to scale with compute and data volumes under static parallelism schemes, such as data and sequence parallelism, leading to significant workload imbalances and hardware under-utilization. In this paper, we present Arachne, a novel training framework for efficient T2V model training at scale. Arachne decomposes the training process into fine-grained computational units, called \textit{cascades}, orchestrating their distributed execution and synchronization across the cluster through coordinated spatial and temporal optimization. Our comprehensive evaluation demonstrates that Arachne reduces iteration time by up to 65\% over leading frameworks, exhibiting a positive scaling trend where its performance advantages amplify as training scale grows.

Problem

• Text-to-Video
• training efficiency
• data heterogeneity
• workload imbalance
• hardware under-utilization

Innovation

• cascades
• distributed training
• text-to-video
• heterogeneous data
• spatiotemporal optimization