TivTok: Broadcasting Time-Invariant Tokens for Scalable Video Tokenization

2026-06-16

AI Summary

This work addresses the computational redundancy and limited scalability of existing video tokenization methods, which repeatedly encode temporally invariant content—such as static backgrounds—in long videos. To overcome this, the authors propose TivTok, the first framework to explicitly separate static and dynamic information during tokenization through a dual-path mechanism comprising temporally invariant (TIV) and temporally variant (TV) tokens, enabling reuse across both frames and video segments. Leveraging Scope-Induced Factorization attention and an Invariant Broadcasting decoding strategy, TivTok achieves highly efficient parallel reconstruction. On the standard 16×256×256 benchmark, it attains an rFVD of 12.65 using only 1.1% of the token count required by a baseline downsampled tokenizer, yielding a 2.91× improvement in compression efficiency.

Abstract

Video tokenization is fundamental to scalable video generation, as the number of tokens directly determines the computational cost and the length of videos that can be modeled. Existing tokenizers mainly improve scalability by compressing videos into fewer tokens, but they often continue to represent persistent content, such as static backgrounds and consistent object appearances, repeatedly across frames and chunks. In this paper, we propose \textbf{TivTok} (\textit{Time-Invariant Tokenizer}), a reuse-aware video tokenizer that makes persistent information reusable across time. TivTok represents a clip with Time-Invariant (TIV) tokens that encode information shared across frames and Time-Variant (TV) tokens that encode frame-specific residuals. To obtain this factorization, we introduce Scope-Induced Factorization (SIF), which assigns different attention scopes to the two token groups: TIV tokens attend to the full clip, whereas each TV token only accesses its corresponding frame together with the TIV tokens. In the decoder, Invariant Broadcasting (IB) reuses the same TIV tokens across frames and chunks for parallel reconstruction and long-video tokenization. Experiments show that TivTok achieves an rFVD of 12.65 on the standard $16{\times}256{\times}256$ benchmark and improves compression efficiency by 2.91$\times$ for 128-frame videos compared with the evaluated baselines, while using only 1.1\% of the tokens required by downsample-based tokenizers in our evaluation.

Problem

• video tokenization
• scalable video generation
• time-invariant content
• token redundancy
• computational efficiency

Innovation

• Time-Invariant Tokens
• Video Tokenization
• Scope-Induced Factorization
• Invariant Broadcasting
• Scalable Video Generation