BEAT: Tokenizing and Generating Symbolic Music by Uniform Temporal Steps

2026-04-21

AI Summary

This work addresses the limitations of existing symbolic music tokenization approaches, which typically rely on event sequences with irregular time steps and struggle to explicitly model rhythmic regularities. The authors propose a novel tokenization method that uses fixed temporal units—such as beats—as the fundamental building blocks, merging all events sharing the same pitch within a single time step into one token, thereby yielding a sparse piano-roll-like representation. This approach enables explicit alignment of temporal structure for the first time. Integrated with a Transformer architecture, it significantly enhances generation quality, structural coherence, and long-range dependency modeling in tasks such as music continuation and accompaniment generation, while also achieving superior efficiency and rhythmic consistency compared to prevailing event-based tokenization methods.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Time-Series/Data Streams
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Tokenizing music to fit the general framework of language models is a compelling challenge, especially considering the diverse symbolic structures in which music can be represented (e.g., sequences, grids, and graphs). To date, most approaches tokenize symbolic music as sequences of musical events, such as onsets, pitches, time shifts, or compound note events. This strategy is intuitive and has proven effective in Transformer-based models, but it treats the regularity of musical time implicitly: individual tokens may span different durations, resulting in non-uniform time progression. In this paper, we instead consider whether an alternative tokenization is possible, where a uniform-length musical step (e.g., a beat) serves as the basic unit. Specifically, we encode all events within a single time step at the same pitch as one token, and group tokens explicitly by time step, which resembles a sparse encoding of a piano-roll representation. We evaluate the proposed tokenization on music continuation and accompaniment generation tasks, comparing it with mainstream event-based methods. Results show improved musical quality and structural coherence, while additional analyses confirm higher efficiency and more effective capture of long-range patterns with the proposed tokenization.

Problem

• symbolic music
• tokenization
• uniform temporal steps
• language models
• musical time representation

Innovation

• uniform temporal steps
• symbolic music tokenization
• beat-based encoding
• piano-roll representation
• long-range pattern modeling