Matchmaker: An Open-source Library for Real-time Piano Score Following and Systematic Evaluation

2025-10-11

AI Summary

Current real-time piano score following research lacks a unified, open-source, and modern MIR-ecosystem-compatible evaluation framework, hindering standardized cross-method and cross-dataset comparisons. To address this, we propose the first open-source Python evaluation framework specifically designed for real-time piano score following. It supports plug-and-play integration of diverse musical representations (e.g., note sequences, frame-level features) and alignment algorithms (e.g., dynamic programming, hidden Markov models). The framework is compatible with major large-scale piano datasets—including nASAP, Batik, and Vienna4x22—and provides multidimensional evaluation metrics: temporal accuracy, robustness, and latency. Experimental results on real performance data demonstrate high-precision, low-latency alignment. The framework significantly improves reproducibility and comparability across studies, establishing a standardized benchmark and practical toolkit for algorithm development and interactive music applications.

Technology Category

• Machine Learning: Evaluation and Analysis
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Evaluation and Analysis

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Real-time music alignment, also known as score following, is a fundamental MIR task with a long history and is essential for many interactive applications. Despite its importance, there has not been a unified open framework for comparing models, largely due to the inherent complexity of real-time processing and the language- or system-dependent implementations. In addition, low compatibility with the existing MIR environment has made it difficult to develop benchmarks using large datasets available in recent years. While new studies based on established methods (e.g., dynamic programming, probabilistic models) have emerged, most evaluations compare models only within the same family or on small sets of test data. This paper introduces Matchmaker, an open-source Python library for real-time music alignment that is easy to use and compatible with modern MIR libraries. Using this, we systematically compare methods along two dimensions: music representations and alignment methods. We evaluated our approach on a large test set of solo piano music from the (n)ASAP, Batik, and Vienna4x22 datasets with a comprehensive set of metrics to ensure robust assessment. Our work aims to establish a benchmark framework for score-following research while providing a practical tool that developers can easily integrate into their applications.

Problem

• Lack of unified open framework for real-time music alignment model comparison
• Difficulty benchmarking due to low MIR library compatibility issues
• Limited evaluation scope across different alignment method families

Innovation

• Open-source Python library for real-time music alignment
• Systematic comparison of music representations and alignment methods
• Evaluated on large piano datasets with comprehensive metrics