Velocity Prediction in Automatic Guitar Transcription

2026-06-19

AI Summary

This work addresses the long-standing neglect of note-wise dynamics prediction in automatic guitar transcription, primarily hindered by the scarcity of real-world recordings with precise dynamic annotations and the ambiguous definition of dynamics for non-piano instruments. To overcome these limitations, the study introduces a novel pretraining and transfer learning framework specifically designed for guitar transcription. It leverages a virtual instrument to generate synthetic audio with accurate dynamic labels, which is used to pretrain a multi-task deep neural network. The pretrained model is subsequently fine-tuned on real guitar recordings. The proposed approach significantly outperforms non-pretrained baselines on synthetic data and achieves state-of-the-art performance on real-world guitar transcription benchmarks. Notably, it enables, for the first time, high-accuracy dynamics prediction in practical scenarios, effectively circumventing the bottleneck imposed by the absence of dynamic annotations in real data.

Abstract

Automatic Music Transcription (AMT) models have achieved a high level of success in polyphonic transcription of various instruments. Velocity, typically a measure of note intensity, is less commonly predicted in these models due to the absence of velocity labels in available datasets and lack of a proper definition for instruments other than piano. We present a methodology and model for velocity prediction in Automatic Guitar Transcription (AGT) which uses virtual instruments to generate synthetic training data with velocity labels. We first pretrain a model on this synthetic data. These weights are then transferred to a different model and trained on real guitar audio, allowing the model to retain the working velocity prediction while also achieving high performance and generalisability from the real training data. The velocity prediction is shown to outperform a baseline model which does not use the pretrained velocity weights, when evaluated on synthetic data. In addition, using the pretrained velocity weights offers a small improvement in note transcription, though the magnitude of this improvement is limited and not always significant depending on the testing data. Overall the model achieves results comparable to the state of the art in guitar transcription, while also successfully predicting velocity.

Problem

• velocity prediction
• Automatic Guitar Transcription
• Automatic Music Transcription
• synthetic training data
• note intensity

Innovation

• velocity prediction
• automatic guitar transcription
• synthetic data
• transfer learning
• virtual instruments