Automatic Lyric Transcription for Greek Songs: Scaling and Task Composition Effects in Whisper Adaptation

2026-09-10

AI Summary

研究通过调整Whisper模型规模、多任务训练和两阶段适应，解决希腊歌曲自动歌词转录中的旋律变化、节奏不规则和伴奏干扰问题。

Abstract

Automatic Lyric Transcription (ALT) remains substantially more challenging than speech recognition due to melodic variability, rhythmic irregularity, and accompaniment interference. This is heightened in low-resource languages like Greek, where no prior benchmark for ALT exists. We present the first controlled study of Whisper adaptation for Greek ALT, investigating model scaling effects, task composition via multitask training in transcribe-translate ratios, and two-stage speech-to-singing adaptation. We also curate a segment-level aligned singing dataset based on the Greek Audio Dataset (GAD) using source separation and CTC forced alignment. Results show that scaling consistently improves performance, while multitask learning acts as a beneficial regularizer primarily for smaller-capacity models. The 2-stage adaptation in Whisper Large-v3 achieves a Word Error Rate (WER) of 27.2%, a significant improvement over zero-shot baselines, establishing the first Greek ALT benchmark.

Problem

• Automatic Lyric Transcription
• Greek Songs
• melodic variability
• rhythmic irregularity
• accompaniment interference

Innovation

• Automatic Lyric Transcription
• Whisper Adaptation
• Multitask Learning
• Two-Stage Speech-to-Singing Adaptation