🤖 AI Summary
This study addresses the scarcity of structured speech–text aligned data for the endangered Chakavian dialect, which has hindered its integration into artificial intelligence applications. We present the first high-quality, word-aligned multimodal dataset of *The Little Prince* in Chakavian, comprising synchronized text, images, and audio, meticulously curated through manual alignment and publicly released via the CLARIN.SI platform. Fine-tuning the Whisper-large-v3 model on this dataset yields substantial improvements in automatic speech recognition performance, reducing the word error rate by 50% and the character error rate by approximately two-thirds on the test set. This work establishes a reproducible data paradigm and technical framework for AI-driven preservation of endangered dialects.
📝 Abstract
This paper documents our efforts in releasing the printed and audio book of the translation of the famous novel The Little Prince into the Chakavian dialect, as a computer-readable, AI-ready dataset, with the textual and the audio components of the two releases now aligned on the level of each written and spoken word. Our motivation for working on this release is multiple. The first one is our wish to preserve the highly valuable and specific content beyond the small editions of the printed and the audio book. With the dataset published in the CLARIN.SI repository, this content is from now on at the fingertips of any interested individual. The second motivation is to make the data available for various artificial-intelligence-related usage scenarios, such as the one we follow upon inside this paper already -- adapting the Whisper-large-v3 open automatic speech recognition model, with decent performance on standard Croatian, to Chakavian dialectal speech. We can happily report that with adapting the model, the word error rate on the selected test data has being reduced to a half, while we managed to remove up to two thirds of the error on character level. We envision many more usages of this dataset beyond the set of experiments we have already performed, both on tasks of artificial intelligence research and application, as well as dialectal research. The third motivation for this release is our hope that this, now highly structured dataset, will be transformed into a digital online edition of this work, allowing individuals beyond the research and technology communities to enjoy the beauty of the message of the little boy in the desert, told through the spectacular prism of the Chakavian dialect.