Entropy of Ukrainian

2026-04-30

AI Summary

This study addresses a gap in information-theoretic research on Ukrainian by applying Shannon’s 1951 human prediction experiment methodology to this underexplored language. Recruiting 184 volunteers via crowdsourcing, the authors conducted an online character-level prediction task to estimate an upper bound on the language’s entropy. The experiment yielded an entropy upper bound of approximately 1.201 bits per character. The work presents the first human-prediction-based estimate of information entropy for Ukrainian and provides a fully reproducible framework, including open-sourced experimental protocols, code, and detailed challenge analyses. Furthermore, the results enable direct comparison with large language model performance, establishing a replicable paradigm for information-theoretic investigations of low-resource languages.

Abstract

In natural language processing, the entropy of a language is a measure of its unpredictability and complexity. The first study on this subject was conducted by Claude Shannon in 1951. By having participants predict the next character in a sentence, he was able to approximate the entropy of the English language. Several follow-up studies by other authors have since been conducted for English, and one for Hebrew. However, to date, Shannon's experiment has never been conducted for Ukrainian. In this paper, we perform this experiment for Ukrainian by recruiting 184 volunteers using social media channels. We rely on techniques used for English to approximate the entropy value of Ukrainian. The final result is an upper bound of $H_{upper}\approx1.201$ bits per character. We compare this to the performance of current Large Language Models. The methods and code used are also documented and published, along with a discussion of the main challenges encountered.

Problem

• language entropy
• Ukrainian
• Shannon experiment
• natural language processing
• predictability

Innovation

• language entropy
• Ukrainian language
• Shannon entropy estimation
• human prediction experiment
• large language models