Enhancing Burmese News Classification with Kolmogorov-Arnold Network Head Fine-tuning

2025-11-26

AI Summary

In low-resource Burmese news classification, freezing pretrained encoders while fine-tuning only MLP classification heads suffers from limited representational capacity and suboptimal computational efficiency. Method: This work introduces Kolmogorov–Arnold Networks (KANs) to this task for the first time, replacing fixed nonlinearities with learnable one-dimensional activation functions to enhance classifier expressivity. We systematically evaluate three KAN variants—FourierKAN, EfficientKAN, and FasterKAN—combined with TF-IDF, fastText, and mBERT embeddings. Results: EfficientKAN with fastText achieves the highest F1 score (0.928); FasterKAN offers the best trade-off between accuracy and inference speed; Transformer-based KANs match or slightly surpass traditional MLPs in performance. This study establishes a novel, lightweight, expressive, and efficient classification head paradigm for low-resource NLP tasks.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Knowledge Representation and Reasoning: Knowledge Representation Languages

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In low-resource languages like Burmese, classification tasks often fine-tune only the final classification layer, keeping pre-trained encoder weights frozen. While Multi-Layer Perceptrons (MLPs) are commonly used, their fixed non-linearity can limit expressiveness and increase computational cost. This work explores Kolmogorov-Arnold Networks (KANs) as alternative classification heads, evaluating Fourier-based FourierKAN, Spline-based EfficientKAN, and Grid-based FasterKAN-across diverse embeddings including TF-IDF, fastText, and multilingual transformers (mBERT, Distil-mBERT). Experimental results show that KAN-based heads are competitive with or superior to MLPs. EfficientKAN with fastText achieved the highest F1-score (0.928), while FasterKAN offered the best trade-off between speed and accuracy. On transformer embeddings, EfficientKAN matched or slightly outperformed MLPs with mBERT (0.917 F1). These findings highlight KANs as expressive, efficient alternatives to MLPs for low-resource language classification.

Problem

• Improving Burmese news classification using Kolmogorov-Arnold Networks as alternative heads
• Addressing limited expressiveness and high computational costs of traditional MLPs
• Evaluating KAN variants for low-resource language classification across diverse embeddings

Innovation

• KANs replace MLPs as classification heads
• Evaluated FourierKAN, EfficientKAN, and FasterKAN variants
• Applied KANs with TF-IDF, fastText and transformer embeddings