Problem
Research questions and friction points this paper is trying to address.
program termination
source code
termination analysis
non-terminating programs
program correctness
Transformers for Program Termination
🤖 AI Summary
Termination analysis is a fundamental challenge in program analysis, with profound implications for correctness, verification, and security. This work proposes a novel approach that leverages an ensemble of small Transformer encoders to directly learn termination patterns from source code. To address the inherent class imbalance in termination datasets, the method introduces an imbalance-aware loss function combined with a class-aware sampling strategy. Furthermore, a syntax-aware attribution mechanism is designed to enhance model interpretability by linking predictions to syntactic structures in the code. Experimental results demonstrate that the proposed ensemble significantly outperforms individual Transformers, general-purpose large language models, and graph neural networks, achieving state-of-the-art performance on program termination classification tasks.
📝 Abstract
Determining whether a program terminates is a core challenge in program analysis with direct implications for correctness, verification, and security. We investigate whether transformer architectures can recognise termination patterns directly from source code and how their strengths can be amplified through ensembles. To overcome the extreme scarcity of non-terminating examples, we design an ensemble framework of compact transformer encoders, systematically trained with a suite of imbalance-aware loss functions and class-aware sampling techniques. By combining models trained with distinct loss functions, our ensembles achieve substantially stronger performance than any single transformer, outperforming both powerful off-the-shelf LLMs and graph-based methods. Finally, we introduce an attribution pipeline that produces syntax-aware explanations for the termination estimation.
Innovation
Methods, ideas, or system contributions that make the work stand out.
program termination
transformer ensemble
class-imbalanced learning
syntax-aware attribution
source code analysis
🔎 Similar Papers
No similar papers found.
