SupGRPO: Enhancing GRPO with Matching-based Online SFT for Text Spotting

2026-09-07

AI Summary

研究通过结合监督微调(SFT)和基于组相对策略优化(GRPO)的方法，提出SupGRPO，以提高文本识别与定位能力，特别是在艺术性文本上。

Abstract

Text spotting requires both accurate text recognition and precise spatial localization. Current specialised spotters excel at predicting tight bounding boxes in natural scenes, but falter on complex or artistic text, whereas multimodal large language models (MLLMs) possess strong recognition capabilities yet remain weak at localisation. To equip the text spotter with general and powerful recognition capabilities and to maximize its localization ability, we explore two MLLM-based fine-tuning methods: Supervised Fine-Tuning (SFT) and reinforcement learning fine-tuning based on Group Relative Policy Optimisation (GRPO). An interesting finding is that SFT is less effective than GRPO at enhancing recognition, while GRPO is less effective than SFT at enhancing detection. To compensate for each other's shortcomings, we introduce a joint training strategy, SupGRPO, which simultaneously optimizes the model using both SFT and GRPO. SupGRPO employs the specially designed reward functions and develops a matching-based online SFT applied solely to coordinate tokens. It both mitigates the reward sparsity problem of GRPO and avoids the instance order dependency problem of SFT. To evaluate particularly challenging cases, we curate ATS, a dataset for artistic text spotting. Experiments demonstrate that SupGRPO improves both text recognition and detection, and attains superior performance. Our code and dataset will be released at https://github.com/Psycho-9/SupGRPO.

Problem

• Text Spotting
• Localization
• Recognition
• MLLMs
• Fine-Tuning

Innovation

• SupGRPO
• Matching-based Online SFT
• Reward Functions