Text Style Transfer with Parameter-efficient LLM Finetuning and Round-trip Translation

2026-02-16

AI Summary

This work addresses the challenge of scarce parallel corpora in text style transfer by proposing a novel approach that operates without authentic parallel data. The method leverages back-translation to generate neutral-style texts as shared input representations and integrates parameter-efficient fine-tuning (PEFT) with retrieval-augmented generation (RAG) to enhance terminological consistency and stylistic control. Evaluated across four domains, the proposed framework significantly outperforms zero-shot prompting and few-shot in-context learning (ICL), achieving state-of-the-art performance in both BLEU scores and style accuracy. This advancement circumvents the traditional reliance on manually annotated parallel corpora, offering a scalable and effective solution for style transfer tasks under low-resource conditions.

Technology Category

• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

This paper proposes a novel method for Text Style Transfer (TST) based on parameter-efficient fine-tuning of Large Language Models (LLMs). Addressing the scarcity of parallel corpora that map between styles, the study employs roundtrip translation to synthesize such parallel datasets from monolingual corpora. This approach creates'neutralized'text devoid of stylistic attributes, essentially creating a shared input style at training-time and inference-time. Experimental results demonstrate consistent superiority of this method over zero-shot prompting and fewshot ICL techniques measured by BLEU scores and style accuracy scores across four investigated domains. Furthermore, the integration of retrieval-augmented generation (RAG) for terminology and name knowledge enhances robustness and stylistic consistency.

Problem

• Text Style Transfer
• parallel corpora
• style mapping
• data scarcity

Innovation

• Parameter-efficient fine-tuning
• Round-trip translation
• Text Style Transfer
• Retrieval-augmented generation
• Neutralized text