MonoTODia: Translating Monologue Requests to Task-Oriented Dialogues

2025-02-24

AI Summary

To address the scarcity of annotated dialogue data hindering task-oriented dialogue (TOD) systems in German-speaking enterprise settings, this paper introduces the first end-to-end framework for automatically transforming real-world, unidirectional German business emails into structured, multi-turn, trainable TOD data. Our method integrates large language model–driven instruction tuning, dialogue rewriting, and joint annotation, augmented by crowdsourced multidimensional evaluation and expert-curated gold-standard validation to ensure high fidelity. The framework enables zero-shot cross-domain adaptation, effectively bridging the modality gap between unimodal, unidirectional text and interactive, multi-turn dialogue. Experiments demonstrate that TOD models trained solely on our synthetic data achieve performance comparable to those trained on authentic human dialogues. To foster low-resource TOD research, we publicly release both the dataset and implementation code.

Technology Category

• Natural Language Processing: Conversational AI/Dialog Systems
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Data scarcity is one of the main problems when it comes to real-world applications of transformer-based models. This is especially evident for task-oriented dialogue (TOD) systems, which require specialized datasets, that are usually not readily available. This can hinder companies from adding TOD systems to their services. This study therefore investigates a novel approach to sourcing annotated dialogues from existing German monologue material. Focusing on a real-world example, we investigate whether these monologues can be transformed into dialogue formats suitable for training TOD systems. We show the approach with the concrete example of a company specializing in travel bookings via e-mail. We fine-tune state-of-the-art Large Language Models for the task of rewriting e-mails as dialogues and annotating them. To ensure the quality and validity of the generated data, we employ crowd workers to evaluate the dialogues across multiple criteria and to provide gold-standard annotations for the test dataset. We further evaluate the usefulness of the dialogues for training TOD systems. Our evaluation shows that the dialogues and annotations are of high quality and can serve as a valuable starting point for training TOD systems. Finally, we make the annotated dataset publicly available to foster future research.

Problem

• Translating monologues into task-oriented dialogues
• Addressing data scarcity for transformer-based models
• Enhancing training datasets for dialogue systems

Innovation

• Transform monologues into dialogues
• Fine-tune Large Language Models
• Crowd workers evaluate dialogue quality