RAGthoven at SemEval-2026 Task 1: A Multi-Stage Pipeline Walks Into a Benchmark and Barely Clears the Bar

2026-07-14

AI Summary

This work addresses the challenge of generating constrained humorous text across multiple languages—English, Spanish, and Chinese—by proposing a multi-stage generation framework that integrates computational humor theories with large language models (LLMs). The approach innovatively incorporates retrieval-augmented generation (RAG) during the planning phase, synergistically combining benign violation theory and script-based semantic theory. It implements an agent pipeline comprising a planner, a multi-candidate generator, a self-reflection module, and an LLM-based judge, exploring both ReAct and multi-branch autonomous orchestration architectures. Evaluated on SemEval-2026 Task 1, the system tied for first place with Gemini 2.5 Flash, outperformed Spanish-language baselines by 42 Elo points, and showed no significant performance gap in English and Chinese, thereby demonstrating both efficacy and strong cross-lingual adaptability.

Abstract

We present RAGthoven, our system for SemEval-2026 Task 1 (MWAHAHA), Subtask A (multilingual constrained humor generation in English, Spanish, and Chinese). RAGthoven decomposes creative text generation into a multi-stage large language model (LLM) pipeline (Planner, Best-of-N Writer, Reflector for self-critique, LLM-as-a-judge Judge) grounded in computational humor theory (Benign Violation Theory, Script-based Semantic Theory of Humor) and refined across ten experiments. In our final configuration, we augment the Planner with retrieval-augmented generation (RAG) from a curated joke corpus, seeding generation with diverse joke mechanisms. We also evaluate two agentic variants -- ReAct-style sequential tool-calling (Exp09) and autonomous multi-branch orchestration (Exp10) -- that expose the same four stages with a deterministic ConstraintAudit checker. Across four frontier models on a held-out 12-instance English sample, neither agentic variant produced outputs we judged superior to the non-agentic pipeline despite substantially higher tool-call budgets. RAGthoven shares Rank 1 with the Gemini 2.5 Flash baseline in all three languages, with overlapping organizer-reported confidence intervals. In Spanish, it leads the baseline by 42 raw Elo points (1182 vs. 1140), while in English (1045 vs. 1081) and Chinese (1045 vs. 1053) the baseline holds the higher raw rating within the same statistical tie. Together, these results suggest language-dependent diminishing returns from elaborate multi-stage prompt engineering and agentic scaffolding once a strong frontier model is in the loop.

Problem

• multilingual humor generation
• constrained text generation
• computational humor
• large language models
• SemEval

Innovation

• retrieval-augmented generation
• multi-stage LLM pipeline
• computational humor theory
• agentic scaffolding
• constrained text generation