Prompt Segmentation and Annotation Optimisation: Controlling LLM Behaviour via Optimised Segment-Level Annotations

2026-05-14

AI Summary

Existing prompt optimization methods suffer from inefficiency in vast, unstructured search spaces and often deviate from the original intent. This work proposes the PSAO framework, which introduces, for the first time, a structured prompt optimization mechanism based on segment-level annotations. The approach decomposes prompts into interpretable segments and augments them with human-readable importance labels (e.g., {important}, {unimportant}) to guide large language models in allocating attention appropriately and resolving ambiguities during inference. By preserving the semantic content of the original prompt, PSAO significantly enhances response controllability, reasoning accuracy, and self-consistency, thereby demonstrating the effectiveness and potential of structured prompt optimization.

Abstract

Prompt engineering is crucial for effective interaction with generative artificial intelligence systems, yet existing optimisation methods often operate over an unstructured and vast prompt space, leading to high computational costs and potential distortions of the original intent. We introduce Prompt Segmentation and Annotation Optimisation (PSAO), a structured prompt optimisation framework designed to improve prompt optimisation controllability and efficiency. PSAO decomposes a prompt into interpretable segments (e.g., sentences) and augments each with human-readable annotations (e.g., {not important}, {important}, {very important}). These annotations guide large language models (LLMs) in allocating focus and clarifying confusion during response generation. We formally define the segmentations and annotations and demonstrate that optimised segment-level annotations can lead to improved LLM responses, with the original prompt retained as a candidate in the optimisation space to prevent performance degradation. Empirical evaluations indicate that PSAO benefits from annotations in terms of improved reasoning accuracy and self-consistency. However, developing efficient methods for identifying optimal segmentations and annotations remains challenging and is reserved for future investigation. This work is intended as a proof of concept, demonstrating the feasibility and potential of segment-level annotation optimisation.

Problem

• prompt optimisation
• large language models
• prompt engineering
• annotation
• controllability

Innovation

• Prompt Segmentation
• Annotation Optimisation
• Large Language Models
• Controllable Prompt Engineering
• Segment-Level Guidance