Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

2026-06-24

AI Summary

This work addresses the vulnerability of large language models in mathematical reasoning, where a single critical token—termed a “cliff token”—can abruptly derail an entire reasoning trajectory. Existing approaches struggle to precisely identify such failure-inducing tokens. To overcome this, the authors propose an adaptive thresholding mechanism combined with a one-sided two-proportion z-test, enabling, for the first time, accurate token-level detection of cliff triggers. They further establish a taxonomy of cliff tokens into three categories: deterministic, uncertainty-driven, and sampling-deviation types. Building on this framework, they introduce Cliff-DPO, a targeted fine-tuning method applied at specific positions to mitigate cliff effects. Experiments show that removing the first detected cliff token and resampling restores pass@64 to 1.0, and Cliff-DPO trained on GSM8K improves accuracy by up to 6.6% across multiple benchmarks, with pronounced gains on uncertainty- and sampling-deviation-related cliffs.

Abstract

Large language models (LLMs) reach high accuracy in mathematical reasoning, but individual traces on the same problem diverge; some arrive at the correct answer while others fail. Prior work analyzes failure at the step, chunk, or sentence level, or at tokens where failure has already occurred. Neither identifies the precise token that triggers the shift toward failure. We introduce the cliff token, a token where the token-wise potential drops significantly under an adaptive threshold that scales with the local token-wise potential, based on a one-sided two-proportion z-test. Across seven models and three mathematical reasoning benchmarks (GSM1K, MATH500, AIME 2025), cliff tokens act as failure triggers; deleting the first cliff token and resampling recovers pass@64 to 1.0, while keeping it limits recovery to between 0.71 and 1.00. We further introduce a cliff taxonomy of deterministic, uncertain, and sampled-off cliffs, defined by greedy choice and token entropy. Each type has distinct probabilistic characteristics, and the taxonomy generalizes across model scales. Finally, we validate the taxonomy via single-token preference optimization at cliff positions (Cliff-DPO). Trained on GSM8K, Cliff-DPO improves accuracy across benchmarks by up to +6.6. Optimizing at uncertain and sampled-off cliffs improves reasoning, while deterministic cliffs do not.

Problem

• cliff token
• mathematical reasoning
• failure trigger
• large language models
• token-level analysis

Innovation

• cliff token
• mathematical reasoning
• token-wise potential
• preference optimization
• failure trigger