ForeSight: Enhancing Risk Monitoring via Early Safety Signal Distillation

2026-09-12

AI Summary

为解决大型语言模型生成有害内容的问题，提出ForeSight框架，通过提炼早期安全信号，实现高效的风险预测。

Abstract

As large language models (LLMs) are increasingly deployed, the generation of harmful content has become a critical safety concern. Existing safeguards operate at the input, output, or streaming-generation stages, while early-risk methods that rely on surface tokens or output logits may suffer from weak initial signals, and internals-based detectors using dense representations may retain highly entangled and redundant safety-irrelevant information. It therefore remains unclear whether the earliest post-generation hidden states already contain reliable signals about final-response harmfulness. To address this gap, we propose ForeSight, a first-token output-risk forecasting framework that distills weak and redundant early safety signals into compact, layer-aware risk representations. Experiments on five safety benchmarks and two target models demonstrate that ForeSight achieves superior and efficient early-risk forecasting while relying solely on first-token hidden states. The code is available at: https://github.com/Scabbards1500/Foresight

Problem

• large language models
• harmful content
• safety signals
• hidden states
• risk forecasting

Innovation

• Early-risk forecasting
• First-token hidden states
• Safety signal distillation
• Layer-aware risk representations