Catching Lies Without Sending the Video: Privacy-Preserving Multimodal Deception Detection

2026-06-21

AI Summary

This work proposes a privacy-preserving multimodal lie detection approach that achieves high accuracy without uploading raw audiovisual data. Instead of transmitting original media, the method extracts compact semantic summaries—such as transcribed text, emotion and intent distributions, and facial action units—directly on the user device. These distilled representations are then processed by a lightweight classifier in conjunction with a large language model (e.g., Claude Opus) for inference. Experimental results demonstrate that the proposed method attains an AUC of 0.755 under speaker-independent settings, matching the performance of Gemini 2.5 Pro while reducing input token count by 7.8×. This study provides the first evidence that state-of-the-art multimodal deception detection can be achieved without access to raw audio or video inputs.

Abstract

Frontier multimodal models can guess whether a person is lying from a testimony video. To do so, they stream that raw face and voice to a third-party model. We ask whether the heavy media is needed at all. On the Real-life Trial Deception dataset, Whissle on-device speech and vision stack extracts a compact digest: transcript, emotion, age, gender, intent distributions, a deception intent filter, fluency and rhythm, per-frame facial behaviour, and prosody. Under speaker-independent evaluation, we report three findings. A small classifier on this digest reaches AUC 0.741, matching Gemini 2.5 Pro on full video. Handing the digest to a frontier LLM reaches AUC 0.755 with Claude Opus 4.8 at 7.8X fewer input tokens, with no media leaving the device. The reported 75% accuracy is a speaker-leakage artifact. We release code and experiments.

Problem

• privacy-preserving
• multimodal deception detection
• on-device processing
• speaker-independent evaluation
• compact digest

Innovation

• privacy-preserving
• on-device processing
• multimodal deception detection
• compact feature digest
• speaker-independent evaluation