Fake or Real: The Impostor Hunt in Texts for Space Operations

2025-07-17

AI Summary

To address security risks—specifically, data poisoning or adversarial manipulation of large language models (LLMs) leading to spurious outputs in space-domain AI applications—this paper proposes the first LLM output authenticity verification framework tailored to aerospace mission scenarios. Methodologically, it integrates textual semantic features, statistical anomaly patterns, and adversarial robustness modeling to construct a supervised binary classifier capable of distinguishing legitimate outputs from manipulated ones with high accuracy. Key contributions include: (1) formal definition and modeling of space-mission-specific textual credibility criteria; (2) open-sourcing a benchmark dataset and evaluation protocol covering diverse aerospace text genres; and (3) empirical validation on a Kaggle competition, delivering a reusable, end-to-end text forensics toolkit. The framework significantly enhances the poisoning resilience and operational robustness of safety-critical AI systems in space applications.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The "Fake or Real" competition hosted on Kaggle (href{https://www.kaggle.com/competitions/fake-or-real-the-impostor-hunt}{https://www.kaggle.com/competitions/fake-or-real-the-impostor-hunt}) is the second part of a series of follow-up competitions and hackathons related to the "Assurance for Space Domain AI Applications" project funded by the European Space Agency (href{https://assurance-ai.space-codev.org/}{https://assurance-ai.space-codev.org/}). The competition idea is based on two real-life AI security threats identified within the project -- data poisoning and overreliance in Large Language Models. The task is to distinguish between the proper output from LLM and the output generated under malicious modification of the LLM. As this problem was not extensively researched, participants are required to develop new techniques to address this issue or adjust already existing ones to this problem's statement.

Problem

• Detect malicious LLM outputs in space operations
• Address data poisoning in AI applications
• Develop techniques to identify fake text

Innovation

• Detect malicious LLM output modifications
• Address data poisoning in AI models
• Develop new techniques for AI security