Towards A Litmus Test for Common Sense

2025-01-17

AI Summary

Current AI systems lack robust commonsense reasoning capabilities, particularly exhibiting deceptive hallucinations when encountering novel situations—posing significant risks to safety and alignment. Method: We introduce the first axiomatic commonsense evaluation benchmark, integrating Minimal Prior Knowledge (MPK) constraints with a Gödelian diagonalization argument to generate unforeseeable tasks that transcend the model’s existing conceptual repertoire. Our framework synergizes ARC-style abstract reasoning, LLM behavioral observation, and embodied cognitive modeling to yield a scalable, diagnostic “touchstone” test paradigm for commonsense competence. Contribution: We are the first to formalize deceptive hallucination as a core risk indicator of commonsense deficiency, uncovering a potential paradox wherein improved performance correlates with degraded safety. Furthermore, we establish a theoretical framework for the decidability of commonsense reasoning, providing a verifiable, diagnosable foundation for AI safety and alignment assessment.

Technology Category

• Knowledge Representation and Reasoning: Common-Sense Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

This paper is the second in a planned series aimed at envisioning a path to safe and beneficial artificial intelligence. Building on the conceptual insights of"Common Sense Is All You Need,"we propose a more formal litmus test for common sense, adopting an axiomatic approach that combines minimal prior knowledge (MPK) constraints with diagonal or Godel-style arguments to create tasks beyond the agent's known concept set. We discuss how this approach applies to the Abstraction and Reasoning Corpus (ARC), acknowledging training/test data constraints, physical or virtual embodiment, and large language models (LLMs). We also integrate observations regarding emergent deceptive hallucinations, in which more capable AI systems may intentionally fabricate plausible yet misleading outputs to disguise knowledge gaps. The overarching theme is that scaling AI without ensuring common sense risks intensifying such deceptive tendencies, thereby undermining safety and trust. Aligning with the broader goal of developing beneficial AI without causing harm, our axiomatic litmus test not only diagnoses whether an AI can handle truly novel concepts but also provides a stepping stone toward an ethical, reliable foundation for future safe, beneficial, and aligned artificial intelligence.

Problem

• Artificial Intelligence
• Common Sense
• Ethical Operation

Innovation

• Commonsense Testing
• Abstract Reasoning
• AI Safety and Ethics