Culturally Grounded Physical Commonsense Reasoning in Italian and English: A Submission to the MRL 2025 Shared Task

2025-10-26

AI Summary

To address the scarcity of physical commonsense reasoning evaluation resources for non-English languages, this paper introduces FormaMentis—the first bilingual (Italian–English) benchmark deeply grounded in Italian linguistic and cultural specificity. Unlike generic machine translation, FormaMentis is manually curated by native Italian experts drawing on local lived experience and employs culture-preserving translation to retain culturally salient semantics while strictly adhering to the PIQA format. Its key contribution lies in the first systematic integration of physical commonsense reasoning evaluation with region-specific cultural knowledge, ensuring cultural semantic fidelity during cross-lingual transfer. The dataset has been submitted to the MRL 2025 Shared Task, providing a high-quality, culturally adapted evaluation resource for multilingual commonsense reasoning research—particularly for assessing AI systems’ physical understanding in low-resource languages.

Technology Category

• Knowledge Representation and Reasoning: Common-Sense Reasoning
• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

This paper presents our submission to the MRL 2025 Shared Task on Multilingual Physical Reasoning Datasets. The objective of the shared task is to create manually-annotated evaluation data in the physical commonsense reasoning domain, for languages other than English, following a format similar to PIQA. Our contribution, FormaMentis, is a novel benchmark for physical commonsense reasoning that is grounded in Italian language and culture. The data samples in FormaMentis are created by expert annotators who are native Italian speakers and are familiar with local customs and norms. The samples are additionally translated into English, while preserving the cultural elements unique to the Italian context.

Problem

• Creating culturally grounded physical commonsense reasoning datasets
• Developing multilingual benchmarks beyond English for evaluation
• Preserving cultural elements in Italian and English translations

Innovation

• Created culturally grounded Italian benchmark
• Used native experts for cultural annotation
• Translated data while preserving cultural elements