Materiality and Risk in the Age of Pervasive AI Sensors

2024-02-17

Nature Machine Intelligence

AI Summary

This paper addresses the overlooked risks arising from sensor hardware layers in large-scale AI deployments. It introduces the first “sensor-sensitive” risk diagnostic framework, positioning materiality as a core dimension of AI governance. Drawing on technical philosophy, interdisciplinary risk modeling, and comparative policy analysis (e.g., NIST AI RMF, EU AI Act), the study systematically identifies cross-dimensional risks—including privacy, environmental impact, and autonomy erosion—emerging from interactions between sensor physical characteristics and algorithmic models. Key contributions include: (1) bridging critical theoretical and practical gaps in existing AI risk management frameworks regarding hardware-layer considerations; and (2) developing an actionable sensor design paradigm that advances equitable, transparent, and accountable edge AI through user empowerment and community engagement. The framework provides concrete guidance for integrating sensor-level awareness into AI governance and responsible deployment. (136 words)

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Humans and AI: Brain-Sensing and Analysis
• Machine Learning: Hardware-aware ML

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Security and Privacy: Security and privacy of machine learning and AI applications
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems

Abstract

Artificial intelligence (AI) systems connected to sensor-laden devices are becoming pervasive, which has notable implications for a range of AI risks, including to privacy, the environment, autonomy and more. There is therefore a growing need for increased accountability around the responsible development and deployment of these technologies. Here we highlight the dimensions of risk associated with AI systems that arise from the material affordances of sensors and their underlying calculative models. We propose a sensor-sensitive framework for diagnosing these risks, complementing existing approaches such as the US National Institute of Standards and Technology AI Risk Management Framework and the European Union AI Act, and discuss its implementation. We conclude by advocating for increased attention to the materiality of algorithmic systems, and of on-device AI sensors in particular, and highlight the need for development of a sensor design paradigm that empowers users and communities and leads to a future of increased fairness, accountability and transparency.

Problem

• Address risks from pervasive AI sensors in privacy, environment, autonomy
• Develop accountability in responsible AI sensor deployment and development
• Create sensor-sensitive framework to diagnose and manage AI risks

Innovation

• Sensor-sensitive framework for AI risk diagnosis
• Complementing NIST and EU AI risk standards
• On-device AI sensor design empowering users