Sketch-based Access Control: A Multimodal Interface for Translating User Preferences into Intent-Aligned Policies

2026-05-11

AI Summary

This work addresses the limitations of traditional access control policy authoring interfaces, which are often complex and insufficiently expressive to accurately capture users’ imprecise permission intents. The paper proposes a sketch-based, AI-assisted access control system that integrates hand-drawn sketches with multimodal large language models (MLLMs) through a human-in-the-loop Specify-Analyze-Test workflow. This iterative process enables users to express, refine, and validate policies collaboratively with the system. Leveraging MLLMs, the approach interprets sketches into evolvable policy representations and incorporates conversational clarification and scenario-based testing mechanisms. User studies demonstrate that the method effectively guides users in transforming vague preferences into precise and complete security policies, significantly identifying policy gaps, resolving ambiguities, and verifying behavioral correctness.

Abstract

Developing simple and expressive access controls -- interfaces to specify policies that define who should have access to resources and under what circumstances -- is a longstanding challenge in usable security. We present Sketch-based Access Control (SBAC), a sketch-based, AI-assisted access control authoring system that combines the expressive power of sketching with the interpretive capabilities of multimodal large language models (MLLMs) to support the interpretation and validation of policy specifications as they are iteratively refined. Through a formative study with 14 participants, we identified three design requirements and developed a human-AI collaborative workflow composed of three stages -- Specify, Analyze, and Test -- enabled by the system's ability to maintain and interpret evolving access control specifications. In a user evaluation with 14 participants grounded in their real-world access control scenarios, we found the system and the workflow helped participants progressively refine initially underspecified preferences into more complete and precise policies -- surfacing gaps they had not anticipated, resolving ambiguities through dialogue, and validating policy behavior through concrete scenarios.

Problem

• access control
• usable security
• policy specification
• user preferences
• intent alignment

Innovation

• Sketch-based Access Control
• Multimodal Large Language Models
• Human-AI Collaboration
• Policy Specification
• Usable Security