A Longitudinal Study of Usability in Identity-Based Software Signing

2026-03-17

AI Summary

This study addresses the usability challenges hindering the real-world deployment of identity-based software signing tools and their impact on integrity guarantees. Conducting the first cross-tool, longitudinal empirical analysis, the authors examine nearly 3,900 GitHub issues from five major ecosystems—including Sigstore and OpenPubKey—spanning 2021 to 2025. Through issue coding and Poisson trend analysis, they systematically characterize the types, distributions, and evolutionary patterns of usability problems. While the overall number of issues shows a declining trend, persistent friction points remain in verification workflows, policy configuration, and integration boundaries, with uneven improvements in documentation and processes. The findings reveal a shifting complexity from key management toward verification semantics and configuration, offering new directions for future tool design.

Technology Category

• Constraint Satisfaction and Optimization: Solvers and Tools
• Humans and AI: User Experience and Usability
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Security and Privacy: Data transparency and provenance
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Responsible Web: Consent frameworks and practices on the web

Abstract

Identity-based software signing tools aim to make software artifact provenance verifiable while reducing the operational burden of long-lived key management. However, there is limited cross-tool longitudinal evidence about which usability problems arise in practice and how those problems evolve as tools mature. This gap matters because unusable signing and verification workflows can lead to incomplete adoption, misconfiguration, or skipped verification, undermining intended integrity guarantees. We conducted the first mining-software-repositories study of five open-source identity-based signing ecosystems: Sigstore, OpenPubKey, HashiCorp Vault, Keyfactor, and Notary v2. We analyzed approximately 3,900 GitHub issues from Nov. 2021 to Nov. 2025. We coded each issue for the reported usability concern and the implicated architectural component, and compared patterns across tools and over time. Across ecosystems, reported concerns concentrate in verification workflows, policy and configuration surfaces, and integration boundaries. Longitudinal Poisson trend analysis shows substantial declines in reported issues for most ecosystems. However, across usability themes, workflow- and documentation-related concerns decline unevenly across tools and concern types, and verification workflows and configuration surfaces remain persistent friction points. These results indicate that identity-based signing reduces some usability burdens while relocating complexity to verification semantics, policy configuration, and deployment integration. Designing future signing ecosystems therefore requires treating verification semantics and release workflows as first-class usability targets rather than peripheral integration concerns.

Problem

• identity-based signing
• usability
• software provenance
• verification workflows
• longitudinal study

Innovation

• identity-based signing
• usability study
• longitudinal analysis
• software provenance
• verification workflows