Measuring Human Contribution in AI-Assisted Content Generation

2024-08-27

arXiv.org

AI Summary

The widespread adoption of generative AI has rendered human contribution in human-AI co-creation difficult to define and quantify. To address this challenge, we propose the first information-theoretic framework for measuring human contribution: specifically, the ratio of mutual information between human input and model output to the self-information of the model output—termed the “Human Contribution Ratio.” This metric ensures cross-task and cross-domain comparability and interpretability. Our approach integrates information theory (mutual information, self-information), human-AI interaction content modeling, and empirical analysis across diverse domains. Evaluations on creative tasks—including text generation and image editing—demonstrate its ability to robustly distinguish varying levels of human intervention. Experimental results confirm the metric’s validity, robustness, and generalizability. The framework provides both a theoretical foundation and a practical tool for assessing originality, adjudicating copyright attribution, and optimizing human-AI collaboration in AI-assisted content creation.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Machine Learning: Information Theory

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

With the growing prevalence of generative artificial intelligence (AI), an increasing amount of content is no longer exclusively generated by humans but by generative AI models with human guidance. This shift presents notable challenges for the delineation of originality due to the varying degrees of human contribution in AI-assisted works. This study raises the research question of measuring human contribution in AI-assisted content generation and introduces a framework to address this question that is grounded in information theory. By calculating mutual information between human input and AI-assisted output relative to self-information of AI-assisted output, we quantify the proportional information contribution of humans in content generation. Our experimental results demonstrate that the proposed measure effectively discriminates between varying degrees of human contribution across multiple creative domains. We hope that this work lays a foundation for measuring human contributions in AI-assisted content generation in the era of generative AI.

Problem

• Quantify human contribution in AI-generated content
• Develop framework using information theory
• Measure originality in AI-assisted creative domains

Innovation

• quantifies human contribution
• uses information theory
• measures mutual information