FoleySet: A Multi-Level Human-Annotated Foley Sound Dataset

2026-06-24

AI Summary

Existing Foley sound datasets generally suffer from insufficient quality and coarse annotations, hindering data-driven research in classification, retrieval, and synthesis. To address this gap, this work introduces and publicly releases FoleySet—a large-scale Foley dataset comprising 10,000 audio clips meticulously recorded following professional Foley practices. The dataset features a two-tier manual semantic annotation scheme that precisely aligns synchronous sound effects with on-screen human actions, such as footsteps, clothing rustles, and prop manipulations. FoleySet is the first to offer multi-level annotations, standardized formatting, and a permissive Creative Commons license, thereby filling a critical resource void in the field. It provides strong support for Foley-related audio tasks and advances research toward automated audiovisual content production.

Abstract

In audiovisual post-production, Foley refers to synchronous sound effects associated with human actions, such as footsteps, cloth rustle, and prop handling, that are recreated to match the on-screen movements and interactions of characters. These sounds are often recorded by professional Foley artists using physical props. This resource-intensive workflow has motivated data-driven research on Foley, including tasks such as classification, retrieval, and generation; however, high-quality annotated Foley datasets for training remain scarce. To address this gap, we present FoleySet, a publicly available Foley dataset of 10,000 audio clips annotated with a two-level Foley taxonomy. This dataset provides a standardized, Creative Commons-licensed resource for data-driven Foley classification, retrieval, and generation.

Problem

• Foley sound
• annotated dataset
• audiovisual post-production
• sound effects
• data scarcity

Innovation

• Foley sound
• annotated dataset
• audiovisual post-production
• two-level taxonomy
• data-driven audio generation