🤖 AI Summary
To address the excessive memory overhead of the D-basis algorithm in attribute implication analysis, this paper proposes Small Space, a space-efficient hypergraph dualization method. The approach abandons full rule storage and instead adopts a backward-search-based hypergraph dualization framework, dynamically and incrementally counting the frequency of antecedents for each consequent attribute while retaining only essential intermediate states. By tightly integrating the structural properties of the D-basis with hypergraph dualization theory—and focusing specifically on antecedent frequency as the key statistic—it achieves substantial memory reduction. Experimental results demonstrate that Small Space reduces peak memory consumption by up to several orders of magnitude, while maintaining nearly unchanged runtime performance. This enables scalable attribute implication analysis on large-scale datasets.
📝 Abstract
We present a new implementation of the $D$-basis algorithm called the Small Space which considerably reduces the algorithm's memory usage for data analysis applications. The previous implementation delivers the complete set of implications that hold on the set of attributes of an input binary table. In the new version, the only output is the frequencies of attributes that appear in the antecedents of implications from the $D$-basis, with a fixed consequent attribute. Such frequencies, rather than the implications themselves, became the primary focus in analysis of datasets where the $D$-basis has been applied over the last decade. The $D$-basis employs a hypergraph dualization algorithm, and a dualization implementation known as Reverse Search allows for the gradual computation of frequencies without the need for storing all discovered implications. We demonstrate the effectiveness of the Small Space implementation by comparing the runtimes and maximum memory usage of this new version with the current implementation.