High-Performance Star-M SVD for Big Data Compression

2026-05-15

AI Summary

This work addresses the challenge of efficiently compressing large-scale datasets in the era of big data while supporting high-precision complex mathematical operations. Building upon the star-M tensor framework, the paper proposes a high-performance parallel tensor singular value decomposition (SVD) algorithm tailored for shared-memory architectures. It presents the first system-level language implementation of parallelized star-M SVD, overcoming the performance limitations inherent in prior approaches confined to productivity-oriented languages. By integrating tensor decomposition with advanced high-performance numerical computing techniques, the method substantially enhances both compression efficiency and reconstruction accuracy on scientific datasets, thereby establishing a robust foundation for downstream data analysis and insight extraction.

Abstract

In the era of big data, effectively compressing large datasets while performing complex mathematical operations is crucial. Tensor-based decomposition methods have shown superior compression capabilities with minimal loss of accuracy compared to traditional matrix methods. Under the star-M tensor framework, tensors can be decomposed in a matrix-mimetic way, including using the star-M SVD. This tensor SVD has optimality guarantees and has shown exceptional performance on specific types of data, but software implementations have been mostly limited to productivity-oriented languages. In this work, we present our development of a shared-memory parallel, high-performance solution designed to efficiently implement the underlying algorithms. This software will enable optimal compression of extensive scientific datasets, paving the way for enhanced data analysis and insights.

Problem

• big data compression
• tensor decomposition
• star-M SVD
• high-performance computing
• scientific datasets

Innovation

• star-M SVD
• tensor decomposition
• high-performance computing
• parallel algorithm
• big data compression