Palace: A Library for Interactive GPU-Accelerated Large Tensor Processing and Visualization

2025-09-30

AI Summary

Large-scale multidimensional tensor data (2D–4D) in scientific imaging and simulation increasingly challenge existing tools, which struggle to simultaneously deliver high performance, interactivity, and out-of-core processing capability. To address this, we propose TensorPipe—a GPU-accelerated, out-of-core–capable general-purpose tensor processing library. Its core innovations are a high-performance asynchronous concurrent execution architecture and a lightweight computational graph programming interface. This design enables real-time construction and execution of tensor pipelines on commodity workstations while ensuring cross-platform compatibility and usability. Experimental evaluation on representative scientific visualization tasks—including volume rendering and hierarchical random-walk segmentation—demonstrates that TensorPipe achieves performance on par with or exceeding state-of-the-art systems. These results validate its generality and practicality for multidimensional scientific data analysis.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Computer Vision: 3D Computer Vision
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Data transparency and provenance

Abstract

Tensor datasets (two-, three-, or higher-dimensional) are fundamental to many scientific fields utilizing imaging or simulation technologies. Advances in these methods have led to ever-increasing data sizes and, consequently, interest and development of out-of-core processing and visualization techniques, although mostly as specialized solutions. Here we present Palace, an open-source, cross-platform, general-purpose library for interactive and accelerated out-of-core tensor processing and visualization. Through a high-performance asynchronous concurrent architecture and a simple compute-graph interface, Palace enables the interactive development of out-of-core pipelines on workstation hardware. We demonstrate on benchmarks that Palace outperforms or matches state-of-the-art systems for volume rendering and hierarchical random-walker segmentation and demonstrate applicability in use cases involving tensors from 2D images up to 4D time series datasets.

Problem

• Processing large tensor datasets beyond memory capacity
• Enabling interactive GPU-accelerated visualization of massive tensors
• Providing general-purpose out-of-core solutions for multidimensional data

Innovation

• GPU-accelerated out-of-core tensor processing library
• High-performance asynchronous concurrent architecture
• Simple compute-graph interface for interactive pipelines