numpy

Python’s scientific computing ecosystem has long lacked high-performance sparse linear algebra libraries optimized for modern CPU and GPU architectures. To address this gap, we introduce Ginkgo-Python: a lightweight, Pythonic sparse computing framework that natively wraps the high-performance Ginkgo C++ backend via Pybind11. It is the first Python library to provide unified sparse linear algebra support across CUDA, HIP, and OpenMP execution backends. Ginkgo-Python offers NumPy- and PyTorch-compatible APIs and fully supports sparse matrix–vector multiplication (SpMV) as well as mainstream iterative solvers. Experimental evaluation demonstrates that Ginkgo-Python significantly outperforms SciPy, CuPy, PyTorch, and TensorFlow in sparse matrix operations and iterative solver performance across diverse hardware platforms—while closely matching the performance of the native Ginkgo C++ library. This work bridges a critical gap in Python’s high-performance scientific computing stack for sparse linear algebra.

Key computational kernels in the Python tensor library pyttb—such as accumulation, tensor multiplication, and low-rank decomposition—suffer from suboptimal runtime performance. Method: This work introduces, for the first time, a systematic integration of Rust to develop high-performance Python extension modules via the Python C API, leveraging Rust’s memory safety and zero-cost abstractions to optimize deeply nested loops and large-scale tensor operations. Contribution/Results: Empirical evaluation on synthetic datasets across multiple problem scales demonstrates consistent and substantial speedups over pure Python, Numba JIT-compiled code, and NumPy-based implementations—reaching up to several-fold acceleration. Performance gains are especially pronounced for loop-intensive kernels. The proposed Rust–Python co-design paradigm provides a reusable, safe, and efficient optimization framework for scientific computing libraries.

Scientific software deployment faces a productivity–performance gap between high-level prototyping environments (e.g., MATLAB) and production-grade C++ code: scripting languages sacrifice performance, while conventional C++ linear algebra libraries (e.g., BLAS/LAPACK) suffer from verbose interfaces and error-prone manual memory management. This paper introduces a novel C++ linear algebra library built on template metaprogramming and expression templates. It pioneers a compile-time expression optimization framework that enables operator fusion, lazy evaluation, and elimination of temporary objects—delivering MATLAB-like syntactic simplicity without explicit memory management. The design achieves both high code readability and performance competitive with hand-optimized C++. Benchmark results demonstrate speedups of several-fold over conventional C++ implementations. The library has been successfully deployed in production-scale machine learning and signal processing systems.

This work addresses the inefficiencies of conventional GPU linear algebra libraries, which often fail to fully exploit hardware capabilities due to redundant memory accesses and runtime overhead. To overcome these limitations, the authors propose Bandicoot, a toolkit that leverages C++ template metaprogramming to perform expression fusion at compile time, thereby automatically generating highly optimized GPU kernels that saturate memory bandwidth—without relying on just-in-time compilation or runtime scheduling. Bandicoot provides an API compatible with Armadillo, facilitating straightforward migration of existing CPU codebases. Experimental results demonstrate that Bandicoot significantly outperforms PyTorch, TensorFlow, and JAX across multiple benchmarks, achieving substantial speedups in several scenarios.

This work addresses the gap between algorithmic prototypes and efficient implementations in scientific research by proposing a lightweight approach to translate statistical and machine learning algorithms—such as kernel ridge regression and stochastic gradient descent matrix factorization—from mathematical formulations into readable, high-performance C++ code. Leveraging the Eigen template library for core linear algebra operations—including kernel matrix construction, regularized solvers, and vectorized updates—the implementation seamlessly integrates into the Python ecosystem via pybind11, enabling efficient interoperability with NumPy arrays. The project provides concise, reproducible code examples that encapsulate common computational patterns in research, significantly lowering the barrier for researchers to adopt C++ for high-performance development while balancing performance, readability, and usability.

This work addresses the longstanding challenge in scientific computing of simultaneously achieving expressive array abstractions, performance portability, and memory efficiency. We propose a NumPy-compatible array computation framework built on lazy evaluation and dataflow graph compilation. Methodologically, we design a multi-level domain-specific intermediate representation (IR) integrating metadata-driven optimization, loop fusion, and GPU parallelization; we introduce the novel “array context” abstraction to unify heterogeneous execution across CPU, GPU, and OpenCL, thereby strictly decoupling application logic, optimization strategies, and hardware-specific implementations. Our contributions demonstrate significant improvements in memory efficiency and GPU computational performance for large-scale hypersonic combustion simulations (MIRGE-Com), while preserving high scientific expressivity. Empirical evaluation confirms the framework’s feasibility, scalability, and performance portability in real-world HPC deployments.

This work addresses the lack of efficient GPU-accelerated support for finite-dimensional type I von Neumann algebras—specifically, direct sums of matrix algebras—in existing computational tools. We introduce torch_vn_algebra, an open-source library built on PyTorch that enables, for the first time, batched and GPU-accelerated numerical simulations of such algebras. The library employs a compact tensor representation combined with lazy evaluation to generate random block-diagonal operators with arbitrary spectral distributions and various unitary ensembles, including those drawn from the Haar measure. It further supports functional calculus, trace-class functionals, and efficient linear algebra operations. Large-scale Monte Carlo experiments—such as 20,000 samples of 100×100 operators on a Tesla P100 GPU—demonstrate excellent performance and successfully validate theoretical predictions concerning Haar moments and trace properties.

This work addresses the limited scalability of traditional distributed FFT algorithms on heterogeneous high-performance computing platforms, which stems from static task allocation and global synchronization barriers. To overcome these limitations, the authors propose a task-graph-based dynamic scheduling approach that models each FFT stage as fine-grained tasks operating independently on distributed arrays. Leveraging the Julia programming language, the Dagger runtime, and the DTask task model, the method enables cross-device work stealing and barrier-free execution. Efficient data distribution is supported through pencil/slab-partitioned DArrays. This approach presents the first task-graph-driven distributed FFT implementation in Julia, achieving speedups of up to 2.6× on CPU clusters and 1.35× on GPU clusters. It has been successfully integrated into the Oceananigans.jl fluid simulation framework, significantly enhancing resource utilization and scalability on heterogeneous systems.

General-purpose programs often struggle to achieve efficient parallel execution on GPUs, hindering advances in program synthesis, superoptimization, and array programming. To address this limitation, this work proposes a highly parallel virtual machine architecture tailored for linear algebra computations on GPUs, enabling efficient execution of massively concurrent array programs. By integrating a parallel virtual machine, an array-program execution model, and a scalable evaluation framework, the proposed architecture achieves up to a 147× speedup over serial execution across workloads involving millions of concurrent tasks. This substantial acceleration significantly enhances GPU resource utilization and improves the efficiency of program synthesis pipelines.

This work presents the first systematic evaluation of Rust’s suitability for high-performance sparse linear algebra, addressing the longstanding trade-off between performance and memory safety in traditional scientific computing that relies on C/C++ and Fortran. The authors natively implement core operations—including sparse matrix-vector multiplication, the Lanczos Krylov method, and matrix exponential computation—leveraging compile-time monomorphization, SIMD vectorization, and careful FFI boundary analysis. Comprehensive benchmarks against established libraries such as Intel oneMKL, Eigen, PETSc, and PSBLAS demonstrate that Rust achieves performance on par with Eigen and PSBLAS in CSC format, approaching state-of-the-art levels while preserving memory safety. However, it still lags behind PETSc in block CSR optimizations, highlighting both the promise and current limitations of Rust for building efficient, safe numerical software stacks.