Architecture of Tianyu Software: Relative Photometry as a Case Study

2025-05-14

AI Summary

The Tianyu One-Meter Survey Telescope requires high-concurrency, scalable software capable of processing real-time data streams exceeding 500 MB/s. To address this, we propose a scalable pipeline architecture tailored for astronomical real-time processing, integrating asynchronous microservices based on message queues, distributed task scheduling, and GPU-accelerated algorithms—thereby overcoming I/O-bound bottlenecks in multi-stage collaborative optimization. Evaluated on a five-node cluster, the system achieves throughput improvements of 41%, 257%, and 107% for image calibration, registration, and photometry, respectively, while photometric precision approaches the theoretical limit. The architecture has successfully enabled fully automated detection of two transiting exoplanets and eleven variable stars—including two newly discovered—demonstrating its effectiveness and advancement in large-scale time-domain surveys.

Technology Category

• Machine Learning: Scalability of ML Systems
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Algorithm Configuration

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Security and Privacy: Large-scale security measurements

Abstract

Tianyu telescope, an one-meter robotic optical survey instrument to be constructed in Lenghu, Qinghai, China, is designed for detecting transiting exoplanets, variable stars and transients. It requires a highly automated, optimally distributed, easily extendable, and highly flexible software to enable the data processing for the raw data at rates exceeding 500MB/s. In this work, we introduce the architecture of the Tianyu pipeline and use relative photometry as a case to demonstrate its high scalability and efficiency. This pipeline is tested on the data collected from Muguang observatory and Xinglong observatory. The pipeline demonstrates high scalability, with most processing stages increasing in throughput as the number of consumers grows. Compared to a single consumer, the median throughput of image calibration, alignment, and flux extraction increases by 41%, 257%, and 107% respectively when using 5 consumers, while image stacking exhibits limited scalability due to I/O constraints. In our tests, the pipeline was able to detect two transiting sources. Besides, the pipeline captures variability in the light curves of nine known and two previously unknown variable sources in the testing data. Meanwhile, the differential photometric precision of the light curves is near the theoretical limitation. These results indicate that this pipeline is suitable for detecting transiting exoplanets and variable stars. This work builds the fundation for further development of Tianyu software. Code of this work is available at https://github.com/ruiyicheng/Tianyu_pipeline.

Problem

• Designing scalable software for Tianyu telescope's high-speed data processing
• Demonstrating pipeline efficiency via relative photometry case study
• Detecting transiting exoplanets and variable stars with precision

Innovation

• Highly automated scalable pipeline for data processing
• Distributed processing boosts throughput significantly
• Near-theoretical photometric precision in light curves