From Sensors to Insight: Rapid, Edge-to-Core Application Development for Sensor-Driven Applications

2026-05-04

AI Summary

This work addresses the challenge scientists face in efficiently transforming raw sensor data streams into actionable insights across edge-cloud infrastructures, hindered by the need for cross-domain expertise to manage heterogeneous systems and emerging platforms such as DPUs, which impedes rapid prototyping. To overcome this barrier, the authors propose a novel paradigm that integrates pattern-based workflow engineering with AI-assisted development. Implemented on the FABRIC testbed using the Pegasus workflow system and exemplified by the Orcasound hydrophone workflow, this approach enables swift construction of applications for air quality, seismic, and soil moisture monitoring. The framework supports modular extensibility and edge deployment, substantially lowering the barrier for non-expert users to iteratively develop distributed applications. Empirical validation across multiple use cases demonstrates its effectiveness in enhancing development efficiency, accelerating prototyping cycles, and accumulating practical deployment experience.

Abstract

Scientists increasingly rely on sensor-based data, yet transforming raw streams into insights across the edge-to-cloud continuum remains difficult. Provisioning heterogeneous infrastructure and managing execution on emerging platforms like Data Processing Units typically requires cross-domain expertise, creating significant barriers to rapid prototyping. This paper introduces an experience-driven methodology for the rapid development of sensor-driven applications. By combining pattern-based workflow engineering with AI-assisted development-implemented via Pegasus on the FABRIC testbed - we utilize an existing Orcasound hydrophone workflow as a reusable template. We introduce a pattern-based engineering methodology to generate and refine workflows for air quality, earthquake, and soil moisture monitoring. Furthermore, we show how these abstract structures are extended to edge resources through modular configuration and placement. Our evaluation focuses on user productivity and practical lessons rather than peak performance. Through these case studies, we illustrate how AI-assisted, pattern-based development lowers the entry barrier for non-experts and enables iterative exploration of sensor-driven applications across distributed infrastructures.

Problem

• sensor-driven applications
• edge-to-cloud continuum
• heterogeneous infrastructure
• rapid prototyping
• cross-domain expertise

Innovation

• pattern-based workflow
• AI-assisted development
• edge-to-cloud continuum
• sensor-driven applications
• modular configuration