Variational Autoencoder Domain Adaptation for Cross-System Generalization in ML-Based SOP Monitoring

2026-04-20

AI Summary

This study addresses the significant degradation in generalization performance of existing machine learning models across different optical fiber systems due to variations in wavelength, fiber characteristics, and network architectures. To tackle this challenge, the work proposes a domain-adaptive framework leveraging shared representation learning, introducing variational autoencoders (VAEs) for the first time into polarization state (SOP) monitoring in optical communications. The approach jointly trains on data from two distinct systems to extract common event representations; after freezing the shared encoder, only system-specific classifiers are fine-tuned, effectively disentangling domain-invariant and domain-specific features. Experimental results demonstrate substantial improvements in cross-system accuracy—reaching 95.3% and 73.5% for transfers from System 1 to 2 and vice versa—outperforming single-system supervised DNN baselines by 83.4% and 51%, respectively, while maintaining excellent in-domain performance.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Learning & Optimization for CV
• Application Domains: Internet of Things, Sensor Networks & Smart Cities

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Machine learning (ML) models trained to detect physical-layer threats on one optical fiber system often fail catastrophically when applied to a different system, due to variations in operating wavelength, fiber properties, and network architecture. To overcome this, we propose a Domain Adaptation (DA) framework based on a Variational Autoencoder (VAE) that learns a shared representation capturing event signatures common to both systems while suppressing system-specific differences. The shared encoder is first trained on the combined data from two distinct optical systems: a 21 km O-band dark-fiber testbed (System 1) and a 63.4 km C-band live metro ring (System 2). The encoder is then frozen, and a classifier is trained using labels from an individual system. The proposed approach achieves 95.3% and 73.5% cross-system accuracy when moving from System 1 to System 2 and vice versa, respectively. This corresponds to gains of 83.4% and 51% over a fully supervised Deep Neural Network (DNN) baseline trained on a single system, while preserving intra-system performance.

Problem

• Domain Adaptation
• Cross-System Generalization
• Optical Fiber Monitoring
• Machine Learning
• Physical-Layer Threat Detection

Innovation

• Variational Autoencoder
• Domain Adaptation
• Cross-System Generalization
• Optical Fiber Monitoring
• Shared Representation Learning