Selective Embedding for Deep Learning

2025-07-16

AI Summary

Deep learning models suffer from poor generalization on non-stationary time-series data and across domains, alongside high computational overhead. To address this, we propose Selective Embedding—a novel data loading strategy that alternates multiple short, multi-source time-series segments within a single input channel, emulating human selective attention mechanisms. Unlike conventional approaches, Selective Embedding requires no architectural modifications and is fully compatible with mainstream deep learning frameworks. It enhances robustness against overfitting, accelerates training, and improves cross-domain adaptability. Evaluated across six real-world time-series benchmarks spanning healthcare, industrial monitoring, and agriculture, our method achieves average classification accuracy gains of 3.2–9.7%, reduces training time by 38–61%, and demonstrates strong robustness—confirming its practical deployability in resource-constrained, heterogeneous temporal applications.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Search and Optimization: Learning to Search
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Deep learning has revolutionized many industries by enabling models to automatically learn complex patterns from raw data, reducing dependence on manual feature engineering. However, deep learning algorithms are sensitive to input data, and performance often deteriorates under nonstationary conditions and across dissimilar domains, especially when using time-domain data. Conventional single-channel or parallel multi-source data loading strategies either limit generalization or increase computational costs. This study introduces selective embedding, a novel data loading strategy, which alternates short segments of data from multiple sources within a single input channel. Drawing inspiration from cognitive psychology, selective embedding mimics human-like information processing to reduce model overfitting, enhance generalization, and improve computational efficiency. Validation is conducted using six time-domain datasets, demonstrating that the proposed method consistently achieves high classification accuracy across various deep learning architectures while significantly reducing training times. The approach proves particularly effective for complex systems with multiple data sources, offering a scalable and resource-efficient solution for real-world applications in healthcare, heavy machinery, marine, railway, and agriculture, where robustness and adaptability are critical.

Problem

• Improves deep learning robustness under nonstationary conditions
• Reduces overfitting and enhances generalization across domains
• Optimizes computational efficiency in multi-source data processing

Innovation

• Alternates data segments from multiple sources
• Mimics human-like information processing
• Reduces overfitting and enhances generalization