A Survey on State-of-the-art Deep Learning Applications and Challenges

2024-03-26

AI Summary

Existing deep learning surveys suffer from incomplete coverage, overemphasis on CNNs, and a lack of cross-domain comparative analysis of model efficacy. Method: We systematically survey state-of-the-art (SOTA) models across five domains—computer vision, natural language processing, time-series analysis, ubiquitous computing, and robotics—encompassing Transformer, CNN, RNN, GNN, self-supervised, and multimodal learning architectures. Contribution/Results: We propose the first unified survey framework spanning multiple domains while integrating theoretical foundations with practical problem-solving capabilities. We establish a cross-domain model capability comparison system that clarifies principled criteria for optimal model selection per scenario. Furthermore, we identify scalability, robustness, and energy efficiency as three fundamental, domain-agnostic challenges—offering concrete directions for future research. This work bridges critical gaps in both scope and analytical depth, enabling more informed architectural choices and fostering cross-disciplinary innovation.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Adversarial Attacks & Robustness
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

Deep learning, a branch of artificial intelligence, is a data-driven method that uses multiple layers of interconnected units or neurons to learn intricate patterns and representations directly from raw input data. Empowered by this learning capability, it has become a powerful tool for solving complex problems and is the core driver of many groundbreaking technologies and innovations. Building a deep learning model is challenging due to the algorithm's complexity and the dynamic nature of real-world problems. Several studies have reviewed deep learning concepts and applications. However, the studies mostly focused on the types of deep learning models and convolutional neural network architectures, offering limited coverage of the state-of-the-art deep learning models and their applications in solving complex problems across different domains. Therefore, motivated by the limitations, this study aims to comprehensively review the state-of-the-art deep learning models in computer vision, natural language processing, time series analysis and pervasive computing, and robotics. We highlight the key features of the models and their effectiveness in solving the problems within each domain. Furthermore, this study presents the fundamentals of deep learning, various deep learning model types and prominent convolutional neural network architectures. Finally, challenges and future directions in deep learning research are discussed to offer a broader perspective for future researchers.

Problem

• Review state-of-the-art deep learning models across domains.
• Highlight key features and effectiveness of deep learning models.
• Discuss challenges and future directions in deep learning research.

Innovation

• Comprehensive review of deep learning models
• Focus on computer vision and NLP applications
• Discussion of challenges and future directions