mlr3torch: A Deep Learning Framework in R based on mlr3 and torch

2026-04-20

AI Summary

This work addresses the lack of a deep learning framework in R that seamlessly integrates with mainstream machine learning ecosystems and supports end-to-end workflows from preprocessing to training and evaluation. By leveraging mlr3 and torch, the authors present the first scalable deep learning framework in R that unifies data preprocessing, augmentation, neural architecture definition, and multimodal modeling through the mlr3pipelines graphical modeling paradigm. The framework accommodates both tabular data and general tensors—such as images—for classification and regression tasks, enables custom network architectures, and allows seamless conversion into mlr3 learners, thereby enhancing modularity and reusability. Empirical evaluations demonstrate its efficiency and flexibility in scenarios including hyperparameter optimization and model fine-tuning.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations

Abstract

Deep learning (DL) has become a cornerstone of modern machine learning (ML) praxis. We introduce the R package mlr3torch, which is an extensible DL framework for the mlr3 ecosystem. It is built upon the torch package, and simplifies the definition, training, and evaluation of neural networks for both tabular data and generic tensors (e.g., images) for classification and regression. The package implements predefined architectures, and torch models can easily be converted to mlr3 learners. It also allows users to define neural networks as graphs. This representation is based on the graph language defined in mlr3pipelines and allows users to define the entire modeling workflow, including preprocessing, data augmentation, and network architecture, in a single graph. Through its integration into the mlr3 ecosystem, the package allows for convenient resampling, benchmarking, preprocessing, and more. We explain the package's design and features and show how to customize and extend it to new problems. Furthermore, we demonstrate the package's capabilities using three use cases, namely hyperparameter tuning, fine-tuning, and defining architectures for multimodal data. Finally, we present some runtime benchmarks.

Problem

• deep learning
• R package
• mlr3
• torch
• neural networks

Innovation

• mlr3torch
• deep learning
• graph-based workflow
• torch integration
• multimodal architecture