Malformer: A Multi-Modal Malware Detector Using Transformers

2026-08-19

AI Summary

本文提出Malformer，一种结合文本、图像、图和音频四种模态的Windows恶意软件检测模型，通过多模态Transformer融合提升检测性能，解决了单一模态检测新威胁能力不足的问题。

Abstract

Traditional malware detection systems that rely on a single representation of malware often fail to identify novel threats. These representations of malware binaries, also known as modalities, do not provide the models with sufficient information to discriminate among all samples. Additionally, individual representations introduce new failure modes, with some modality extraction being dependent upon the success of disassembling. Past works have integrated either additional modalities or more discriminative representations for classification. In this work, we present Malformer, a quadrimodal malware detection model that incorporates text, image, graph, and audio representations of Windows executables. We demonstrate that multimodal transformer fusion can enhance the performance of Windows malware detectors over that of unimodal and bimodal detectors. Malformer employs a combination of two RoBERTa encoders paired with a modified Vision Transformer for image data, WavLM for audio data, and an adaptive loss-weighting scheme to fuse modality-specific representations. Evaluated on a dataset of 201,549 binary samples, Malformer achieved 98.3% accuracy and an F1 score of 0.9833, outperforming both unimodal baselines and bimodal detectors by 4.6-17.6 percentage points. Malformer demonstrates that multimodal fusion provides a promising foundation for countering the growing scale of malware threats, equipping defenders with generalized and resilient detection capabilities.

Problem

• malware detection
• single representation
• novel threats
• modalities
• failure modes

Innovation

• multimodal transformer fusion
• quadrimodal malware detection
• adaptive loss-weighting scheme