A Systematic Review of Hybrid Deep Learning and Explainable Artificial Intelligence Approaches for Polymorphic Malware Detection

Abstract

Polymorphic malware remains difficult to detect because it alters its observable code structure while preserving malicious functionality, thereby weakening signature-based, static-only and single-modality learning systems. Objective: This paper systematically reviews recent literature on hybrid deep learning, multimodal feature representation, malware datasets and explainable artificial intelligence for polymorphic malware detection. Methods: A structured review protocol was applied to peer-reviewed and high-quality scholarly sources on malware detection, deep learning, explainability, datasets, obfuscation, polymorphism and metamorphism, with emphasis on works from 2020 to 2026 and foundational studies where necessary. Studies were grouped thematically according to detection approach, feature modality, dataset support, robustness and interpretability. Findings: The review shows that convolutional models are effective for byte and image representations, recurrent and gated models support API and opcode sequence learning, and multimodal fusion improves coverage against representation-changing malware. However, most studies remain limited by single-modality designs, weak polymorphic dataset validation, insufficient cross-dataset testing and limited explanation of feature contribution. Conclusion: The literature supports a research direction that combines hybrid CNN-LSTM-GRU modelling, polymorphism-aware dataset construction, ablation analysis and multi-method XAI. The review provides a structured research agenda for developing robust and interpretable malware detection systems suitable for evolving polymorphic threats.

DOI: https://doi.org/10.5281/zenodo.21777609