Guarding Organizations Against Malware Risk: A Novel Graph-Based Malware Detection Method

Abstract

Organizational digitalization expands cybersecurity risks, making cybersecurity an increasingly important research area in Information Systems (IS). Among these risks, malware has become a pervasive and destructive threat. Byte-based machine learning (ML) methods are widely used for malware detection but remain vulnerable to evasive behaviors that manipulate raw bytes to evade detection. Graph-based methods are less affected by such manipulations because they represent software as program graphs that capture execution behavior. However, they do not explicitly identify cohesive groups of basic blocks that jointly realize meaningful program behaviors, nor do they learn sufficiently expressive program graph representations for accurate detection. To this end, we propose MalGuard, a graph-based malware detection method for organizational malware risk management. MalGuard introduces two methodological innovations: an operational role identification approach and a program graph representation learning method. The former identifies these cohesive groups of basic blocks as operational roles, enabling the detector to capture program behaviors that may not be visible from isolated basic blocks. The latter learns expressive program graph representations by modeling interactions among operational roles, preserving sparse malicious signals, and capturing hierarchical graph structure. Extensive experiments show that MalGuard improves detection performance and reduces the expected cost of undetected malware.