As numerous recent examples have shown, executing unknown binaries carries inherent risks; even those originating from seemingly trustworthy sources can, in fact, contain malicious code. For reverse engineers, determining the presence of such malicious elements within software poses significant challenges. This talk aims to address these challenges by discussing a range of strategies designed to extract potential malicious behaviour from complex binaries.

Initially, our presentation outlines common methods for identifying malicious behaviour, such as signature-based checks, string analysis, identification of suspicious API calls, and packer detection. However, in recent years, more sophisticated malware has often evaded detection by these traditional strategies. To address this, we introduce various techniques and heuristics for analysing and navigating more sophisticated binaries.

Throughout the talk, we examine the advantages and disadvantages of these heuristics, along with their potential applications. By employing these strategies, we tackle various use cases, such as identifying state machines, command and control (C&C) server communication, and string decryption routines in malware. Furthermore, we delve into the detection of API functions in statically linked executables, detection of obfuscated code, and pinpointing cryptographic algorithms.

About the speaker

Tim Blazytko

Chief Scientist and Co-Founder at emproof

Tim Blazytko is a well-known binary security researcher and co-founder of emproof. After working on novel methods for code deobfuscation, fuzzing and root cause analysis during his PhD, Tim now builds code obfuscation schemes tailored to embedded devices. Moreover, he gives trainings on reverse engineering & code deobfuscation, analyses malware and performs security audits.
Read more …