Fighting malware and spam

Towards agile reverse engineering

Frederic Perriot Symantec

Over the last 50 years, software engineering has progressed from a black art to a formalized discipline. Teams have tackled huge software projects thanks to great strides in the domains of programming languages, software design and development methodologies, and understanding the human aspects of programming.

Compared to "forward engineering", reverse engineering is in its infancy. Today's successful reverse engineering projects are still mostly heroic individual efforts, where the experience, ingenuity and tenacity of the analyst play a prominent role. Guiding principles of rational software engineering such as team work, code reuse and testing have little or no equivalent in reverse-engineering.

As malware grows ever more complex, and the need for information ever more pressing in outbreak situations, the reverse engineer faces an uphill battle: one where the attacker(s) produces malware using a rich set of tools and libraries, working at a high level of abtraction, whereas the defender suffers from a relative lack of appropriate tools and methodologies.

Fighting dragons with toothpicks is a perilous endeavour. For the brave knights among the readers, this paper exposes some avenues of research and defines rational, incremental, collaborative, abstract, agile, and extreme reverse engineering. We present some tools and strategies demonstrating these principles.