Fighting malware and spam

Defeating polymorphism through code optimization

Frédéric Perriot Symantec

Defeating polymorphism through code optimization

In the last decade dealing with ever more complex polymorphic viruses has been one of the prominent challenges faced by the anti-virus industry. The traditional approach of emulating polymorphic decryption loops to reach the constant virus body underneath is widely regarded as the most powerful defence against polymorphism. Once decrypted the virus body can be used for detection purposes and lends itself to a detailed analysis. Unfortunately this approach is computationally expensive and reaches its limits when faced with metamorphic viruses.

This paper explores an alternative solution related to code simplification techniques borrowed from the field of optimizing compilers. Such techniques as copy propagation, constant folding, code motion and dead-code elimination may be used instead of, or prior to, emulation. These turn out to be powerful allies in the fight against the likes of Zmist and Simile.