Daze of whine and neuroses (but testing is FINE)
David Harley ESET
Larry Bridwell AVG
download slides (PDF)
According to Aerosmith (not to mention The Italian Job), FINE is an acronym for (in its politer version)
Freaked out, Insecure, Neurotic and Emotional. We could (and probably will) offer alternatives, but there's no doubting that
anti-malware testing inspires all those reactions.
Sometimes it seems that AMTSO has become a dumping ground for the rest of the world's misgivings about the AV industry,
even though it originated in a coalition with some of the testers who are monitoring that industry's performance with the
most assiduous professionalism: indeed, that coalition has in itself inspired mistrust. And recently, it has become plain
that even within AMTSO both testers and vendors sometimes find the alliance problematical.
AMTSO's purpose is simple to state, but much harder to achieve. It represents a realization by professional testers and
security vendors that the quality of anti-malware testing was so variable that it was at best confusing for people who
need guidance on how to select the best product for their needs. Perhaps testing has improved more in the past few years
than it would have without AMTSO's presence, and discussions and generation of material in a single forum has accelerated a
much-needed move away from static testing towards dynamic testing.
But it's time to ask (and attempt to answer) a number of tough but critical questions.
- Looking over the historical evolution of testing before and since AMTSO, is that move towards dynamic testing enough
to set the testing world to rights?
- Are the aims of testers and vendors close enough to allow continued cooperation within AMTSO?
- Has AMTSO already outlived its usefulness?
- If not, what should it do next?
- What is the future of comparative detection testing?