Evaluating anti-virus products with field studies

Fanny Lalonde Lévesque École Polytechnique de Montréal
Carlton R. Davis École Polytechnique de Montréal
José M. Fernandez École Polytechnique de Montréal

The evaluation of anti-virus (AV) products is a vital component in helping the industry develop better products that match the evolving malware threats, and in helping users to make informed decisions about product selection. Traditional evaluation methods involve testing in laboratory environments under various threat scenarios, some more realistic than others. In this paper, we present a first study of an alternative method of product evaluation involving real users. We report on the performance of one AV product in a four-month field study involving 50 users, using their own machines in their normal daily business. In addition, we cross-analyse detection data with user behaviour and demographic characteristics in order to determine what factors are conducive to higher risks of infection. We conclude by discussing options that would allow this methodology to migrate to multi-product evaluations, and become a repeatable and viable alternative to traditional lab-based comparative testing.

 del.icio.us  digg this! digg this

Quick Links

Poll
Should software vendors extend support for their products on Windows XP beyond the end-of-life of the operating system?
Yes - it keeps their users secure
No - it encourages users to continue to use a less secure OS
I don't know
Leave a comment
View 23 comments

AusCert2014

Virus Bulletin
In this month's magazine:
  • VBSpam comparative review March 2014
  • VB100 comparative review on Ubuntu Server 12.04LTS
  • The shape of things to come
  • Threat intelligence sharing: tying one hand behind our backs
  • The curse of Necurs, part 1
  • More fast or more dirty?
  • Tofsee botnet
  • Back to VBA
  • Is the security industry up to the new challenges to come?
  • Greetz from academe: No place to Hyde
Virus Bulletin 04 2014
Subscribe now!

Virus Bulletin currently has 231,288 registered users.