VBSpam testing
|
After more than a decade of experience in the field of anti-malware testing, Virus Bulletin began running an anti-spam testing and certification scheme at the beginning of 2009. The VBSpam test has since become a recognized industry standard. The test uses Virus Bulletin's live email stream, which is sent to all the filters participating in the test in random order, thus exposing each to exactly the same email stream, in real time. The test measures both the false positive rate and the spam catch rate of the products, and VBSpam certifications are awarded to products that exceed a predefined benchmark based on both measurements. Awards
It should be mentioned that both the email corpus used in testing and the way in which a test is set up could significantly influence the test results; hence an SC rate of 98% recorded a VBSpam test does not necessarily indicate the same as an SC rate of 98% measured by a customer or in another test. However, because all products are tested under the same conditions and are required to filter the same emails, the performance of two (or more) products in the same VBSpam test can be compared against each other. The VBSpam test therefore gives customers a good picture of how well a product performs relative to its competitors. (Note: for the first four tests, the award structure and criteria were slightly different; each test report clearly states the criteria used in that particular test.) Product submissionAny vendors interested in submitting anti-spam products for review are advised to contact Allison Sketchley, VB Sales Executive (+44 1235 544034; allison.sketchley@virusbtn.com) and/or Martijn Grooten, Anti-spam Test Director (+44 1235 540235; martijn.grooten@virusbtn.com). |
Project Honey Pot
Testing scheduleVBSpam tests will be carried out six times per year, with the results published in Virus Bulletin magazine and on this website. The product submission deadline for the next anti-spam test is 26 March 2010. The test will run in April and the results will be published in the May issue of Virus Bulletin, with a summary available on this website. MethodologySpam changes and so does anti-spam testing; hence the methodology of the VBSpam anti-spam tests will be tweaked and updated to reflect the latest developments and technologies. However, the methodology will not be altered during the course of a test. The most up-to-date methodology can be found here, while descriptions of the thought processes behind it can be found in the following introductory articles:
Test results
|
VBSpam results chart
Poll
Do you use the same password(s) across multiple websites?Leave a comment
View 4 comments

VB2010
VB2010 will take place 29 September-1 October 2009 at the Westin Bayshore, Vancouver, BC, Canada.
Early bird discount available until 15th June 2010.
Virus Bulletin currently has 190,601 registered users.


VBSpam certification is awarded based on the "final score"
of a product, which is defined as the spam catch rate (the percentage
of spam blocked) minus three times the false positive rate (the
percentage of legitimate email blocked). A product wins an award if
the value of (SC - 3*[FP]) exceeds 96%.