Testing patience
Consumer Reports does it again.
After having come in for a great deal of criticism in recent weeks over its
AV testing methodology (see VB, September 2006, p.2),
Consumer Reports has - amazingly -
damaged its credibility further after having confirmed that, during its
testing of anti-spyware applications, CR did not test against any spyware.
CR's review of anti-spyware products was based on running the applications
against the Spycar set of applications that mimic spyware behaviour. While
it would be understandable (if not entirely forgivable) that testers
lacking experience in the anti-malware field could make such a gaffe, what
makes this more astounding is that the creators of Spycar state clearly and
specifically that Spycar is not suitable (as a sole test method) for
anti-spyware testing.
The EULA states '...Spycar ... is intended to be used
to see how anti-spyware tools cope with new spyware for which they didn't
have a signature. It is not intended to provide perfect anti-spyware tests,
or to act as a substitute for any other form of evaluation.' And the Spycar
website reads: 'Is Spycar a comprehensive test of
anti-spyware tools? No ... Spycar does not evaluate the signature base, the
user interface, and other vital aspects of an anti-spyware tool. Thus,
Spycar alone cannot be used to determine how good or bad an anti-spyware
product is.'
After such a controversial performance CR will need to work hard if it is
to recoup its credibility in the anti-malware field.
01 September 2006
Tags:
del.icio.us 
digg this
