ICSA Labs report reveals testing details
Certification issues unmasked in 20-year lab survey.
Renowned testing and certification group ICSA Labs has released a fascinating report drawing back the curtain on
its security product tests - the detailed results of which are generally kept quiet as products go through the process of achieving
certification.
Among the most widely noted facts in the report is the large number of products that require multiple cycles
through the test programme before they manage to achieve certifiable standards.
As with many certification bodies, ICSA generally reports only the status of those products that have made the grade
and been awarded certification, while failure to reach the required standards usually results in a private report to the
vendor involved so that they can fix any problems with the product and have another go at the test.
According to the report, only 4% of all products tested achieve certification on the first attempt - although 27% of
anti-malware solutions are found to be certifiable from the off, the average is heavily dented by the VPN and web
application firewall categories, where no product has ever made the grade without receiving advice and assistance from the
certifiers.
While these figures have been picked up by the sensational press as indicators of general inadequacy across the security
industry, the report aims to highlight the tough and challenging nature of the certification process, and the efforts
gone to by certification labs to work with vendors to improve their solutions, providing expertise and cross-industry
cooperation.
The report also provides an overview of the history of ICSA and the security industry and offers advice to both
product developers and potential buyers, with a lengthy checklist of criteria to consider when selecting a security
solution. The report was created in collaboration with ICSA owner Verizon Business, and was issued as
part of the inauguration of a new blog. The first entry of the blog, penned by contributing writer and ICSA MD George
Japak, is available
here.
The report itself can be found here, with comments on the
ESET blog here, and in The Register
here.
11 November 2009
Tags:
certification, icsa, report, testing.
del.icio.us 
digg this
0 comments
