I think the testing labs are doing great in certifying malware scanners
Failure to find all malware in the famous WildList can cause an anti-malware product to fail VB100 certification. Sometimes this is scandalous, as when Microsoft's OneCare failed WildList testing last year to widespread derision. But what does the WildList really prove?
In fact, insiders in the anti-virus industry, especially vendors, are widely derisive of the WildList, looking on it as an outdated burden on their development. The malware in it is outdated and not representative of the true threats facing users.
http://www.eweek.com/c/a/Security/The-AntiMalware-Certification-Problem/
VB100, Av-test, Av-comparatives.org, ICSA etc are doing great in certifying malware scanners. I actually would like them to have a certification for other scanners like antispyware, anti-trojan. This is because there are huge # of fake and misleading malware scanner vendors that is installed on many system. And there are some malware scanners that fail miserably. With certifications (detection and removal certifications)... it will help companies or users in choosing which product will likely detect the threats.
I hope not all vendors find it as outdated. Burden, yeah because they have to make sure that their product can still detect old threats but to call it outdated requies proof that the old threat does not exist anymore in any method.
Our own test http://www.dozleng.com/updates/index.php?showtopic=18279 also show there lots of malware scanner (as per VirusTotal.com's online scan result and own test using 8 free malware scanner) will not prevent or detect the rogue installers that consists also of old threat.
And just look at SANS Internet Storm Center's blog entry about old threat that many malware scanner failed to detect:
http://isc.sans.org/diary.html?storyid=3938
ISC wrote:
While it is fair that AV companies need time to come up with signature and defenses for the latest malware coming up the horizon, this keylogger has been sitting on download.com for years (file date shows Aug 2005), maybe the AV engine somehow forgotten about it? What really worries me is when I do a search on download.com for "keylogger", there're 248 hits, makes me wonder how many of those keyloggers are caught by different anti-virus and anti-apyware engines.
I think malware scanner need not to bloat their product with lots of malware signature. If their heuristic technology or whatever new technology (behaviour based or not) they plan to make, they should ensure that their product can find something. If they can't ID it, the new scanner that they added should also find something on it suspicious.
Look also at the new av-comparatives.org testing result published yesterday.... Lots of # of threats on the test. The samples are new and yet, not all antivirus can pass the test.
This just show that whether the samples are old and new... two or more will fail.
via Calendar of Updates