Gone Phishing: Evaluating Anti-Phishing tools for Windows
3sharp, a Redmond based technical services company, has been commissioned by Microsoft to undertake a competitive study of various anti-phishing technologies. The results of that study were released just minutes ago.
The IE team comment on the study:
Before we proceed, I will say, right at the outset, that the only safe antiphishing technology is one that *BLOCKS* access to known phishing sites. Why? Because in its July report (released on 11 September 2006), the Anti-Phishing Working Group reported 182 unique websites hosting password stealing trojans, 1850 sites hosting password stealing malicious code (exploits) and a large increase in traffic redirecting, also known as pharming:
In short, it is not enough to simply warn a user that a Web site is a known phishing site yet still display the page. Just opening a phishing site in your Web browser can be dangerous, even if you have absolutely no intention of entering any information on a page, if that site attempts to infect your system with a trojan, keylogger or other nasty. Please keep this in mind when deciding which protective technology you wish to use. I cannot recommend strongly enough that you choose a product that BLOCKS access to known phishing sites.
Unfortunately IE7 allows sites to continue to load while the phishing filter makes its checks, meaning that it is still theoretically possible for a site to infect a PC even when "blocked" by IE, but any hostile activity that requires user interaction is neutralised. Your security settings would have to be lowered allowing automatic execution of code or active x, or a exploitable vulernerability would have to be used, and we know that IE7 has been immune to virtually all vulnerabilities.
Ok, now to the results....
The products were tested using 100 known phishing URLs (which had to be tested within 48 hours of collection) and 500 known good URLs.
The "winner", with the best overall performance, and a composite accuracy score of 172 out of 200 was Internet Explorer 7 Beta 3 (V7.0.5450.33).
2nd place went to the NetCraft toolbar (V1.6.2) with IE6 with a score of 168 out of 200.
A distant third was Google's Toolbar for Firefox with "Safe Browsing" (V2.0) with Firefox 220.127.116.11 with a score of 106 out of 200.
The remaining products rated:
eBay's toolbar with AccountGuard (V2.3.1) with IE6 - 92 out of 200 (note, eBay restricts itself to eBay and PayPal spoofs and will not detect any other type of phish)
Earthlink's ScamBlocker (V3.1.5) with IE6 - 76 out of 200
GeoTrusts TrustWatch (V3b1) with IE6 - 67 out of 200
Netscape (V8.1) - 56 out of 200
McAfee SiteAdvisor (V18.104.22.168 build 3083) with IE6 - 3 out of 200
Total catch rate for known phish URLs - pay particular attention to the block versus warn percentages
Mistakes made on known "good" URLs
- Although GeoTrust did very well with a 99% catch rate, it also had a very high rate of false positives at 32.2%. Not only that, it does not block access to known phishing sites.
- When scoring results, a false block on a good site was scored as twice as bad as a false warning. Allowing a good site had zero value.
- The known phishing URLs were not taken from any feeds from known third-party data providers or end users to the Microsoft Phishing Filter Service in IE7.
- Known good URLS were pulled from a feed of randomly selected traffic-weighted URLs provided by Microsoft and were independent of, and confirmed not to be included in, the Microsoft Phishing Filter system (they are not in the Phishing Filter white list).
The full report, and associated Press Release, can be found the URL below. The report provides comprehensive information about how the products were tested, the rules under which the tests were conducted, how and where the phishing URLs and good URLs were sourced and how scores were calculated, and a full list of the URLs used during testing is also included.
FAQs about "Gone Phishing: Evaluating Anti-Phishing Tools for Windows"
The most important thing to me is that users are safe when browsing the Internet. That is why I am doing all I can to encourage users to update their copy of Internet Explorer to IE7 Release Candidate 1. I also strongly recommend that, if you are using IE7, you enable the Phishing Filter.
If, for whatever reason, you are not able to run IE6 then I recommend that you download and install the NetCraft's toolbar.
Quick statistics about IE7's phishing filter
The Phishing Filter is a “real time” service that does not require a user to download or regularly update a list of “bad” sites.
Microsoft has been adding up to 17,000 URLS a month to its Phishing Filter service.
From February to Mid Aug 2006 the Phishing Filter helped block over 800,000 instances of people trying to access reported phishing websites using IE7 or MSN/Windows Live Toolbar. This figure includes almost 500,000 blocks since IE7 Beta 2 was released.
IE7 users are reporting up to 4,500 potential phishing sites per week.