Out of the Shadows

I came across an interesting article at Shadowserver which doesn’t directly mention AMTSO, but I suspect that freed0 had in mind some of the recent controversy on the AMTSO blog and in other blogs and articles.

However, the the article “Of Opinions and Anti-Virus Testing” mainly describes what he calls as Shadowserver’s testing. I’m not altogether convinced that this is testing in the sense of product testing at all, and while he’s equivocal about whether it’s meant to be comparative testing, I guess it’s inevitable that people are tempted to use it as a means of ranking products according to the percentage of detections. I don’t think this really works because it’s a long way from being whole product testing.

Still, an interesting blog, though I can’t say I’ve come across those criticisms in reference to Shadowserver at all. Interesting 0-day detection statistics, too: I’ll be spending some more time tracking those, I think, though probably not in the comparative testing context.

David Harley CITP FBCS CISSP
ESET Senior Research Fellow