Complaints have been made regularly over the years about ‘testers’ who try to assess product performance by throwing them at VirusTotal’s site to see which products flag them as malicious. In fact, I’ve been one of the most persistent critics of this quasi-testing methodology, and a few years ago wrote a paper with Julio Canto, one of the masterminds behind the VT service, about the reasons why it’s a bad methodology.
VirusTotal has moved on since then, in quite a few ways, not least in the technologies it has adopted and the way in which it uses those technologies. While I still don’t in the least regard submission to VT as a substitute for competent product testing, it has, for instance, adopted a form of sandbox testing analogous to the way in which some anti-malware scanners and other sandbox products and services implement behavioural detection. VT has already addressed ‘Windows PE files in 2012, and Android in 2013‘, and has now added ‘equal treatment for Mac OS X apps‘.
This perhaps blurs the distinction slightly between VirusTotal’s service and other security services in a way that might cause further confusion among pseudo-testers. But that’s not VT’s fault, and I think the value added to its services more than compensates.