Posted by: David Harley | August 19, 2016

AV-Test on Android security apps

Davey Winder asks some interesting questions about AV-Test’s latest test of Android security apps. Is Android as easy to secure as the latest AV-TEST results appear to suggest?

A number of people, including ESET’s Mark James, attempt to answer those questions, but unfortunately the article boils them down to soundbites. Maybe I’ll come back to this one.

David Harley
ESET Senior Research Fellow

Posted by: David Harley | August 5, 2016

SE Labs tests products against ransomware

Testing lab SE Labs has been testing anti-malware programs in order to evaluate their effectiveness against ransomware: Anti-malware vs. ransomware: latest reports

There are reports covering products intended for large businesses/enterprises, small-to-medium businesses, and home users/consumers. I haven’t looked at them in detail yet, but I expect them to be up to Simon Edwards’ usual high standards.

[This item also posted to the AVIEN blog.]

David Harley

Posted by: David Harley | July 19, 2016

AV ‘fossils’ versus ‘next-gen’

Now there’s a topic guaranteed to raise eyebrows and clenched fists. Next-generation vendors who insist that ‘traditional’ anti-virus is dead because it relies on signature detection;  traditional companies who point out that they haven’t relied on static signatures in decades, and that the groundbreaking technologies claimed by next-gen vendors are not so dissimilar to those used nowadays by ‘traditional’ security suites; complaints that next-gen companies have been using VirusTotal (inappropriately and misleadingly) to ‘prove’ it using stacked testing methodologies, while silently benefiting from the research of the ‘fossils’ whose marketshare they hope to capture.

Well, since I derive a large proportion of my income from a ‘fossil’ anti-malware-oriented security company, I don’t expect you to assume that I’m unbiased. I will say, though, that this article by Kevin Townsend – Inside The Competitive Testing Battlefield of Endpoint Security – strikes me as a pretty good, balanced summary of many of the issues.

Oddly enough, while some of the marketing-rich, fact-impoverished statements I’ve seen from next-gen vendors infuriate me more than I care to say – I prefer to blog without profanity, in general – I’m not altogether without sympathy for their mistrust of mainstream product testing. On the whole, I think AMTSO, warts notwithstanding, has helped in raising the standard of mainstream testing far higher than I could have hoped a few years ago, but I’m not sure that comparative tests can be as effective as testers would like you to think. Nonetheless, consumers need and deserve some impartial guidance as to which vendors deserve their custom. As long as next-gen vendors claim that no-one is capable of running accurate tests of their products, and while they at the same time claim to be able to run their own flawed pseudo-tests for marketing purposes, they can’t expect to avoid independent and informed criticism. If they actually showed willingness to work with testers (even if not within AMTSO) to work towards more effective testing of their products, they’d gain in trust and credibility. Or would that be too traditional?

David Harley

Posted by: David Harley | May 26, 2016

ICSA Labs: Testing the Internet of Things

ICSA Labs, nowadays a division of Verizon, has a long history in the world of testing and certification (and is a longstanding member of AMTSO). The company has come up with rather a good idea: security certification for the disparate range of devices and sensors that make up the Internet of Things. There’s no doubt that anything that might help in raising IoT security standards is worth a cheer or three.

Unfortunately, at the time of writing I can’t seem to access the ICSA Labs web site, but there’s an article by Richard Chirgwin for The Register that goes into a little more detail – ICSA Labs wants IoT industry to seek security certification – though he’s sceptical as to how much interest there’ll be. The article links to an announcement here, and a white paper describing the programme here, and I’ll be taking a look at those as soon as I can.

Chirgwin also mentions a somewhat similar programme announced by the Underwriters Laboratories (UL): UL Launches Cybersecurity Assurance Program. The announcement claims that:

New UL 2900 Series of Standards Offer Testable Cybersecurity Criteria for Network-Connectable Products & Systems

Clearly also worth a look.

I may come back to this topic in the near(-ish) future.

David Harley


Posted by: David Harley | May 12, 2016

SE Labs: what Simon did next…

…Simon Edwards, that is. Simon has had considerable influence on the testing scene in recent years both as a tester with Dennis Technology Labs, where he was Technical Director, and as one of the leading lights of AMTSO, where he was formerly chairman of the Board of Directors.

The web site for his new venture, SE Labs, is now up and running (though it has a couple of rough edges at the time of writing), and already includes a report on Home Anti-Malware Protection that compares a number of products. Registration (which is painless and not over-intrusive) is required to access enterprise and business reports.

As you’d expect from Simon, the site is more informative than most sites about methodologies. There’s also a blog page which I will follow with interest.🙂

Hark! What is that rumbling? I think it might be the trembling of some of those ‘next-gen’ and APT-detection vendors who claim that their technology is too magical advanced to be tested. The site’s About page claims:

Constantly innovating, SE Labs has developed next-generation testing to prove the abilities of ‘next-generation’ security products using a comprehensive, full-stack approach to security assessment powered by true and detailed threat intelligence.

And given Simon’s exhaustive work in that area, I fully expect that he’ll make good on that promise.

Interestingly, the recent cat-meets-pigeons announcement by VirusTotal about Maintaining a healthy community by discarding subscribers who take data from VT but don’t share data or include their service in VT’s API includes this observation:

Additionally, new scanners joining the community will need to prove a certification and/or independent reviews from security testers according to best practices of Anti-Malware Testing Standards Organization (AMTSO).

This will presumably not affect all those vendors who are insisting that they do not use signatures and that losing access to VirusTotal’s data will not affect them in the slightest. But if you need a good testing service, guys, I think there might be one that meets VT’s requirements over here.😉

This is probably not the last you’ll hear from me on this…

David Harley

Older Posts »



Get every new post delivered to your Inbox.