Posted by: David Harley | July 19, 2016

AV ‘fossils’ versus ‘next-gen’

Now there’s a topic guaranteed to raise eyebrows and clenched fists. Next-generation vendors who insist that ‘traditional’ anti-virus is dead because it relies on signature detection;  traditional companies who point out that they haven’t relied on static signatures in decades, and that the groundbreaking technologies claimed by next-gen vendors are not so dissimilar to those used nowadays by ‘traditional’ security suites; complaints that next-gen companies have been using VirusTotal (inappropriately and misleadingly) to ‘prove’ it using stacked testing methodologies, while silently benefiting from the research of the ‘fossils’ whose marketshare they hope to capture.

Well, since I derive a large proportion of my income from a ‘fossil’ anti-malware-oriented security company, I don’t expect you to assume that I’m unbiased. I will say, though, that this article by Kevin Townsend – Inside The Competitive Testing Battlefield of Endpoint Security – strikes me as a pretty good, balanced summary of many of the issues.

Oddly enough, while some of the marketing-rich, fact-impoverished statements I’ve seen from next-gen vendors infuriate me more than I care to say – I prefer to blog without profanity, in general – I’m not altogether without sympathy for their mistrust of mainstream product testing. On the whole, I think AMTSO, warts notwithstanding, has helped in raising the standard of mainstream testing far higher than I could have hoped a few years ago, but I’m not sure that comparative tests can be as effective as testers would like you to think. Nonetheless, consumers need and deserve some impartial guidance as to which vendors deserve their custom. As long as next-gen vendors claim that no-one is capable of running accurate tests of their products, and while they at the same time claim to be able to run their own flawed pseudo-tests for marketing purposes, they can’t expect to avoid independent and informed criticism. If they actually showed willingness to work with testers (even if not within AMTSO) to work towards more effective testing of their products, they’d gain in trust and credibility. Or would that be too traditional?

David Harley

Posted by: David Harley | May 26, 2016

ICSA Labs: Testing the Internet of Things

ICSA Labs, nowadays a division of Verizon, has a long history in the world of testing and certification (and is a longstanding member of AMTSO). The company has come up with rather a good idea: security certification for the disparate range of devices and sensors that make up the Internet of Things. There’s no doubt that anything that might help in raising IoT security standards is worth a cheer or three.

Unfortunately, at the time of writing I can’t seem to access the ICSA Labs web site, but there’s an article by Richard Chirgwin for The Register that goes into a little more detail – ICSA Labs wants IoT industry to seek security certification – though he’s sceptical as to how much interest there’ll be. The article links to an announcement here, and a white paper describing the programme here, and I’ll be taking a look at those as soon as I can.

Chirgwin also mentions a somewhat similar programme announced by the Underwriters Laboratories (UL): UL Launches Cybersecurity Assurance Program. The announcement claims that:

New UL 2900 Series of Standards Offer Testable Cybersecurity Criteria for Network-Connectable Products & Systems

Clearly also worth a look.

I may come back to this topic in the near(-ish) future.

David Harley

 

Posted by: David Harley | May 12, 2016

SE Labs: what Simon did next…

…Simon Edwards, that is. Simon has had considerable influence on the testing scene in recent years both as a tester with Dennis Technology Labs, where he was Technical Director, and as one of the leading lights of AMTSO, where he was formerly chairman of the Board of Directors.

The web site for his new venture, SE Labs, is now up and running (though it has a couple of rough edges at the time of writing), and already includes a report on Home Anti-Malware Protection that compares a number of products. Registration (which is painless and not over-intrusive) is required to access enterprise and business reports.

As you’d expect from Simon, the site is more informative than most sites about methodologies. There’s also a blog page which I will follow with interest.:)

Hark! What is that rumbling? I think it might be the trembling of some of those ‘next-gen’ and APT-detection vendors who claim that their technology is too magical advanced to be tested. The site’s About page claims:

Constantly innovating, SE Labs has developed next-generation testing to prove the abilities of ‘next-generation’ security products using a comprehensive, full-stack approach to security assessment powered by true and detailed threat intelligence.

And given Simon’s exhaustive work in that area, I fully expect that he’ll make good on that promise.

Interestingly, the recent cat-meets-pigeons announcement by VirusTotal about Maintaining a healthy community by discarding subscribers who take data from VT but don’t share data or include their service in VT’s API includes this observation:

Additionally, new scanners joining the community will need to prove a certification and/or independent reviews from security testers according to best practices of Anti-Malware Testing Standards Organization (AMTSO).

This will presumably not affect all those vendors who are insisting that they do not use signatures and that losing access to VirusTotal’s data will not affect them in the slightest. But if you need a good testing service, guys, I think there might be one that meets VT’s requirements over here.😉

This is probably not the last you’ll hear from me on this…

David Harley

Posted by: David Harley | April 22, 2016

EICAR Call for Papers

I haven’t had much to do with the EICAR conference in recent years – well, it’s nearly 18 months since I went to any conference, and even longer since I went to Virus Bulletin, rather to my own surprise – but I note that the Call for Papers is now on the web site. I feel obliged to note these things when I’m on the Review Team.😉 The announcement says that:

The 24th Annual EICAR Conference will be held on October 17th and 18th with a pre-conference program on the EICAR Minimum Standard in Nuremberg, Fairground, at the IT-SA conference facilities.

The conference theme is ‘Trustworthiness in IT security products’. Which no doubt has something to do with the EICAR Trustworthiness Strategy, which is a topic I may well come back to in due course (hence its inclusion on this site).

More information at http://www.eicar.org/17-0-General-Info.html.

David Harley

 

Posted by: David Harley | April 22, 2016

AV-Comparatives test security product support

One aspect of security software that isn’t often tested is the quality of its support. AV-Comparatives, however, has grasped that particular nettle with support evaluation reports from security vendors with support desks in the UK and in Germany. The reports are available here, and I commented at more length for Infosecurity Magazine: Testing Anti-Malware Support.

I like the idea, but would like to see AMTSO consider generating some guidelines.

David Harley

Older Posts »

Categories

Follow

Get every new post delivered to your Inbox.