Schneier on Testing…

…Not, I hasten to add, on anti-malware testing, on this occasion. And since I’m not a subscriber to the Cult of Schneier – certainly when he pontificates on the shortcomings of the anti-malware industry – I would have examined any thoughts he had expressed on that specific topic with enough salt to hand for several large pinches.

Nonetheless, his essay for CNN on how the VW scandal could just be the beginning makes some very good points. The title is misleading by the way, though of course it probably wasn’t Schneier’s own choice (the title he uses in his newsletter is Volkswagen and Cheating Software): early in the article he makes the point that ‘Cheating on regulatory testing has a long history in corporate America.’ (And that isn’t, of course, only true of the US.) And I haven’t spent all these years monitoring security product testing without becoming aware of products that have crossed the line between legitimate optimization and outright cheating.

Emissions testing and security product testing have more in common than you might think. While there are plenty of security product tests by people who reviewed cameras last week and refrigerators the week before, there is a moderately healthy security product testing industry heavily populated by people who know quite a lot about what they’re testing: the same is (or at any rate should be) true of government agencies that test (or at least have input into testing) products that have to conform to safety and other standards (and in some contexts that includes security software).

Schneier observes that:

We’re ceding more control of our lives to software and algorithms. Transparency is the only way verify [sic] that they’re not cheating us.

In the age of the Internet of Things, there’s more to safety regulation than ensuring that electrical wiring and the height of stair risers meets standards, though software security is not yet taken into account nearly as much as it ought to be. But we shouldn’t just be thinking about Things: to take just one example, social media services are notoriously cavalier with our behavioural (and other) data, yet less than transparent when it comes to disclosing the algorithms on which their marketing of our data depend.

I note, by the way, that AMTSO’s statement on the recent round of cheating in anti-malware tests on which I commented here and elsewhere has not survived the refurbishment of the AMTSO web site (unless it’s been moved somewhere I failed to find it). Hopefully that’s a matter of housekeeping rather than AMTSO putting its head back behind the parapet at any hint of contention.

David Harley