Posted by: David Harley | October 19, 2011

Testing: the Knight’s Tale

Strange how the whole AV testing issue seems to attract metaphors and analogies harking  back in some sense to mediaeval romance. For the past few days I’ve been muttering grimly about tilting at windmills, having recently come across an unhappy example of a test apparently relying on a simulation. (You’ll probably hear more from me on that, though I won’t necessarily be performing my impersonations of Don Quixote on this blog) .

And now I’ve come across a recent blog by Eugene Kaspersky in which he talks about The Holy Grail of AV Testing, and Why It Will Never Be Found: in fact, making a plea for something that sounds very similar to “whole product testing” as defined by AMTSO (which features strongly in his argument), and building on another of his articles, as I previously mentioned here. Essentially, he comes to the depressing conclusion that “…sadly, proper results from proper tests are these days simply nowhere to be found, despite their Holy Grail status.”

“Nowhere” might be a little harsh, but I have to admit that I’ve heard a lot of similar sentiments expressed recently. Yet here I am at yet another AMTSO workshop, halfway through a gruelling schedule of meetings. AMTSO is still here, and still trying to establish some form of chivalric code. Please keep any mutterings about honour among thieves to yourself: after many years in or on the fringes of the AV industry, I’ve heard it all, but I haven’t (altogether) lost faith in the ability of the vendor and tester communities to put personal differences and vested interests aside for the protection of the end user. Flashbacks to Monty Python notwithstanding.

David Harley CITP FBCS CISSP
ESET Senior Research Fellow

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: