The testers state that while no samples were executed in the course of the test, cases were considered where malware would be recognized on-access but not on-demand. Well, it’s true that executing a file is not the only way to access it, and the difference between on-access and on-demand scanning is less clear-cut in modern top-tier security products. Perhaps we should be revisiting those terms in order to establish a reasonably standard definition.
AV-C does acknowledge that the test only looks at one aspect of product functionality. And I like the fact that results in the detection test are balanced by a false positive test, to lessen the risk that a product will get a high score by simply flagging all unknown files as malicious. So potentially quite a useful test, despite its limitations.