False Positive Testing Automation

In the security software world, a "false positive" occurs when antivirus software mistakenly identifies a legitimate file as "malware" and quarantines it.

As every IT professional is aware, *every* antivirus company in the industry suffers from the occasional false positive. But we're collectively getting better: Avoidance of false positives has become a key service level metric, and because of this, process innovation and automation has received significant levels of new investment.

That doesn't mean the occasional screw-ups won't happen. Recently, one member of the antivirus industry released two definition files (two out of 320,000) that mistakenly identified two system files as malware. As normally happens with the release of a false positive, there was a fair amount of press.

One of the articles I read caught my interest. It seemed to be calling out the company in question for using "automated systems" to check for false positives. I thought this was unfair - and a little surprising, given their effectiveness.

Ten years ago, Symantec, McAfee, Command, and other antivirus companies were probably adding a handful of virus definition files a day to a database containing maybe 20,000 examples of malware, and updating our customers maybe once a week on average.

It was still possible back then to check these new entries manually.

Things have changed dramatically. Today, the Authentium (Command) 5.0 antivirus engine detects almost three quarters of a million viruses and variants, and it is by no means uncommon for us to add several thousand definitions a week, and push out new updates within an hour of the old.

Add to this number the rapid proliferation of new operating system files (Vista, for one), new application files, and web-based threats like spyware, and you start to get an idea a) why we love our malware researchers, and b) why automated systems designed to check for false positives are now permanently bolted into the racks at every antivirus company.