As long as we have connected our computers to the internet, miscreants have endeavoured to steal our data, disrupt our businesses, siphon off funds and generally make themselves a pain in the ass.
When the “World Wide Web” and “E-Mail” started to take off in the ’90s, Windows 9x and MS-DOS were prevalent. Viruses at that time would typically embed themselves in boot records of hard disks and floppy disks so that even re-installing your operating system or formatting the disk didn’t get rid of them. To stop that, adding write protection to your boot sectors via the BIOS or anti-virus software was an easy defence to erect.
Fast-forward to now, and malware (short for “malicious program”, which is an umbrella term for viruses, trojans, rootkits, worms, etc) is much more diverse and clever. Some malware infects your browser, some infect your Word documents, and some infect Windows programs. With some malware, infection is obvious – e.g. ransomware encrypts your files and prompts you for a ransom to decrypt. Some malware is quieter, embedding itself in your system without your knowledge, perhaps siphoning off your data one byte at a time, perhaps waiting for you to log into your bank in order to steal your password, or perhaps using your machine as a “zombie” in a botnet to attack other machines.
Anti-virus (AV) software vendors rose up to offer users protection when viruses first appeared. While the capability of anti-virus software has improved a lot since then, the basic technology that drives anti-virus remains the same.
AV software traditionally works on signatures (sometimes known as fingerprints). Every file in your system has a signature, which is unique to that file, much like a real-life fingerprint. If I send you a file, you have a copy, and the signature is the same. When the first viruses were roaming the web this was an effective method of containing infections: an email containing a program (which contained the virus) may be sent to thousands of users, and each of those people received the same program, thus the same signature. So the AV software those users were running simply needed one signature in its database to look for, and the entire infection could be rendered inert. Cue handy diagram:
Malware writers cottoned on to this pretty quickly. I said that two copies of the same program share the same signature. If you change your copy at all, even only slightly, the signature changes. So what malware writers started doing was changing their malware in a way that didn’t alter what it actually did, but that made it look different and thus triggered a change in signature.
For example, the program could pose as a program that clears the user’s temporary files to save disk space, something which is perfectly benign:
The signature change means the virus is no longer in the AV database, and the signature match fails. AV vendors started to get around this by employing “partial signatures”, whereby only part of the file’s signature has to match.
This is all well and good, but the examples I’m showing you are very simplistic. It is easy for malware writers to use similar techniques to disguise their malware and thus change signatures even more. Even in this simple example, there are a handful of ways I can change the signature without changing the end result:
You would need a set of AV signatures that covers all the different combinations of these possible approaches in order for the new malware to be detected, and that simply isn’t practical as most of the time the possibilities are far more than half a dozen; they could easily run into thousands of permutation. This scrapes the surface of the level of indirection that can fool signature-based AV scanning.
Seeing that the arms race was going to be lost with just signature detection in their tools, AV vendors started incorporating heuristics and behavioural detection into their products, to run alongside signature detection. These detection methods try to identify things that “look” like malware, trying to catch anything that appears to be taking advantage of vulnerabilities, writing system files or other suspicious activity. An article by Symantec written in 2002 on behavioural detection gives more detail to this approach. Their list of example behaviours that you may block are:
“This is more like it,” you may think. Judge a program not by its words, or its signature, but by its actions. And it’s a very sensible approach. For traditional threats, this model works well as long as the behaviours you’re looking out for are up to scratch.
In the wild, where malware is much more complex, signature- and behaviour-based detection of malware is falling behind. These days, malware can be easily put together via modular toolkits such as Flame (which was behind Stuxnet, for example). These toolkits make mutating malware an easy task, evading signature detection, and employing gimmicks and indirection to dupe behaviour-based protection.
AV scanners cannot for example protect against buffer overflows, where another program running on the system (e.g. Internet Explorer) can be made to execute code injected by an attack overflowing a memory buffer. Buffer overflows are arguably the most severe and commonly-occurring class of software vulnerability in the last 25 years, representing the highest proportion of high-severity vulnerabilities, i.e. resulting in high-impact breaches of security.
Symantec has even admitted that AV is dying (as have others). Back in 2014, Symantec's Senior President estimated that AV catches only 45% of cyber attacks, thanks to the newer methods that malware authors and hackers are using to attack vulnerable systems.
So, I hope you now feel enlightened (if you didn’t already, that is) about how AV really works. When the malware was just leaving the start line, AV was an effective means to stop attacks, but that time is long gone. Malware is now in full flow, funded and used by criminal groups across the globe every day.
AV also focuses on the user’s machine, protecting its programs and data. As we move more of our data into the cloud, and onto cloud applications, the value of attacking the user’s machine is changing. Owning a machine is now valuable for a botnet, or for attacking others. It is less valuable (necessarily) for the data that the machine contains; for that, it is better to attack banks, email providers and cloud providers directly and potentially extract millions of users’ details in one fell swoop in the process.
What, then, for we who want to stay secure? The answer is that AV, like a firewall, forms part of an overall “security architecture”. It is one line of defence only, and should never be solely relied upon to protect you.