Security looks to machine learning technology for a cognitive leg up
Advances in machine learning technology and artificial intelligence have proven to work well for some information security tasks such as malware detection. What's coming next?
Keen Footwear sells its iconic boots, shoes and sandals through thousands of retailers worldwide. But the Oregon manufacturer, which is working hard to its commitment to become "American Built," does not have the manpower to support a dedicated information security staff. With a team of six information technology professionals -- all but two focused on handling the day-to-day client issues of its 450 employees -- the IT staff would fall behind in triaging incidents the company's security software flagged.
"We fit squarely in the realm that we have the problems of all the big players, but we don't have the resources of a large enterprise," said Clark Flannery, Keen's director of IT in Portland.
To solve the problem, Flannery augmented his IT staff with machines. While the company had a traditional firewall and antivirus software to block the most obvious threats, Flannery opted to deploy Darktrace's Enterprise Immune System, a physical appliance that passively monitors network data and then uses machine learning technology and probability theory to model patterns of behavior and flag anomalous activity. (The "self-learning platform" from U.K. cybersecurity startup Darktrace is based on unsupervised learning -- anomaly detection -- and recursive Bayesian estimation, and was developed at the University of Cambridge.)
For Flannery, who considers the system a form of artificial intelligence, the machine learning technology means that his team has less work piling up: "With this AI, we do not have to look through the minutia unnecessarily." Instead, he gets reports on events on which the team needs to focus: brute-force login attacks, shadow IT usage or other anomalous traffic.
Flannery is not alone. With the high volume of data that most security teams have to prioritize, machine learning technology is increasingly being adopted as a way to reduce the noise (alerts) that traditional security products produce and to bubble up mid- and high-level concerns to IT staff. The discipline of machine learning finds its way into many large companies through the hiring of data scientists, who use algorithms to efficiently analyze event logs for their security teams.
"If you look at any of the large companies with excellent security teams, they have all integrated data scientists and machine learning, creating new skill sets contributing to this domain of cyber and network security," John Lambert, general manager of Microsoft's Threat Intelligence Center, told attendees of the company's January BlueHat IL security conference in Tel Aviv.
Machines benefit smaller companies most
Smaller companies, such as Keen, have turned to platforms that incorporate machine learning and AI techniques -- and soon automated defense -- to solve a variety of problems. "I don't need to go hire someone dedicated to security," Flannery said. "It just feels like a whole team back there -- who are way more qualified than [staff] I would be able to pay."
While machine learning and artificial intelligence are often used interchangeably, the concepts are different. Machine learning is a branch of data science that uses data sets to train statistical methods of analysis; it is the launching point to developing approaches to adding intelligence to software. The predictive models and algorithms generally fall into one of three classifications: supervised learning, unsupervised learning and reinforcement learning.
Artificial intelligence seeks to create software that can think about problems like a human. IBM's Watson for Cyber Security, which relies on machine learning technology and natural language processing, may be moving in that direction. The technology can consume unstructured security data -- research papers, blogs, video -- and uses cognitive processes developed by IBM's research and development in deep learning and neural networks to provide algorithms out of the box. Forty companies in banking, healthcare, insurance and other industries signed up in December 2016 to participate in the IBM Watson Cyber Security beta program.
Many cybersecurity providers claim their technologies represent the generation of AI -- IBM is not among them. In general, computational procedures or processes that can be characterized as intelligent remain open to debate. A software replacement for a security analyst would arguably be artificial intelligence.
Gunter Ollmann
"In the security industry, no one is using artificial intelligence," said Gunter Ollmann, CSO at Vectra Networks Inc., an automated threat management startup in San Jose, Calif. A mentor to tech companies and self-described executive for hire, Ollmann has performed CTO or research roles for NCC Group, IOActive, Damballa and IBM.
Security applications are more likely to use advanced machine learning while incorporating basic AI techniques. Machine learning technology is used in malware detection, dynamic risk analysis and anomaly detection. The technology can perform threat detection in dynamic environments, but it still requires humans in the loop.
The promise of machine learning -- especially as it evolves into something resembling artificial intelligence -- is its ability to significantly reduce complexity for human analysts, said Bryan Lares, director of cognitive security solutions at SparkCognition Inc., based in Austin, Texas. The company's DeepArmor antimalware platform uses machine learning, natural language programming and AI techniques to detect infections across networks and devices, including the internet of things. It is still accepting participants for the DeepArmor beta program.
Who's Got the (Machine) Smarts?
Outside of enterprises adding data scientists to their security teams, machine learning is typically applied to security in three types of companies.
Traditional security vendors -- think Symantec and Intel's McAfee -- have adopted machine learning in their products, but also use it to reduce the workload of their analysts as they try to keep up with the deluge of malware. In 2015, the latest numbers available, Symantec had to analyze and classify 431 million malware variants.
Companies focused specifically on generalized machine learning techniques and artificial intelligence goals have targeted the information security sector as a lucrative application of their technology. Recorded Future, for example, initially pursued natural-language processing to produce intelligence, but has strongly focused on using the technology to gather information on cybersecurity threats.
A number of cybersecurity startups have developed their technology by applying machine learning to specific cybersecurity problems. Cylance and SparkCognition are early adopters of AI techniques to detect unknown malware, for example.
Investments have skyrocketed in these firms. Cylance raised $100 million in a Series D funding round last year, valuing the company around $1 billion, according to CB Insights. Texas-based StackPath, which uses machine learning for real-time threat detection, raised $180 million in private equity.
With a greater number of devices to worry about, and a burgeoning amount of data from those devices to parse, security teams are running up against a productivity barrier. Unless the data is whittled down more effectively, security incidents will continue to be missed.
Yet, when the techniques work, systems using machine learning can bring consistency to the analysis of security events, catching those that might otherwise fall through the cracks. The application of the technology also reduces work for security analysts and IT staff by weeding out the chaff and highlighting the most serious security concerns. Machine learning technology can give IT and security teams added depth of knowledge, detecting patterns or issues they may not have known about.
Promise of the machines
If the technology can move beyond threat detection and into incident response and prediction -- essentially acting as a software-based analyst -- then we will have moved into artificial intelligence, said William Altman, tech industry analyst at York venture intelligence firm CB Insights.
"That's the difference between knowing someone is in your house when they rob it versus knowing they are going to rob it and how they will get in before they do it," Altman said. "With the skills shortage of qualified cybersecurity pros, increased automation of security monitoring and controls is allowing for an augmented approach that allows fewer analysts to respond to the most relevant security red flags."
Some companies have already started down that path. Telecom firm IDT Corp., based in Newark, New Jersey, uses an incident-response system powered by AI techniques to whittle down the volume of data and speed response time. In the past, the company required at least 30 minutes to even detect and start triaging an incident, which then required a minimum of four hours to forensically analyze. And that's for incidents classified as critical, said Golan Ben-Oni, CIO at IDT Corp. Many low- and medium-severity ranked issues ended up being critical, he said.
You can't say that you are going to replace your analysts. What ends up happening is that you put your analysts to better use.
Golan Ben-OniCIO at IDT Corp.
To address the problem, IDT adopted Hexadite's Automated Incident Response Solution (AIRS) to triage incidents and act, automatically in most cases, to stop potential attacks. In 30 minutes, the company now not only automatically analyzes every alert, but can quickly place a system on a quarantined "remediation network" to isolate it. Following quarantine, a full investigation of the host is conducted with AIRS. For instance, command-and-control server IP addresses can then be pushed to the dynamic, block access lists of the firewall to protect the remainder of the organization. If the system is found to have been infected with something more critical than a potentially unwanted program, as in the case of malware, the host will be flagged for re-imaging back to factory default.
"You can't say that you are going to replace your analysts. What ends up happening is that you put your analysts to better use. It saves you from getting additional people," Ben-Oni said. "The analysts can work on problems that they never had a chance to, like getting in touch with other companies to discuss the origination of the threat."
Machine evolution
Machine learning is really good at crunching through data, but we are still far from replacing security analysts, said Joseph Blankenship, a senior analyst for security and risk at Forrester Research.
"One thing we need to do [is] to make any automation possible -- and make better and faster decisions," he said. "So the next milestone will be when we will see something from a tool and have a high enough confidence level to not have an analyst in the equation."
Current machine learning systems have problems with false positives. While numbers are not available, Lares acknowledged that the goal is 99% accuracy in both detecting malware and determining whether a file is clean.
Joseph Blankenship
The challenge for companies is that they are trying to hit a moving target. Machines must be able to adapt to detect evolving threats, said Jon Miller, chief research officer at Cylance Inc., in Irvine, Calif. The company offers an AI engine called CylanceProtect for detecting malware and other threats.
"Replacing human detection of cancer with an AI solution is totally possible because cancer today is the same as cancer tomorrow," Miller said. "Unfortunately, in the information security world, there is no natural evolution. Attacks that come at you today are not the attacks that are going to come at you tomorrow."
Organizations can also expect attackers to adopt machine learning and AI techniques. Even with advances in machine learning technology, and even if everyone agrees that it should be considered artificial intelligence, attackers will use these techniques against businesses. By automating software models and the search for vulnerabilities, adversaries will be able to create their own machine learning and AI systems, Miller said. "In the end, the adversary for AI will be AI."
Gunter Ollmann
Joseph Blankenship
