Page 30 - FCW, May 2017

30
May 2017 FCW.COM
ExecTech
The hazards of
machine learning
Self-taught cybersecurity systems are evolving rapidly, and hackers are finding ways to turn the approach to their advantage
BY KAREN EPPER HOFFMAN
As government agencies begin to hand over security to automated systems that can teach themselves, the idea that hackers could influence those systems is becoming the latest — and perhaps the greatest — concern for cybersecurity professionals.
Fortunately, researchers are fight- ing back through an approach called adversarial machine learning (AML), which focuses on creating machine learning algorithms that can resist such sophisticated attacks while also providing insight into attackers’ capabilities.
Nicolas Papernot, Google Ph.D. fel- low in security at Pennsylvania State University, said AML seeks to better understand the behavior of machine learning algorithms once they are deployed in “any setting where the adversary has an incentive, may it be financial or of some other nature, to force the machine learning algorithms to misbehave.”
“Unfortunately, current machine learning models have a large attack surface [because] they were designed and trained to have good average per- formance but not necessarily worst- case performance, which is typically what is sought after from a security perspective,” Papernot said. Therefore, those models are vulnerable to generic attacks that can often be conducted
regardless of the machine learning approach or the task being solved.
Yevgeniy Vorobeychik, an assistant professor of computer science and computer engineering at Vanderbilt University, said that although many agencies are “reaching a level of sophistication that we do not have,” AML is just beginning to emerge in the government sector. Some metropolitan and national law enforcement agen- cies are using it to forecast criminal activity, for example.
Tudor Dumitras, an assistant pro- fessor in the Electrical and Computer Engineering Department at the Univer- sity of Maryland, said machine learning has many public-sector applications, including “techniques for defending against cyberattacks, for analyzing sci- entific data such as astronomy obser- vations or data from large-scale experi- ments conducted by the Department of Energy, for biological and medical research or for building crime-predic- tion models used in parole and sen- tencing decisions.” All those systems are susceptible to attacks, he added.
To illustrate the problem, he used the example of cyber defense systems, which must classify artifacts or activi- ties — such as executable programs, network traffic or email messages — as benign or malicious. To do that, machine learning algorithms learn
models of malicious behavior based on a few known benign and malicious examples without requiring a predeter- mined description of those activities.
“An intelligent adversary can sub- vert these techniques and cause them to produce the wrong outputs,” Dumi- tras said. There are three ways adver- saries can do that:
• Attack the trained model by craft- ing examples that cause the machine learning algorithm to mislabel an instance or learn a skewed model.
• Attack the implementation by finding exploitable bugs in the code.
• Exploit the fact that users often have no knowledge of a machine learning model’s inner workings.
“As a consequence, users may not realize that the model has a blind spot or that it is based on artifacts of the data rather than meaningful features [because] machine learning models often produce malicious or benign determinations but do not outline the reasoning behind these conclusions,” Dumitras said.
Staying ahead of attackers
AML is becoming important in the public sector and law enforcement because computer scientists “have reached sufficient maturity in machine learning research for machine learning models to perform very well on many