“You are going to need algorithms that you trust, algorithms that you understand, and ones that you can let run and not really have concerns about,” said Stacey Dixon, the newly appointed head of the Intelligence Advanced Research Projects Activity on the “Intelligence Matters” podcast. During the interview, Dixon said machine learning was “extremely important,” to the intelligence community’s research portfolio.
According to an analysis by Fifth Domain, more than half of the new challenges and programs announced by IARPA in 2018 involve machine learning or predictive analytics.
Dixon was named the head of IARPA Aug. 14, the same day the podcast was released. She was previously agency’s deputy director and replaces Jason Matheny.
Dixon said the agency has invested in predictive analytics programs which “forecast that someone is going to be attacked through cyber means.” The approach uses “sensors” – physical or analytical – that can predict which industry or entity is vulnerable to cyberattacks, she said.
Research related to the program has already spurred several forecasting models to predict cyberattacks. One automated system was based on conversations of malicious actors on the dark web and posts from cybersecurity experts on social media. It was found to be 84 percent accurate when it came to predicting cyberattacks. Another predictive model was based on historical cyberattack data. It was found to be 14 percent more effective than other systems.
But the new IARPA chief also warned that predictive tools need to be honed to avoid false positives and negatives.
“If you’re chasing down things that really aren’t going to happen all the time, its really not a very good tool. So how do you balance the sensitivity of the tool with not over predicting things that are going to happen such that nobody really believes the predictions,” Dixon said.
She also said that creating an audit trail for how predictive models make their decisions will boost trust with analysts.
“Not understanding how something came up with an answer is a very uncomfortable place to be in,” Dixon said.
Meanwhile, Dixon also warned that the U.S. was losing out to China in at least two fields of innovation.
Billion-dollar projects in artificial intelligence and synthetic biology by the Chinese “is scary because those are large investments that can really shift what research is accomplished,’ said Dixon. “We are sort of losing some of the talent that would have stayed in the U.S.”