Nature has an editorial regarding the threats which they fear will come from AI. They note:
Artificial intelligence (AI) is poised to revolutionize this
activity. Attacks and responses will become faster, more precise and
more disruptive. Threats will be dealt with in hours, not days or weeks.
AI is already being used to verify code and identify bugs and
vulnerabilities. For example, in April 2017, the software firm DarkTrace
in Cambridge, UK, launched Antigena, which uses machine learning to
spot abnormal behaviour on an IT network, shut down communications to
that part of the system and issue an alert. The value of AI in
cybersecurity was $1 billion in 2016 and is predicted to reach $18
billion by 2023. By
the end of this decade, many countries plan to deploy AI for national
cyberdefence; for example, the United States has been evaluating the use
of autonomous defence systems and is expected to issue a report on its
strategy next month. AI makes deterrence possible because attacks can be punished.
Algorithms can identify the source and neutralize it without having to
identify the actor behind it. Currently, countries hesitate to push back
because they are unsure who is responsible, given that campaigns may be
waged through third-party computers and often use common software.
The problem is not primarily the threats it is the fundamental architecture and the users.
First, the architecture uses the Internet. The Internet is a "public toilet". Anyone can use it and you have no idea what you may be exposed to. It was designed that way, as an open network with no security. It is why DoD abandoned the Internet in the late 1980s and went back to its own secure private networks.
Second, workers at companies are all too social. Send them an email and they open it, and then they set loose an attack from an attachment. They look at videos, many of which contain threats. It is estimated that over 90% of the network penetrations are facilitated by employees!
Thus we have a two prong attack strategy; a grossly insecure network and a collection of employees who have no idea what they are doing.
As for AI, after 40+ years of looking at it, I still do not know just what it is other that possibly an adaptive IF, THEN, ELSE set of statements. You can call it whatever you like, neural nets, adaptive processing etc but it still falls back on the primitive three statements.
Thus if one wants a secure network, do not use the Internet. I know it is expensive, but security is that.