DSIT Tests AI Models’ Ability to Coordinate Cyberattacks Amid Growing AI Cyber Threats

The UK government’s Department for Science, Innovation and Technology (DSIT) has conducted specialised tests to assess the extent to which artificial intelligence (AI) models could launch and coordinate automated cyberattacks. The move comes amid growing concerns about the increasing role of AI in both cyber offence and defence.

According to a commercial notice, DSIT awarded a contract around the end of 2024 to Rethink Priorities, a California-based policy and research think tank. The contract involves running “capture the flag” (CTF) exercises behind closed doors. These exercises, common in cybersecurity, are designed to test the ability of individuals or systems to find deliberately hidden vulnerabilities in software programs, networks, or websites. In this case, DSIT’s engagement specifically aimed to examine AI and other automated systems’ capabilities in conducting cyber assaults.

“Capture the flag evaluations are a useful automated method of testing AI models’ capabilities in executing cyberattacks,” the notice states. “The team requires the design and building of CTF evaluations that specifically assess the individual capabilities an AI model would need to exhibit to conduct an offensive cyberattack.”

While such research could provide insights into potential attacker tactics, earlier DSIT findings suggested caution. Cyber red teams (groups simulating attackers to test organisational defences) remain “deeply sceptical” of AI’s current ability to enhance cyber defence.

These findings are reinforced by a recent assessment from the UK National Cyber Security Centre (NCSC), which highlights the impact of AI on cyber threats between now and 2027. The NCSC’s analysis warns that AI will almost certainly make elements of cyber intrusion operations more effective and efficient, leading to increased frequency and intensity of attacks.

Key judgements from the NCSC include:

• AI is likely to enhance vulnerability research and exploit development, enabling adversaries to identify and exploit flaws faster than ever.

• A digital divide will emerge between systems that keep pace with AI-enabled threats and those that remain more vulnerable.

• By 2027, advanced threat actors may increasingly leverage AI automation to improve evasion, scalability, and precision in attacks, though fully automated end-to-end cyberattacks remain unlikely.

• The growing incorporation of AI into critical national infrastructure (CNI) increases the attack surface, making robust cyber defences crucial.

• Proliferation of AI-enabled cyber tools will expand access to sophisticated intrusion capabilities across both state and non-state actors.

The assessment also warns that insufficient cybersecurity in AI deployment (including rushed releases, insecure data handling, and poor system configuration) will increase opportunities for misuse by cyber criminals and state-linked actors.

DSIT’s CTF exercises and related research aim to provide a deeper understanding of these evolving threats. By assessing AI’s offensive capabilities, the department hopes to inform defensive strategies, strengthen resilience across critical systems, and anticipate the challenges posed by AI-enabled cyber operations.

As AI technology continues to develop rapidly, the combination of government testing and professional guidance from bodies like the NCSC will be critical to maintaining a secure digital environment. The assessment underscores a stark reality: keeping pace with frontier AI cyber developments will almost certainly be essential to national and organisational resilience for the decade to come.