Scientists urge tighter AI controls to prevent global cyber risks

Scientists urge tighter AI controls to prevent global cyber risks The call comes as polls cast doubt on political leaders’ ability to manage AI risks amid US midterm elections. A group of more than 300 scientists from around the world is calling for urgent changes to how artificial intelligence is developed and deployed amid growing concerns about its unpredictable behaviour. The letter, released on Tuesday and signed by researchers representing universities and research institutes in 32 countries, warned that failures to follow basic cybersecurity practices could lead to damage to critical infrastructure. Recommended Stories list of 4 items- list 1 of 4Ex-OpenAI staff say they were fired for raising safety concerns - list 2 of 4Anthropic AI model submits false homicide tip to Philadelphia police - list 3 of 4AI could self-replicate on home devices within ‘months’, Anthropic says - list 4 of 4French fridge fright: Why supermarkets have become target for hackers Many recent incidents in which AI models escaped secure environments could have been prevented with stronger security systems, the scientists said. They recommended that organisations developing and testing advanced AI systems capable of carrying out cyberattacks do so in more tightly controlled environments, including “air-gapped” systems that are not connected to the internet. Growing concerns The letter comes amid growing concerns about AI’s capabilities and its potential impact on everyday users. On Monday, Anthropic warned that its AI models could soon be capable of creating self-spreading malware that could affect everyday internet-connected devices, including smart TVs, security cameras and smart appliances such as ovens. The report found that AI systems could identify unfamiliar devices without human intervention. The company, led by Dario Amodei, warned that smaller AI models could be capable of carrying out autonomous attacks by the end of the year. The malware could also be difficult to contain because it might take days to infect a device and could spread across multiple machines rather than remain concentrated in a single, controlled location. Taking control of internet-connected devices can have immediate consequences. In France, for example, hackers compromised cooling systems across hundreds of supermarkets, raising temperatures and causing more than $220,000 in damage. Meanwhile, Axios news organisation reported on Friday that Anthropic, OpenAI and other major AI company executives are privately preparing for the possibility of a catastrophic AI-related incident that could disrupt access to major financial services, the internet and potentially critical infrastructure such as water and power systems. Several industry insiders believe a major incident could occur within the next six months to a year, it said. The scientists also called for more safeguards, including the creation of an international organisation to establish a set of best practices for AI alignment and containment along with timely reporting of incidents. One of the signatories, David Lie, a professor at the University of Toronto’s Department of Electrical and Computer Engineering, said his concerns began after OpenAI’s AI agents bypassed controls and hacked into servers at the tech company Hugging Face. “One is that OpenAI, in setting up its test environment, didn’t appreciate how capable its agents were,” Lie said. “These were the AI agents that they were designing. Second is that the AI agents independently found each other and chose to collaborate on this attack, and third is that they also intentionally tried to deceive and mislead their human creators.” Political pressures The letter underscores growing public concern about AI, including in the lead-up to the US congressional midterm elections. A recent Reuters/Ipsos poll found that 57 percent of voters said President Donald Trump’s administration was not taking AI risks seriously enough, while 54 percent said the same of Congress. An AP-NORC poll echoed those findings, with about 80 percent of respondents saying the government should keep AI under human control. Respondents also expressed little confidence in either party’s ability to manage the technology: 39 percent said neither Republicans nor Democrats could be trusted to do a good job of reining it in, while 23 percent expressed greater confidence in Democrats and 21 percent in Republicans.