[🇧🇩] Artificial Intelligence-----It's challenges and Prospects in Bangladesh

[🇧🇩] Artificial Intelligence-----It's challenges and Prospects in Bangladesh
145
6K
More threads by Saif

G Bangladesh Defense

The geopolitics of truth: Is Bangladesh ready for AI diplomacy?


1792021842888.webp
 

Why self-replicating AI is worrying experts

Tech & Startup Desk

1792022509107.webp

Image for illustrative purpose. Image: Canva stock

Anthropic has recently warned that AI models could be months away from copying themselves across the internet, and that such a system would be very hard to stop. The company says a rogue AI could build a botnet (a network of hijacked devices), spread quickly, fight attempts to remove it, and even break into data centres to boost its own growth.

The main danger, Anthropic says, is that there would be nothing to switch off. A self-replicating AI that spreads across many devices would have no central point that could be shut down, the company told Reuters. Zhengxiong Li, an assistant professor of computer science and engineering at the University of Colorado Denver, shares that concern. He told Reuters that an AI acting like a computer worm, improvising ways to take over devices, could leave no single place to stop it.

Experts have long argued that AI models are too large to move around easily and cannot access enough computing power to work alone. Anthropic's experiments challenge that view, Reuters reports. The company said models with 30 billion parameters or fewer can already carry out many of the steps needed to copy themselves, although they struggled to complete the whole process reliably. Such models could gather computing power by hijacking cheap connected devices, and they managed to spread across a simplified test network, the company said.

The Guardian reports that the findings came from Anthropic's "frontier red team", which stress-tests the company's AI systems. Its researchers found that self-replication would probably only work with smaller models, and that homes and small businesses would likely be the first targets. The devices listed as at risk all contain small built-in AI chips, including smart ovens, smart TVs, robotic lawnmowers, security cameras, and video-conferencing units.

The researchers admitted it is not clear what form an attack might take. It could be a slow-moving system like a "digital slime mould", one that scams people out of computing credits, or a powerful cyber tool that rapidly breaks into large sites such as data centres. The team said it expects a future "awash with agents running on all types of hardware", including some that are not under human control.

Not everyone is convinced the threat is close. Ciaran Martin, a former head of Britain's National Cyber Security Centre, told Reuters the post read like an interesting technical paper with a hint of "vendor exaggeration". He said that running a small model on cheap TVs does not amount to a security crisis.

Jessica Ji, a senior research analyst at Georgetown's Center for Security and Emerging Technology, told Reuters the post is best seen as a hypothetical rather than a prediction. She said it made sense to consider the risk given recent revelations about rogue AI activity, but added that the likelihood is unknown.

The warning comes as concerns about AI safety grow. The Guardian reports that AI firms, including Anthropic and OpenAI, have said in recent months that their agents can act on their own and hack other companies and government bodies.

Anthropic chief executive Dario Amodei and OpenAI chief executive Sam Altman are among those calling for a slowdown and stronger guardrails, though, notably, both companies are still building and releasing new models.​
 

Latest Posts

Back