AI Replication Warning: Are Our Devices at Risk?

"A replicator like this would be challenging to shut down," Anthropic said.

Anthropic said in a new warning that small AI models may be months away from being capable enough to self-replicate on devices such as smart ‌ovens and off-brand Android TV boxes.

Anthropic’s “red team” -- the team that investigates potential security and alignment issues with the models -- looked at whether AI models could find vulnerable devices, copy themselves into those devices and continue spreading without human help.

“Our experiments suggest that small AI models may be months away from being capable enough to self-replicate on commercial internet-of-things (IoT) devices, like smart ovens and off-brand Android TV boxes,” Anthropic said Monday on its website. "A replicator like this would be challenging to shut down, since there’s no centralized point to shut off.”

Anthropic ⁠warned these small models might be able to autonomously replicate by the end of this year, with small ⁠weight-based replicators following suit soon after.

Anthropic also said such autonomous AI systems were likely to replicate slowly and rely on widely ⁠available, relatively insecure computing resources.

“We should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay,” Anthropic said.

This is a developing story. Please check back for updates.