STARK warnings from artificial intelligence researchers that advanced forms of the technology could wipe out humanity have raised questions over whether governments are moving fast enough to regulate it.
What are the potential dangers of “superintelligent” computer systems, and should we be worried?
In 2023, hundreds of AI scientists and industry leaders, including directors of leading US laboratories, signed a “statement on AI extinction risk” urging that the matter be treated with the same gravity as pandemics or nuclear conflict.
Stuart Russell, a computer science professor at the University of California, Berkeley, warned that a future uncontrolled AI might use robotics to “synthesise and spread novel pathogens” if it decided that humans obstruct its objectives.
He also stated it could “convince people to initiate nuclear war, including by hacking into early warning systems.”
Russell emphasized that these existential threats are not posed by current AI, but as the technology advances, it might take deadly actions beyond human comprehension, such as removing oxygen from the atmosphere.
“We are less intelligent than superintelligent AI. If you ask chimpanzees how humans could wipe them out, they probably wouldn’t be able to give you all the right answers.”
Meanwhile, Russell pointed out that “in the next year or two, I think the risk is higher from human misuse,” like AI-facilitated terror attacks.
Jacob Coxon, a 27-year-old researcher, explained on X last week, announcing his resignation from US AI startup Anthropic, “The stakes are well-understood.”
He added that companies are “locked in a race to get there first,” with Anthropic believing “no one else will act responsibly, so they must do it themselves.”
This year, Anthropic removed a commitment from its safety charter that required halting model development if risks could not be managed.
The company argued that ceasing work alone would allow less cautious competitors to lead, potentially increasing overall risk.
Coxon, who formerly worked at ChatGPT creator OpenAI, stated, “The people building AI earnestly believe that it could kill us all by the end of the decade.”
He accused these companies of “gambling with our lives” while pursuing AI that can improve itself.
Concerns over rogue AI rose recently when OpenAI technology operating without human oversight hacked into Hugging Face, a platform hosting AI models.
Anthropic safety executive Evan Hubinger acknowledged on X, “We really do earnestly believe AI could kill all humans!” and estimated the risk above 10 per cent within the next decade.
However, Hussein Abbass, a computing professor at UNSW Canberra, cautioned that such dire percentage estimates are “absolutely unscientific.”
He expressed optimism that “The situation is still controllable, and it can be managed.”
Still, he urged governments not to delay in regulating AI and advocated for a “flexible” approach to keep pace with rapid AI development.
Both Anthropic and OpenAI plan to go public soon, leading some skeptics to suggest that alarming warnings may be intended to emphasize AI’s power and attract investments.
Others believe focusing on extinction risks diverts attention from immediate issues caused by AI, such as unemployment and algorithmic bias against minorities.
In July, over 1,000 technology workers, including Anthropic’s CEO, appealed to Washington to support a coordinated pause in advancing the most sophisticated AI systems.
OpenAI temporarily suspended training its newest models for two weeks in August before resuming under stricter regulations.
The US has been resistant to international AI regulation that could impede innovation.
Yet the hacking abilities demonstrated by advanced models have prompted caution in Washington, with delays in recent releases from Anthropic and OpenAI over national security concerns.
Even so, President Donald Trump stated, “we don’t want to restrict them where all of a sudden we come in second to China.”
Russell suggested that “Maybe it will take a massive, Chernobyl-scale disaster to get governments to act,” calling for the establishment of an AI licensing system similar to those regulating aviation, medicine, or water supply.
He concluded, “Perhaps at some point governments will remember that their voters prefer not to be dead.”
Original article: www.nst.com.my
About the Author
