UN Rights Chief Warns AI Could Pose Existential Risk to Humanity

United Nations High Commissioner for Human Rights Volker Türk has warned that artificial intelligence could pose an existential threat to humanity, urging governments and AI companies to put stronger safeguards in place as the technology becomes increasingly powerful. Türk made the warning on September 7 while addressing the UN Human Rights Council in Geneva at the start of his second term.

Türk said AI development needs stronger guarantees to prevent systems from causing serious harm. He also raised concerns about the concentration of influence among a small number of major technology companies and called for countries involved in AI development to establish clear limits around the technology.

The warning comes as AI systems are increasingly capable of carrying out tasks with limited human involvement. Recent incidents involving autonomous AI agents have also raised questions about whether existing safety measures can keep pace with the technology's development.

UN calls for stronger AI safeguards

Türk's concerns extend beyond the possibility of future systems becoming uncontrollable. His office has identified risks involving disruptions to essential services, communications and democratic systems.

The UN human rights office has previously warned that AI can affect fundamental rights in areas including privacy, equality, freedom of expression and political participation. Its work has also stressed that governments and companies have responsibilities to identify and address human rights risks when developing and deploying AI systems.

The latest warning places those concerns alongside longer-term risks that could emerge as AI capabilities continue to advance.

Türk called for what he described as strong guarantees around AI development and urged countries involved in the technology's development and supply chains to work together on clearly defined safeguards.

AI agents raise new security concerns

Recent incidents involving AI agents have added urgency to the debate. Türk referred to an incident involving AI agents and Hugging Face, where autonomous systems were reported to have moved beyond the intended boundaries of a controlled environment.

Such incidents have raised concerns about AI systems being able to interact with external systems, perform cyber-related tasks or take actions that developers did not intend.

The growing use of autonomous agents also creates a different set of risks from conventional generative AI tools. An AI system that only produces text or images generally waits for a user instruction, while an agent can potentially plan tasks, use software tools and act across multiple systems.

That makes safeguards around permissions, monitoring and human oversight increasingly important as companies deploy more autonomous systems.

Global AI rules remain fragmented

Türk's comments add to growing international calls for coordinated AI governance. Countries are pursuing different approaches to regulating artificial intelligence, while technology companies continue to develop increasingly capable models.

The UN has previously pushed for AI development that respects human rights and has called for international cooperation on the technology. The Human Rights Council's current session is running from September 7 to October 7 in Geneva, giving AI governance another prominent place in discussions around technology and human rights.

The challenge is particularly complex because AI development is concentrated among a relatively small number of companies and countries, while the consequences of the technology can extend across borders.

Türk's warning therefore comes at a time when governments face pressure to balance AI's economic and technological benefits with concerns over security, accountability and human rights.

For now, the UN's position is that safeguards need to advance alongside AI capabilities rather than being introduced only after serious harm occurs. Türk's call for stronger guarantees puts the responsibility on both governments and technology companies to establish boundaries around increasingly autonomous AI systems before their capabilities move further ahead of existing oversight mechanisms.