Search papers, labs, and topics across Lattice.
This paper systematically analyzes the risks associated with the cognitive capabilities of agentic AI systems powered by large language models (LLMs), employing a three-level framework that spans physical, social, and self-referential cognition. The study highlights how these cognitive engagements can threaten human agency, autonomy, and control as LLMs become more integrated into various domains. Key findings include specific risks identified at each cognitive level and proposed strategies for mitigating these risks to enhance the controllability of such AI systems.
Expanding cognitive capabilities in agentic AI systems could jeopardize human autonomy and control, necessitating urgent risk mitigation strategies.
Frontier agentic systems powered by large language models (LLMs) exhibit human-like patterns of cognition. As these systems become deeply integrated across different domains, their cognitive engagement raises critical concerns for human society that remain insufficiently studied. To address this gap, we systematically analyze risks induced by expanding cognitive capabilities, following a three-level framework defined by their cognitive scope, from physical cognition to social cognition, and finally to self-referential cognition. We study their potential risks to human agency, autonomy, and control capability, corresponding to each cognitive level. We finally propose strategies to mitigate these risks and enhance the controllability of agentic AI systems, ensuring their long-term safe development.
