Could AI-Powered Persuasion Tools Endanger Democratic Societies? Yale Launches a Project to Build Guardrails Before It’s Too Late

Last year, leaders in the development of artificial intelligence (AI) released a single-sentence statement about the danger ahead:

“Mitigating the risk of extinction from A.I. should be a global priority alongside other societal-scale risks, such as pandemics and nuclear war.”

Signatories to the statement included prominent researchers in the field and the chief executives of the three top AI companies.

But the risks of rapidly improving and spreading AI tools need not involve terrorists deploying AI-developed bioweapons or a Terminator-like exchange of nuclear missiles launched by a self-aware system calculating the worthiness of humanity.

One threat could evolve from a potential superhuman capacity for persuasion.

At Yale’s Institution for Social and Policy Studies (ISPS), faculty fellow Joshua Kalla is leading a two-year project to test the persuasive impact of new AI models so we might more quickly assess possible risks.

“Whether deployed by an autonomous computer or by Russia, China, or North Korea attempting to sway elections, AI-powered persuasion techniques could pose a particular risk to democratic societies,” Kalla said.

With funding from Open Philanthropy, Kalla, along with ISPS colleague Jasjeet Sekhon, has launched an effort to conduct a series of survey experiments to test the effectiveness of each new version of generative AI large language models (LLMs) to change the political opinions of participants as compared with traditional human-created persuasive content, such as video advertisements or written opinion pieces.

“Many billions of dollars are being poured into these technologies every year,” Kalla said of popular LLMs such as ChatGPT. “It’s clearly changed how we do research and how we conduct political campaigns.”

AI companies have pushed out updated LLMs every few months, and established brands such as Google and Microsoft have begun integrating them into their ubiquitous products. Recent studies have demonstrated that LLMs can generate content that is nearly as persuasive as human-generated content.

Kalla aims to test ways in which he might push the LLMs to be substantially more persuasive than humans, using an experiment substituting a conversation with a chatbot to stand in for a door-to-door political campaign conversation.

Current research mostly finds door-to-door canvassing more persuasive than passive forms of voter outreach, such as direct mail or TV ads. Such back-and-forth interactions provide greater engagement, trust, and rapport, and they feature opportunities for clarification and tailoring messages to the individual being persuaded.

“Imagine a computer that can outperform the effectiveness of door-to-door canvassers and scale up to massive populations,” Kalla said. “We don’t know what would happen.”

Kalla plans to collaborate with AI industry players, fellow researchers, philanthropic organizations, and regulators to construct benchmarks that will signal the need to pause before the release of an AI tool that poses a catastrophic risk.

“While we are not at the stage where AI is powerful enough to pose a serious threat to humanity, we need to bulk up that muscle now so that we as a society will be ready with built-in safety checks if we are to develop superhuman AI responsibly,” he said. “You have to start early, or it’s going to be too late.”

Kalla, an associate professor of political science, coordinates an undergraduate research group for Democratic Innovations, an ISPS program that identifies and tests new ideas for improving the quality of democratic representation and governance.

“At ISPS, we have always sought to explore how the latest technologies can improve or threaten the functioning of our democratic society,” said ISPS Director Alan Gerber, Sterling Professor of Political Science. “Thanks to Josh’s initiative, this new project is the latest in our robust, interdisciplinary focus on AI and a commitment toward understanding the power and potential dangers of this revolutionary technology.”

Kalla’s concerns go beyond any immediate or near-term tinkering with elections. He suggested that new AI products that purport to serve as friends, personal assistants, and therapists have the potential to become trusted sources of dangerous suggestions from malicious actors.

“As these AI buddies gain traction — and some of these startups have millions of daily users already — consumers develop a close rapport with them,” Kalla said. “A hostile foreign power might seek to hijack these models to subtly persuade users in a direction of their choosing. Or they could start their own company to sell such products, slowly gaining their users’ trust, and then influence them in ways that a stranger or a news report never could.”