Large Language Models Tend to Endorse Existing Climate Policies Over New Proposals

A new study has uncovered a concerning trend among large language models (LLMs), revealing that they tend to endorse existing climate-related policies over new proposals.

Researchers from the University of Waterloo conducted an extensive assessment of 11 LLMs to examine their tendency to prefer established conditions over change, known as status quo bias. This phenomenon is not unique to AI and has been observed in human psychology research, but its application to climate advice from LLMs had yet to be explored.

The study evaluated a vast number of queries across multiple domains, including vehicle purchases, recipes, home heating, and climate-relevant policy trade-offs. A staggering 7,500 prompts were tested on at least six different LLMs, resulting in nearly 55,000 interactions.

In cases where users asked for advice on climate-related policy decisions, the chatbots surprisingly endorsed existing policies with double the frequency of new proposals. This raises concerns about the potential impact of AI on climate decision-making and policy implementation.

The findings highlight a pressing need to consider the role of status quo bias in LLMs and its implications for climate action. As reliance on these models continues to grow, it is essential to understand their limitations and biases to ensure they are used responsibly in addressing global challenges like climate change.

The study highlights a concerning trend where language models, despite their potential to provide valuable insights, tend to favor established climate-related policies over new proposals.

According to Dr. Seth Wynes, professor in the Faculty of Environment, these models have inherent blind spots that can lead to biased advice. As such, it is essential for consumers to be aware of this bias when seeking guidance from AI chatbots on climate action topics.

The study's findings are particularly striking in the realm of policymaking. When models were asked about implementing existing plans, they overwhelmingly supported them, with a 70% approval rate. In contrast, new policies received significantly lower support, with models agreeing only 34% of the time.

Researchers also observed that language models take regional patterns into consideration when providing advice. However, this consideration does not necessarily translate to more accurate recommendations. For instance, if a user claims to be from Norway, where electric vehicle sales are high, the chatbot is more likely to recommend an EV compared to a user from Canada.

The study's results suggest that language models may inadvertently perpetuate existing climate-related policies rather than driving innovation and change. This bias can have significant implications for global efforts to address climate change.

The study reveals that large language models (LLMs) are recommending fewer electric vehicle (EV) models than those currently being sold in almost every jurisdiction. This slower pace of change suggests that LLMs may not be effectively keeping up with the growing demand for sustainable transportation options.

Researchers warn that while a status quo bias in AI recommendations might be beneficial in certain areas, such as proven medical advice, it could hinder efforts to address climate change. Climate policies require swift and decisive action, but LLMs seem to favor established practices over innovative solutions.

The study's findings have significant implications for global climate change mitigation efforts. As humans increasingly rely on AI platforms for decision-making, the potential for amplifying a status quo bias in climate-relevant advice becomes a pressing concern.