Language Accuracy Concerns Emerge with AI Chatbots Ahead of Midterms

A growing concern is emerging about the accuracy of information provided by AI chatbots to non-native English speakers ahead of the midterms.

The issue lies in the fact that AI models are predominantly trained on English language data, resulting in a lack of understanding and nuance when it comes to other languages. This can lead to inaccurate or ambiguous responses to questions posed by voters whose native language is not English. Researchers and community activists warn that this could create a barrier for some voters.

Chatbots may provide overly literal translations of electoral terms, lacking context and failing to capture the subtleties of various languages' unique dialects. This can result in confusion about crucial information such as voting procedures, including whether or not one can vote by mail in their state. The accuracy of this information is critical for voters.

Community leaders are sounding the alarm that Latino communities, in particular, may be disproportionately affected by this digital divide.

The digital divide in language proficiency is creating a significant challenge for non-English speaking voters across the country. With over 20% of Americans speaking a language other than English at home, this discrepancy in accuracy can unintentionally disenfranchise these individuals.

National voter organizations serving Latino and Asian American communities are racing to bridge the gap between their constituents and the rapidly evolving technology. They face the daunting task of keeping up with AI advancements while ensuring that their communities don't get left behind in the process.

Mi Familia Vota, a leading organization overseeing large-scale Latino voter mobilization efforts, has developed its own AI assistant called "Nubi". This tool is designed specifically for Spanish speakers seeking information on voting, voter registration, and citizenship.

The accuracy of AI chatbots has been found to decline significantly when prompted in Spanish about voter information. A study released in September by the Institute of Strategic Dialogue discovered that this language gap results in a 16% drop in accuracy.

Researchers noted that Spanish prompts often produce incomplete or ambiguous answers, frequently omitting crucial details and relying on literal translations. This can have serious consequences for voters seeking accurate information on registration deadlines or ID requirements.

The study tested six frontier models through their respective APIs, which are available for other developers to use. However, experts point out that these interfaces do not accurately represent how consumers interact with chatbots in real-life scenarios.

Some tech companies, including OpenAI, Anthropic, and Google, have acknowledged the issue and are working to refine their models for non-English languages.

To address concerns over election-related information in non-English languages, leading AI companies are taking steps to improve their offerings for a broader range of voters.

OpenAI and Anthropic are integrating official election data from Democracy Works into their responses. This move aims to provide more accurate information on elections in various languages, particularly those with smaller speaker bases such as Swahili, which has around 150,000 speakers.

The goal is to support people's ability to participate freely in elections by providing reliable and transparent information, according to OpenAI spokesperson Nate Evans. However, some organizations are choosing to avoid AI technology due to concerns over inaccuracies.

Asian Texans for Justice, a nonprofit that works with Asian American communities, has made the decision not to use AI for their in-language outreach efforts ahead of the midterms. Executive director Nabila Mansoor cited inaccuracies as the reason behind this choice, despite acknowledging the growing presence of AI technology within these communities.

The growing presence of AI technology within communities is being hindered by a language gap that could have significant implications for voters. This barrier is not due to technological limitations, but rather the fact that many languages are underrepresented in AI models.

Community organizers emphasize that changes need to happen quickly, as immigrant communities require access to reliable information for civic engagement, particularly during elections. Mi Familia Vota's Barba notes that the way people receive information has fundamentally shifted and civic engagement must adapt accordingly.