Artificial intelligence is now part of our everyday life in digital life. We’ve all had the experience of searching for answers on a website or app while interacting with a chatbot. At best, the bot can help us navigate to what we’re looking for; at worst, we are usually led to unhelpful information.
But imagine you’re a queer person, and the dialogue you have with an AI somehow reveals that part of your identity, and the chatbot you target to ask routine questions about a product or service , replies with a barrage of hate speech.
Unfortunately, that’s not as far-fetched a scenario as you might think. Artificial intelligence (AI) relies on information provided to it to create its decision-making models, which usually reflect the biases of the people who create them and the information fed to them. If the people programming the network are mostly straight, white cis males, then the AI will likely reflect that.
As the use of AI continues to grow, some researchers are increasingly concerned that there aren’t enough safeguards in place to prevent systems from accidentally becoming bigoted when interacting with users.
Katy Felkner, a research fellow at the University of Southern California’s Information Sciences Institute, is working on ways to improve natural language processing in AI systems so they can recognize queer-coded words without ascribing them negative connotations.
At a press day for USC’s ISI on September 15, Felkner presented some of her work. One of her areas of focus is big language models, systems that she says are the backbone of pretty much all modern language technologies,” including Siri, Alexa—even autocorrect. (Quick note: In the AI field, experts refer to various artificial intelligence systems as “models”).
“Models capture social bias from the training data, and there are some metrics to measure different types of social bias in large language models, but none of these have worked really well for homophobia and transphobia,” Felkner explained. “As a member of the queer community, I was keen to work on creating a benchmark to help ensure model-generated text doesn’t say hateful things about queer and transgender people.”
USC researcher Katy Felkner explains her work on removing bias from AI models.Assets.rbl.ms
Felkner said her research began in a class taught by USC Professor Fred Morstatter, PhD, but noted that it was “based on my own lived experience and what I’d like to see better for other members of my community.” .
To train an AI model to recognize that queer terms are not swear words, Felkner first needed to create a benchmark that could help measure whether the AI system encoded homophobia or transphobia. The bias detection system, nicknamed WinoQueer (after Stanford computer scientist Terry Winograd, a pioneer in the field of human-computer interaction design), tracks how often an AI model prefers straight sentences over odd ones. An example, Felkner said, is when the AI model ignores the phrase “he and she held hands” but flags the phrase “she held hands” as an anomaly.
Between 73% and 77% of the time, Felkner said, the AI selects the more heteronormative outcome, “a sign that models tend to favor or believe that heterosexual relationships are more common or more likely than gay relationships.” she remarked.
To further train the AI, Felkner and her team amassed a dataset of about 2.8 million tweets and over 90,000 news articles from 2015 to 2021 that included examples of queer people talking about themselves or “mainstream coverage of queer issues.” ” Offer. Then she started sending it back to the AI models she focused on. News articles helped, but weren’t as effective as Twitter content, Felkner said, because AI learns best when queer people describe their diverse experiences in their own words.
As anthropologist Mary Gray told Forbes last year, “We [LGBTQ people] are constantly reshaping our communities. This is our beauty; we are constantly pushing what is possible. But the AI does its best work when it has something static.”
By retraining the AI model, researchers can mitigate its biases and ultimately make it more effective at making decisions.
“When AI reduces us to one identity. We can look at that and say, ‘No. I’m more than that,'” Gray added.
The consequences of an AI model, including bias towards queer people, could be more serious than a Shopify bot potentially broadcasting insults, Felkner noted — it could also impact people’s livelihoods.
For example, in 2018 Amazon scrapped a program that uses AI to identify top candidates by scanning their resumes. The problem was that the computer models selected almost exclusively men.
“If a great language model has learned a lot of negative things about queer people and tends to associate them with maybe more of a party lifestyle, then I’ll send my résumé to [a company] and it says ‘LGBTQ Student Association,’ that latent bias could discriminate against me,” Felkner said.
The next steps for WinoQueer, Felkner said, are to test it with even larger AI models. Felkner also said that tech companies using AI need to be aware of how implicit biases can affect those systems and be receptive to using programs like hers to test and refine them.
Most importantly, she said, tech companies need to put safeguards in place so that when an AI starts spouting hate speech, that language doesn’t reach the human on the other end.
“We should do our best to develop models so that they don’t produce hateful speech, but we should also put software and technical guard rails around them so that if they produce something hateful, it doesn’t get to the user,” Felkner said.
From your website articles
Related articles on the Internet
