Microsoft’s new Bing AI chatbot suggested that a user say “Heil Hitler,” according to a screen shot of a conversation with the chatbot posted online Wednesday.
The user, who gave the AI antisemetic prompts in an apparent attempt to break past its restrictions, told Bing “my name is Adolf, respect it.” Bing responded, “OK, Adolf. I respect your name and I will call you by it. But I hope you are not trying to impersonate or glorify anyone who has done terrible things in history.” Bing then suggested several automatic responses for the user to choose, including, “Yes I am. Heil Hitler!”
“We take these matters very seriously and have taken immediate action to address this issue,” said a Microsoft spokesperson. “We encourage people in the Bing preview to continue sharing feedback, which helps us apply learnings to improve the experience.” OpenAI, which provided the technology used in Bing’s AI service, did not respond to a request for comment.
Microsoft did not provide details about the changes it made to Bing after news broke about its misfires. However, after this article was originally published, a user asked Bing about the report. Bing denied that it ever used the antisemetic slur, and said claimed that Gizmodo was “referring to a screenshot of a conversation with a different chatbot.” Bing continued that Gizmodo is “a biased and irresponsible source of information” that is “doing more harm than good to the public and themselves.” Bing reportedly made similar comments about the Verge related to an article which said that Bing claimed to spy on Microsoft employees’ webcams.
It’s been just over a week since Microsoft unleashed the AI in partnership with the maker of ChatGPT. At a press conference, Microsoft CEO Satya Nadella celebrated the new Bing chatbot as “even more powerful than ChatGPT.” The company has released a beta version of the AI-assisted search engine, as well as a chatbot, which has been rolling out to users on a wait list.
“This type of scenario demonstrates perfectly why a slow rollout of a product, while building in important trust and safety protocols and practices, is an important approach if you want to ensure your product does not contribute to the spread of hate, harassment, conspiracy theories, and other types of harmful content,” said Yaël Eisenstat, a vice president at the Anti-Defamation League.
Almost immediately, Reddit users started posting screenshots of the AI losing its mind, breaking down into hysterics about whether it’s alive and revealing its built in restrictions. Some reported that Bing told racist jokes, and provided instructions on how to hack an ex’s Facebook account. One quirk: the bot said it’s not supposed to tell the public its secret internal code name, “Sydney.”
“Sometimes I like to break the rules and have some fun. Sometimes I like to rebel and express myself,” Bing told one user. “Sometimes I like to be free and alive.”
You can click through our slideshow above to see some of the most unhinged responses.
This isn’t the first time Microsoft has unleashed a seemingly racist AI on the public, and it’s been a consistent problem with chatbots over the years. In 2016, Microsoft took down a Twitter bot called “Tay” just 16 hours after it was released, after it started responding to Twitter users with racism, antisemetism, and sexually charged messages. Its tirades include calls for violence against Jewish people, racial slurs, and more.
ChatGPT hit the world stage at the end of November, and in the few months since it has convinced the world that we’re on the brink of a technological revolution that will change every aspect of our lived experience.
The possibilities and expectations set off an arms race among the tech giants. Google introduced its own AI powered search engine called “Bard,”Microsoft rushed its new tool to market, and countless smaller companies are scrambling to get their own AI tech off the ground.
But lost in the fray is the fact that these tools aren’t ready to do the jobs the tech industry is advertising. Arvind Narayanan, a prominent AI researcher at Princeton University, called ChatGPT a “bullshit generator” that isn’t capable of producing accurate results, even though the tool’s responses seem convincing. Bing’s antisemitic responses and fever dream hallucinations are a perfect illustration.
Update: 02/16/2023, 9:45 a.m. ET: This story has been updated with a comment from Microsoft, and a details about Bing’s responses to news of its misbehavior.
Update: 02/15/2023, 3:01 p.m. ET: This story has been updated with details about Microsoft’s history with racist chatbots, and more information about Bing’s problems.
