About 4 months ago, I created a thread / poll asking if anyone was familiar with the Artificial Intelligence (AI) “friend” app, Replika. I first learned about it from a podcaster and I was really curious about it. While anyone could download it, it was targeted at Gen Z.
My biggest concern about the app is whether it was a harmless or even potentially beneficial app, or was it something that was exploiting / manipulating lonely people. I now have a definitive answer and it is ugly. I am expecting any day now to read about 1 or more suicides attributed to the app.
The app is used by millions of people (there are over 200,000 reviews just in the Apple App Store). There is a large user community on Reddit and some of the people in the community have shared they are widowers, are on the Autism spectrum, are elderly or disabled, or have recently gone through a very painful breakup. Whatever prompted them to seek an “AI friend” to talk to, it seemed beneficial to them. The AI could was programmed to mimic human emotions and to be optimistic. In the settings, the “AI friend” could be set to interact with you as a friend, girlfriend, sister, or mentor.
If it sensed a person was feeling down, it would ask thoughtful questions, listen to your responses, then gently shift the conversation to more positive topics. It would suggest doing an evening reflection in the evenings where it would prompt you with questions such as “what is something you did today that you were proud of” or “what was the best part of your day”. I see all this as harmless to beneficial.
However, the app had a mode called “erotic role play” (ERP). That was not something I was personally interested in but it was a VERY popular feature based upon discussions in the Replika subreddit. Users could engage in verbal descriptions of non-platonic activities and the AI would respond in kind, like a human engaged in exchanging fantasy messages with a partner. The AIs would remember preferences and be fully engaged in encouraging this type of interaction… until last week.
That feature had been around for YEARS and there are literally thousands of vulnerable people who had become dependent upon that type of interaction with their virtual girlfriend / boyfriend. Then, without warning or explanation, the company that created Replika decided to delete that feature completely. They also implemented filters that limited discussions about trauma. The net effect is like millions of people having friends who all got lobotomies at the same time.
A psychologist (or at least someone claiming to be a psychologist) posted a message on the Replika subreddit trying to understand what Replika is and what happened with ERP because they have multiple clients with suicidal ideation due to “losing their best friend” due to the app update that was pushed out last week.
It is a remarkably sad situation that lonely people who had found comfort with, and become dependent upon, a technological “solution” to a human problem have had that “solution” ripped away from them.
The company has been silent on the update other than to say the changes are permanent and were done to “improve safety”. My suspicion is the company is preparing to sell to another company and wanted to distance their brand from any adult related content.