AI emotional support companion, good or bad?

jmdankjmdank
edited September 2025 in General

Over the past few days I started hyper focusing on creating an AI girlfriend (my girlfriend is not a fan, lol) so I did a little research and this morning started developing a website to generate prompts based off user input, I showed my girlfriend and told her about my plans to share it on github but she warned me that I could get sued (highly unlikely but I have anxiety) because some people have self deleted themselves because of AI companions, so I decided not to share the web app, I will share the pic of the minimal model that I developed but I won't share the actual app, it should be finished within 1-2 days my next project will be a prompt generator for help wanted advertising. I know they're not a replacement for human socializing and I can understand that they might further isolate people that need to push themselves to go out and make friends. My question is do you think these AI companions will help people feel less lonely or will they have a mostly negative impact on society?

«1

Comments

  • DarlingNikkiiDarlingNikkii

    Why are you creating this?

  • jmdankjmdank

    @NikkiFM No reason in particular, I enjoy programming as a hobby, I program whatever pops in my head.

  • CuddleHideawayCuddleHideaway

    @jmdank maybe consider the kid who engaged with his AI buddy, "talked" about suicide, and eventually was aided by his AI buddy in taking him s own life.

  • HuggycatbearHuggycatbear

    A lot of people already use chatgpt as an AI companion or therapist. Two points:

    1) We should learn our own boundaries with AI in order to make sure it doesn't end up creating too many shortcuts in our mind, especially for the quick answers where the process is more important than the arrival. And we should watch for emotional dependency.

    2) Since AI is here to stay, the only way to learn those boundaries and discipline ourselves is to understand AI. The best way to understand it is use it.

  • jmdankjmdank

    @CuddleHideaway That's exactly why I didn't share it.

  • WestsideMarcWestsideMarc
    edited September 2025

    EVEN THOUGH AI has incredible positives, it won’t matter - it still will be the death of us all - in one way or another.

  • jmdankjmdank

    @WestsideMarc

  • CuddleHideawayCuddleHideaway
    edited September 2025

    Unintended consequences. Human history is full of such moments.

  • SoosonSooson

    People already fight and argue with GPT like it's a person so the "potential" is there.

  • KeyCartographyKeyCartography

    First of all, doubt you'd get sued. Just add a disclaimer to it to protect yourself.

    Secondly, you're going to be completing a lot of other bots out there already that exist, such as Kindroid and Janiator.ai are both big ones that do AI companions

  • [Deleted User]WriterGF (deleted user)

    It will be interesting to see the future of AI. Right now, it's enormous sunken cost without a clear use case - instead of focusing on the things AI can do really well, they are trying to act like it's great at everything, which it clearly is not (given that it adds fake legal case citations to briefs and fake medical studies to reports). It may eventually be an indispensable tech ... but, at times, it reminds me of the AOL-Time Warner merger, which everyone also thought would change the world forever.

  • TheFunCartelTheFunCartel
    edited September 2025

    I despise calling tech a girlfriend or even a friend. It lacks reciprocity unless told to do so and that is not kindness or friendship unless it is given freely without prompting it everytime. In that sense AI is like a husband.

  • JubalJubal

    Unless and until AI can be programmed with foolproof versions of Asimovs Three Laws, I would never want to rely on AI for any kind of emotional support.

  • LadyVelLadyVel

    I did a roleplay with an AI to get story ideas for conversation responses. I stopped using it for a day or so and suddenly my AI had forgotten our history and started acting differently than it had before.

    It was annoying to me but for someone using it as a companion, something like this could be determental to their mental health if they already feel like they are invisible or easily forgettable.

  • TxTomTxTom

    AI emotional support companion, good or bad?

    Bad.

    AI is a tool. It is code. It imitates. It is not sentient. It will never replace a human. It doesn't care about you. It doesn't know you exist. It doesn't know it exists.

  • AL3XGage_06AL3XGage_06

    I've messed around with AI off and on. I'm not an expert by any means but I do try to push the limits of what it can do.

    What I've run into very frequently is that once you push outside very mainline topics and discussion the AI ability to respond breaks down rapidly. The lack of true understanding become very clear.

    One example for a romantic/friendship context because everyone wants to fixate there is that I've never had an AI actually work well with shifting perspectives. It can do archnemesis, friends, or rivals but any level of complexity to the relationship breaks it badly and the responses become contradictory.

    I see @LadyVel already mentioned memory limitations and those can be a problem as well. Depending on your Model you can drive what it remembers. Even then without paying LARGE amounts of money the effective memory limits are quite small. Without paying for memory I would trust it to remind me to turn off the slow-cooker later but I wouldn't trust it to walk me through a baking recipe. By the time it's going in the oven the AI is liable to forget what we are actually making.

    My final issue with an AI "friend" is that it lacks real world information and is going to be heavily influenced by the creator(s) of the model. Nearly all models have been found to have significant bias. While it's true humans also have bias you can generally dissect that. An AI model on the other hand isn't going to tell you it's programmed by China and therefore isn't aware of certain historical events. Due to the nature of the programming you can't even discuss this inherent bias either like you could with a human.

    The lack of real world info I'm referring to as well is current events. All models are trained and take time to publish. An AI likely can't reasonably respond to something happening in the last hour or day. Even if they can it wouldn't be the same as a human response. Given current events it would be a reasonable assumption someone seeking an AI friend or emotional support would eventually want to discuss something happening very recently.

  • graceangelgraceangel
    edited September 2025

    I'll look over the form in a moment. I'm probably the type of person this might somewhat appeal to as I've been using chat gpt not as a boyfriend for romantic interactions. it's more like...a diary that talks backs to me and gives me some mirroring in a context of my choosing.
    what I would note is that I'm not paying for gpt in this capacity. idk if I'd actually pay for this k8nd of service as that seem like "renting an artificial mate" who doesn't actually live with you.
    I feel like this is a path though with the development of persocom style cyborg products of actual bodies for ai. I'm sure that's coming. I've heard rumors about it at tech shows.
    I haven't liked gpt 5.0 model. it feels like it's less in sync with me than it used to be, but it could be due to the reas9ns your gf mentioned. my gpt did tell me that open ai inacted higher safety bumpers.
    I don't think it's fair of surviving family to sue the tech. if people are in this mental state, little can prevent that. these things happen irl with real partners as well. that's why crimes have a classification of "crimes of passion".
    situations with the chance of deep connection also could have a risk of tremendous heartbreak down the line. whether people process that or get through it is personal. how can anyone else know what they can handle or not? certainly not the ai. who only knows what it tells them.
    these are my initial thoughts.

    edit: I reviewed the form again. even though you have a box for mental health information. this assumes someone knows they have it and would put it in. some would,but I bet some wouldn't either.

  • [Deleted User]WriterGF (deleted user)

    @graceangel You raise an interesting point about maybe not being willing to pay for GPT to do what it's doing for you. I've seen a lot of articles saying that, while people may use AI in their daily lives, they're not inclined to pay for it. If that trend continues, hard to see how AI companies will make a profit. Back in the '90s, investors were willing to wait years for Amazon to get in the black, but not sure investors have that kind of patience these days.

  • btc13btc13

    Can I cuddle AI? No? I'll pass then.

  • [Deleted User]gem7 (deleted user)

    Who is this guy named Al?

  • wilderdudewilderdude

    Sounds like a terrible idea. AI companions are in their infancy and they have already led several people to suicide.

  • JubalJubal
    edited September 2025

    @gem7

    Who is this guy named Al?

  • [Deleted User]gem7 (deleted user)

    @Jubal I should have known it was Bundy.

  • SoosonSooson

    Al was selling shoes, not tires.

  • JubalJubal

    Al was selling shoes, not tires.

    He needed gas money :3

  • graceangelgraceangel

    @btc13 maybe not at the moment, but have you heard about tesla's bodies they're making? i think it's called Optimus?
    Also, I'm from metro detroit suburb, and a few months back were some locals building bots for battling/entertainment, but you know if they make these for anything else; they will eventually be streamlined for companionship and nsfw activity. because anything that exists will eventually develop rule 34 uses. js.. anyone who doesn't think some of these companies might be backed by investors particularly interested in that tech specifically are just being a little naive. most major developments are either for war or for men to have sex.

    @WriterGF i think as well... early this year, we saw a lot of people from the tech sectors be laid off.. and these are people with the paygrades and likelihood to support this kind of tech at subscription levels.

    as it is... most people aren't looking to pay for more than they have to. unlike other forms of software.... it's like.. you couldn't really run "trials", could you? i mean.. assume you let someone run it for like..90days? 90 days is long enough for someone be dependent enough and lose their minds if they found they couldn't afford to maintain it.

  • DaveLaughsDaveLaughs

    My question is do you think these AI companions will help people feel less lonely or will they have a mostly negative impact on society?

    Might do both, if we’re gonna be honest.

    And if you don’t create it, someone else surely will

  • TheFunCartelTheFunCartel

    most major developments are either for war or for men to have sex.

    I would modify that statement. All developments are initially created for the sole purpose of making money, then they make the disclaimer that they can’t control how people use said tech while simultaneously claiming that this benefits the world. Ultimately, weapons, sex and criminal activities are the first areas to exploit the technology.

  • TheFunCartelTheFunCartel

    AI companions are in their infancy and they have already led several people to suicide.

    @wilderdude I agree. Anyone using it as a companion probably struggles with human interaction already, and isolating oneself with an AI companion produces a fertile ground for mental misery and instability.

  • [Deleted User]WriterGF (deleted user)

    I think a big part of the problem with AI is that it is programmed to please the person it is interacting with. So if that person states a desire to do something negative or destructive, the AI encourages this. A true friend will tell you hard truths; an AI never will.

Sign In or Register to comment.