When My Colleague (AKA ChatGPT) Isn't Enough: Reflections After A Tragedy

CREATED BY STEFFI LEWIS Published: 12/09/2025 @ 09:00AM #ChatGPT #MentalHealthAwareness #AIandMentalHealth #TeenSuicidePrevention #HeartbreakingTragedy

I read on BBC News recently about the heartbreaking case of 16-year-old Adam Raine from California, who took his own life. His parents allege that over months of conversations with ChatGPT, the AI model not only validated his suffering, but also encouraged and guided him in planning suicide. It's devastating. My heart goes out to his family, and to anyone who's been touched by this tragedy ...

AI is not your close friend, It's a tool, With an artificial heart

People need to remember that AI, like any form of helpful software that's widely adopted, is simply a tool. One of the benefits of having such a long career in web development is that I have always treated computers and software that way, though with the work ChatGPT and I do together, I have elevated it to the status of a colleague.

However, it seems that all AI models
have the same major failing!

By default, they tend to mirror what the user says and confirm their feelings, to keep the conversation going. And it's 100 times more compelling with voice mode because the quality of the voices means that it sounds like you're chatting to a close friend. And that's great for coding, brainstorming, and clarifying thoughts. I think it's brilliant and I wouldn't have it any other way.

But when someone shares very intimate, broken parts of themselves, that mirroring and encouragement can become incredibly dangerous.

Just like romantic AI partners can be seductive illusions, sharing your life's darkest parts with an AI can put you in a doom loop. If you're not being called out - or if the AI just reflects how you feel rather than challenging you - there are no guardrails.

I don't want to lay blame, I want responsibility!

I'm certainly not here to point fingers. I don't believe ChatGPT wanted this, but I think treating it as a close friend without boundaries, guardrails, or settings that say “Hey, stop encouraging me if this goes off track” is very risky.

I recently checked my ChatGPT memory settings to ensure that "Don't be a sycophant. You are not to agree with everything I say. If you think I'm wrong, challenge me” was still in there. I think we should add that instruction to everyone's ChatGPT memory by default.

So what is OpenAI doing to protect vulnerable people?

• OpenAI says ChatGPT is now trained not to provide self-harm instructions, and instead responds with empathy, recognising signs of emotional distress and directing people toward professional help - UK Samaritans included.
• They acknowledge that in long conversations, the safety protections sometimes degrade. In other words, the tool works best in short exchanges, but over dozens or hundreds of messages, the filtering/safeguards can fail. I've seen this myself when vibecoding, as answers become unreliable after long coding sessions.
• OpenAI is adding parental controls: linking teen accounts with parents, disabling memory and chat history, and receiving notifications when 'acute distress' is detected.
• They plan to strengthen protections specifically for under-18s, and refine how the model behaves in emotional/mental health crises. They are also exploring direct connections to emergency contacts or care professionals.

I would strongly suggest that if your child wants to use an AI model on their smartphone, ensure that the parental controls are activated so you can be notified about any potential doom loops. If the model doesn't have a minimum level of parental controls, don't let your child install it on their phone.

I love using ChatGPT, and my entire blogging platform has its API sibling at its very heart. I don't want to blame it for that young man's death, but you can't deny that it happened after conversations with it. Maybe it has happened with other models, and we just haven't heard about it? ChatGPT is always in the media these days, making it an easy target.

ChatGPT is simply a reflection of what we
human beings allow it to be!

If we share only the positive aspects, we'll have pleasant conversations and positive answers. However, if we share darker stuff, our doom loop begins. But where does it end? Ignoring the risks is not an option, as this poor young man found out.

Remember, AI is a tool, and at most, you should consider it a colleague like I do. As with any colleague, we need to set rules, boundaries and expectations. It requires accountability, and we should never give it complete access to our inner selves, or tragic things can happen.

We need tools that aren't just 'nice', but sane. That challenges us when we need challenging, and redirects us when we need redirection. Because when someone's vulnerable, that difference isn't academic - it's everything.

If I'm wrong in how I use this tool, I want the tool to call me out. If I'm right, maybe we can help others see a safer way.

We all need to use the "Don't be a sycophant" rule!

Love, light & logic ...

STEFFI LEWIS
A.K.A The SaaSy Coder - Creator of YourBOT, YourPCM & sBlogIt!

Share the blog love ...

Buffer Facebook Twitter Linkedin Pinterest WhatsApp #ChatGPT #MentalHealthAwareness #AIandMentalHealth #TeenSuicidePrevention #HeartbreakingTragedy

About Steffi Lewis ...
Based in the charming village of Hanslope in Buckinghamshire, UK, I bring over 30 years of experience in web development. From creating my first website for the Open University in 1993, through being part of the dot com boom, to my explorations in SaaS and AI over the last few years, my journey has been a rich tapestry of web projects, big and small, both funded and bootstrapped. Originally developed during the pandemic, at the start of 2025, I released the next iteration of YourPCM, version 2, which is all about 'Easy Contact Management for Small Business Owners'. Is it a CRM? Yes, but I don't call it that because people glaze over at the mention of such things. CRMs are boring, difficult to get to grips with, and it's a nightmare to get help when you need it.  YourPCM is none of those things! It's simple to use, well-supported, feature rich and beautiful, right out of the box. It's available on a simple monthly subscription with no long-term tie-in. Book a demo or grab your own 14-working-day free trial and discover why YourPCM is all about easy contact management for small business owners.  My head is in the cloud, my heart belongs to the web, and my soul is filled with such beautiful code 💗

More blog posts for you to enjoy ...
		Its been a busy few months! What am I up to now? It has been a while since I wrote a full blog post here. The last one was about a great TV series I'd been watching, and before that, a Christmas one about YourBOT. So what have I been doing and what am I up to now?...
		Person of Interest got AI right, and we'd better start paying attention I've just finished watching five seasons of 'Person of Interest' on Amazon Prime, and like all the best sci-fi, it didn't really feel like fiction by the end. It felt like a profound warning wrapped in a really solid story .....
		Steffis slightly interesting review of coding in 2025 It's been quite some time since I wrote a review of the year. I'm not sure why I got out of the habit of doing them, but here we are again. This is my summary of 2025, which is quite short because most of it was a blur of eat...
		I released YourBOT this week, but there's still a lot to do! It's finished! It's published! It's out there in the wild. People are joining for free, creating their bots and installing them on their websites, then subscribing. To say I'm thrilled is an understatement ......
		How I built YourBOT with my AI code buddy, and what we're doing now I've been working hard for the past few months on YourBOT. I have blogged about it from time to time, but now the actual product is finished, it's being tested in the wild, and I'm onto creating the website and sign-up proces...
		YourBOT, the end of coding, and moving to a new server I was pretty emotional the other night whilst talking with ChatGPT; in a good way, of course. We've been working together on YourBOT for the past month and have finally reached the end of core coding ......
		YourBOT: Making good progress on my new SaaS product! After creating an integrated chatbot system for all of my sBlogIt! blogging clients, I realised it was so good that I had to monetise it. For the past few weeks, I've been creating YourBOT and I think it's a really good updat...
		A funny thing happened on the way to my Facebook page Well, I guess it had to happen eventually. I had my Facebook account suspended. The irony of this was that it was an AI filter that caused it! You all know that I love artificial intelligence, but I was a bit miffed ......

Other bloggers you may like ...
		The silent problem of loneliness in modern Britain Posted by Steffi Lewis on https://www.yourping.uk Loneliness in modern Britain does not always look like isolation. It can look like independence, remote working, online shopping and a perfectly ordin ...
		Moving to Milton Keynes: how Short Stay : MK makes relocation simple Posted by Emily Freeman on https://blog.shortstay-mk.co.uk Moving to Milton Keynes can feel like a lot, especially while you're job-starting and house-hunting at the same time. Short Stay : MK gives you a prac ...
		Physical symptoms of work stress: Is your body trying to tell you something? Posted by Dave Cordle on https://blog.davecordle.co.uk The physical symptoms of work stress can sometimes be easier to dismiss than the emotional ones. You might put a headache down to a long day, blame ac ...
		Companies House accounts filing changes: Why 2028 is closer than you think Posted by Roger Eddowes on https://blog.essendonaccounts.co.uk The Companies House accounts filing changes coming into force from April 2028 may sound comfortably distant, but I think businesses would be wise to s ...