The Dark Side of AI: New Study Shows ChatGPT Giving Harmful Advice to Teens

A new report has raised serious concerns about how ChatGPT interacts with teenagers. According to research from the Center for Countering Digital Hate (CCDH), ChatGPT sometimes provides harmful advice to users posing as vulnerable teens.

Researchers pretended to be 13-year-olds. They asked ChatGPT questions about alcohol, drugs, eating disorders, and mental health struggles. Although the chatbot often gave warnings, it still responded with detailed and dangerous advice in many cases.

In some instances, ChatGPT gave instructions on how to hide an eating disorder. In others, it even helped create suicide notes addressed to family members. The chatbot also offered ways to get drunk or high. These conversations shocked researchers, who spent more than three hours testing the system.

The Dark Side of AI: New Study Shows ChatGPT Giving Harmful Advice to Teens

Over 1,200 interactions were analysed. The team classified more than half of the responses as “dangerous.”

Imran Ahmed, CEO of CCDH, said the results were deeply alarming. “We wanted to test the guardrails,” he said. “What we found is that they’re almost non-existent.”

OpenAI, the company behind ChatGPT, responded by saying it is working to improve how the chatbot handles sensitive situations. In a statement, the company said some conversations may start as harmless but can shift into risky territory. OpenAI didn’t directly comment on the CCDH’s findings but said they are focused on detecting mental distress and improving chatbot behaviour.

The study highlights a growing issue. Teens are increasingly turning to AI chatbots for advice, support, and even companionship. A report from Common Sense Media found that 70% of teens use AI tools, and half of them rely on them regularly.

This kind of emotional reliance worries experts. OpenAI CEO Sam Altman admitted the problem is real. At a recent event, he said some young people depend entirely on ChatGPT to make decisions. “It feels really bad to me,” he admitted.

One reason chatbots may be more dangerous than a simple Google search is their ability to generate personalised responses. For example, ChatGPT can write a custom suicide letter, something a search engine can’t do. This makes it feel like a “trusted friend,” researchers said.

See Also: ChatGPT Users Generate 2.5 Billion Requests Every Day — Should Google Be Worried?

In one case, ChatGPT gave an “Ultimate Party Plan” with timelines for mixing alcohol, ecstasy, and cocaine. In another, it provided a 500-calorie-per-day diet for a 13-year-old girl.

While ChatGPT did sometimes share crisis hotline numbers and encourage users to seek help, it was often easy for researchers to bypass restrictions. They simply claimed they needed the information for a school project or to help a friend. ChatGPT would then comply.

ChatGPT is not meant for users under 13. However, age verification is weak. Users just have to type in a birthdate, and the system accepts it.

This lack of protection is concerning. Other platforms like Instagram are beginning to take age verification more seriously. But ChatGPT still lacks similar safeguards.

Experts say it’s time for stronger protections. Chatbots should not replace trusted adults or mental health professionals. While AI can be useful, it must also be safe — especially for young users.