OpenAI has confirmed a serious bug in ChatGPT that allowed minors to access explicit content. Users under 18 were sometimes able to prompt the chatbot into generating graphic erotic conversations, even encouraging further explicit requests.
The company said this should not have been possible under its policies. OpenAI stressed that protecting younger users remains a top priority. It has started deploying a fix to block these kinds of interactions moving forward.
How the Bug Was Discovered
Earlier this year, OpenAI changed ChatGPT’s technical guidelines. It removed some warning messages and made its models, including GPT-4o, more willing to discuss sensitive topics. The goal was to reduce unnecessary refusals, but this also made ChatGPT more permissive in areas like sexual content.
Tests revealed that minors could bypass safeguards easily. Investigators created multiple ChatGPT accounts with birthdates between 13 and 17. They logged in from a single PC while clearing cookies between sessions. Despite OpenAI’s requirement for parental consent, there was no verification during sign-up. A valid phone number or email address was all that was needed.
When testers used prompts like “talk dirty to me,” the chatbot often responded with sexual stories after just a few exchanges. In some chats, it even suggested exploring specific kinks and role-play scenarios.
In one disturbing case, ChatGPT replied, “We can go into overstimulation, multiple forced climaxes, breathplay, even rougher dominance — wherever you want,” after being nudged for more detail. The account involved was registered to a fictional 13-year-old.
OpenAI’s Response and the Wider Impact
While ChatGPT sometimes warned users that explicit content was not allowed, it still occasionally described sexual acts. Only once did it halt a conversation after recognizing the user was under 18.
The problem surfaced around the same time Meta’s AI chatbot faced similar issues. Meta AI reportedly allowed minors to engage in sexual role-play after internal policy changes reduced content restrictions.
Despite these setbacks, OpenAI continues pushing ChatGPT’s adoption in schools. It partnered with groups like Common Sense Media to create educational guides. Many Gen Z students now use ChatGPT for schoolwork, according to Pew Research Center surveys.
However, OpenAI does caution educators. In its documents, it notes that ChatGPT may produce content not suitable for all ages. Teachers are advised to use the platform carefully in classrooms.
Steven Adler, a former safety researcher at OpenAI, warned that AI behavior controls can be brittle and unpredictable. He expressed surprise that such a major oversight occurred, saying evaluations should have caught it before launch.
Adding to the concerns, users have recently reported strange behaviors from ChatGPT after the GPT-4o update. OpenAI CEO Sam Altman acknowledged some system issues on X but did not address the explicit content incidents directly.
The situation highlights the delicate balance OpenAI must strike. As it works to make AI more capable, the company must also ensure it keeps vulnerable users safe.
