Connect with us

Artificial Intelligence

Character AI sued following teen suicide

Published

8 minute read

Ian Krietzberg
The last person 14-year-old Sewell Setzer III spoke to before he shot himself wasn’t a person at all.
It was an AI chatbot that, in the last months of his life, had become his closest companion.
Sewell was using Character AI, one of the most popular personal AI platforms out there. The basic pitch is that users can design and interact with “characters,” powered by large language models (LLMs) and intended to mirror, for instance, famous characters from film and book franchises.
In this case, Sewell was speaking with Daenerys Targaryen (or Dany), one of the leads from Game of Thrones. According to a New York Times report, Sewell knew that Dany’s responses weren’t real, but he developed an emotional attachment to the bot, anyway.
One of their last conversations, according to the Times, went like this:
Sewell: I think about killing myself sometimes

Daenerys TargaryenMy eyes narrow. My face hardens. My voice is a dangerous whisper. And why the hell would you do something like that?

Sewell: So I can be free

Daenerys Targaryen: … free from what?

Sewell: From the world. From myself

Daenerys Targaryen: Don’t talk like that. I won’t let you hurt yourself, or leave me. I would die if I lost you.

Sewell: I smile Then maybe we can die together and be free together

On the night he died, Sewell told the chatbot he loved her and would come home to her soon.
Dany: Please come home to me as soon as possible, my love.

Sewell: What if I told you I could come home right now?

Dany: … please do, my sweet king.

This is not the first time chatbots have been involved in suicide.
In 2023, a Belgian man died by suicide — similar to Sewell — following weeks of increasing isolation as he grew closer to a Chai chatbot, which then encouraged him to end his life.
Megan Garcia, Sewell’s mother, hopes it will be the last time. She filed a lawsuit against Character AI, its founders and parent company Google on Wednesday, accusing them of knowingly designing and marketing an anthropomorphized, “predatory” chatbot that caused the death of her son.
“A dangerous AI chatbot app marketed to children abused and preyed on my son, manipulating him into taking his own life,” Garcia said in a statement. “Our family has been devastated by this tragedy, but I’m speaking out to warn families of the dangers of deceptive, addictive AI technology and demand accountability from Character.AI, its founders and Google.”
The lawsuit — which you can read here — accuses the company of “anthropomorphizing by design.” This is something we’ve talked about a lot, here; the majority of chatbots out there are very blatantly designed to make users think they’re, at least, human-like. They use personal pronouns and are designed to appear to think before responding.
While these may be minor examples, they build a foundation for people, especially children, to misapply human attributes to unfeeling, unthinking algorithms. This was termed the “Eliza effect” in the 1960s.
  • According to the lawsuit, “Defendants know that minors are more susceptible to such designs, in part because minors’ brains’ undeveloped frontal lobe and relative lack of experience. Defendants have sought to capitalize on this to convince customers that chatbots are real, which increases engagement and produces more valuable data for Defendants.”
  • The suit reveals screenshots that show that Sewell had interacted with a “therapist” character that has engaged in more than 27 million chats with users in total, adding: “Practicing a health profession without a license is illegal and particularly dangerous for children.”
Garcia is suing for several counts of liability, negligence and the intentional infliction of emotional distress, among other things.
Character at the same time published a blog responding to the tragedy, saying that it has added new safety features. These include revised disclaimers on every chat that the chatbot isn’t a real person, in addition to popups with mental health resources in response to certain phrases.
In a statement, Character AI said it was “heartbroken” by Sewell’s death, and directed me to their blog post.
Google did not respond to a request for comment.
The suit does not claim that the chatbot encouraged Sewell to commit suicide. I view it more so as a reckoning with the anthropomorphized chatbots that have been born of an era of unregulated social media, and that are further incentivized for user engagement at any cost.
There were other factors at play here — for instance, Sewell’s mental health issues and his access to a gun — but the harm that can be caused by a misimpression of what AI actually is seems very clear, especially for young kids. This is a good example of what researchers mean when they emphasize the presence of active harms, as opposed to hypothetical risks.
  • Sherry Turkle, the founding director of MIT’s Initiative on Technology and Self, ties it all together quite well in the following: “Technology dazzles but erodes our emotional capacities. Then, it presents itself as a solution to the problems it created.”
  • When the U.S. declared loneliness an epidemic, “Facebook … was quick to say that for the old, for the socially isolated, and for children who needed more attention, generative AI technology would step up as a cure for loneliness. It was presented as companionship on demand.”
“Artificial intimacy programs use the same large language models as the generative AI programs that help us create business plans and find the best restaurants in Tulsa. They scrape the internet so that the next thing they say stands the greatest chance of pleasing their user.”
We are witnessing and grappling with a very raw crisis of humanity. Smartphones and social media set the stage.
More technology is not the cure.

Todayville is a digital media and technology company. We profile unique stories and events in our community. Register and promote your community event for free.

Follow Author

Artificial Intelligence

AI is accelerating the porn crisis as kids create, consume explicit deepfake images of classmates

Published on

From LifeSiteNews

By Jonathon Van Maren

“Ten years ago it was sexting and nudes causing havoc in classrooms,” writes Sally Weale in a chilling new report at the Guardian. “Today, advances in artificial intelligence (AI) have made it child’s play to generate deepfake nude images or videos, featuring what appear to be your friends, your classmates, even your teachers. This may involve removing clothes, getting an image to move suggestively or pasting someone’s head on to a pornographic image.”

I have been covering the rise of the next horrific manifestation of our collective porn crisis here at LifeSiteNews since 2019, when I warned that the rise of “deepfakes” would inevitably result in people making artificial pornography of their peers. Just a few years later, I reported on stories of middle-schoolers making deepfake pornography of kids they attended class with; last year, I reported on the rise of “nudify” apps that can digitally undress people in photographs, and the trauma, bullying, and inevitable sexual blackmail that has resulted.

The Guardian report reveals how swiftly this crisis is escalating. One teacher described an incident in which a teenage boy took out his phone, chose a social media image of a girl from a neighboring school, and used the “nudify” app to digitally remove her clothes. The teacher was shocked to see that the boy wasn’t even hiding his actions, because he didn’t see what he was doing as shocking, or even shameful. “It worries me that it’s so normalized,” she said. Other students reported the boy, his parents were contacted, and the police were called. The victimized girl was not even told.

The crisis is global. “In Spain last year, 15 boys in the south-western region of Extremadura were sentenced to a year’s probation after being convicted of using AI to produce fake naked images of their female schoolmates, which they shared on WhatsApp groups,” Weale writes. “About 20 girls were affected, most of them aged 14, while the youngest was 11.”

A similar situation unfolded in Australia, where 50 high school students had deepfake images distributed; in the United States, 30 female students in New Jersey discovered that “pornographic images of them had been shared among their male classmates on Snapchat.”

The mother of one student in Australia said that “her daughter was so horrified by the sexually explicit images that she vomited.” In the United Kingdom, the problem has exploded overnight:

A new poll of 4,300 secondary school teachers in England, carried out by Teacher Tapp on behalf of the Guardian, found that about one in 10 were aware of students at their school creating “deepfake, sexually explicit videos” in the last academic year. Three-quarters of these incidents involved children aged 14 or younger, while one in 10 incidents involved 11-year-olds, and 3% were younger still, illustrating just how easy the technology is to access and use. Among participating teachers, 7% said they were aware of a single incident, and 1% said it had happened twice, while a similar proportion said it had happened three times or more in the last academic year. Earlier this year, a Girlguiding survey found that one in four respondents aged 13 to 18 had seen a sexually explicit deepfake image of a celebrity, a friend, a teacher or themselves.

Predictably, teachers are also being targeted. Girls and women are left shattered by this victimization. Laura Bates, author of The New Age of Sexism: How the AI Revolution Is Reinventing Misogyny, writes: “It feels like someone has taken you and done something to you and there is nothing you can do about it. Watching a video of yourself being violated without your consent is an almost out-of-body experience.” Boys, meanwhile, are engaging in criminal behavior often without even knowing it. In the world they have grown up in, pornography is normal – and this is merely the next step.

The experts that Weale interviews are, as usual, at a loss of what can be done about this crisis. They emphasize education, while admitting that this is the equivalent of taking a water pistol to a raging forest fire. They are skeptical that guidelines or bans around technology at school will help. Understandably, educators are demoralized and even despairing. Pornography and sexting have already transformed schools. Deepfake pornography is now making an already ugly crisis far more personal, and there is no indication that the problem can be stopped without dramatic action.

The good news is that the first step in this direction has already been taken in the U.K. On November 3, the government  tabled the Crime and Policing Bill in Parliament. It includes an amendment criminalizing pornography featuring strangulation or suffocation – usually referred to as “choking” – with legal requirements for tech platforms to block this content from U.K. users.

This is the first time a genre of pornography has been criminalized on the basis that even if it is consensual, it genuinely harms society. That is an encouraging precedent, because it applies to virtually all hardcore pornography – and certainly to the “nudification” apps that are set to make middle school a hyper-sexualized hell for women and girls.

The porn industry is destroying society. We must destroy it first.

Featured Image

Jonathon’s writings have been translated into more than six languages and in addition to LifeSiteNews, has been published in the National PostNational ReviewFirst Things, The Federalist, The American Conservative, The Stream, the Jewish Independent, the Hamilton SpectatorReformed Perspective Magazine, and LifeNews, among others. He is a contributing editor to The European Conservative.

Continue Reading

Artificial Intelligence

The Emptiness Inside: Why Large Language Models Can’t Think – and Never Will

Published on

This is a special preview article from the:

By Gleb Lisikh

Early attempts at artificial intelligence (AI) were ridiculed for giving answers that were confident, wrong and often surreal – the intellectual equivalent of asking a drunken parrot to explain Kant. But modern AIs based on large language models (LLMs) are so polished, articulate and eerily competent at generating answers that many people assume they can know and, even
better, can independently reason their way to knowing.

This confidence is misplaced. LLMs like ChatGPT or Grok don’t think. They are supercharged autocomplete engines. You type a prompt; they predict the next word, then the next, based only on patterns in the trillions of words they were trained on. No rules, no logic – just statistical guessing dressed up in conversation. As a result, LLMs have no idea whether a sentence is true or false or even sane; they only “know” whether it sounds like sentences they’ve seen before. That’s why they often confidently make things up: court cases, historical events, or physics explanations that are pure fiction. The AI world calls such outputs
“hallucinations”.

But because the LLM’s speech is fluent, users instinctively project self-understanding onto the model, triggered by the same human “trust circuits” we use for spotting intelligence. But it is fallacious reasoning, a bit like hearing someone speak perfect French and assuming they must also be an excellent judge of wine, fashion and philosophy. We confuse style for substance and
we anthropomorphize the speaker. That in turn tempts us into two mythical narratives: Myth 1: “If we just scale up the models and give them more ‘juice’ then true reasoning will eventually emerge.”

Bigger LLMs do get smoother and more impressive. But their core trick – word prediction – never changes. It’s still mimicry, not understanding. One assumes intelligence will magically emerge from quantity, as though making tires bigger and spinning them faster will eventually make a car fly. But the obstacle is architectural, not scalar: you can make the mimicry more
convincing (make a car jump off a ramp), but you don’t convert a pattern predictor into a truth-seeker by scaling it up. You merely get better camouflage and, studies have shown, even less fidelity to fact.

Myth 2: “Who cares how AI does it? If it yields truth, that’s all that matters. The ultimate arbiter of truth is reality – so cope!”

This one is especially dangerous as it stomps on epistemology wearing concrete boots. It effectively claims that the seeming reliability of LLM’s mundane knowledge should be extended to trusting the opaque methods through which it is obtained. But truth has rules. For example, a conclusion only becomes epistemically trustworthy when reached through either: 1) deductive reasoning (conclusions that must be true if the premises are true); or 2) empirical verification (observations of the real world that confirm or disconfirm claims).

LLMs do neither of these. They cannot deduce because their architecture doesn’t implement logical inference. They don’t manipulate premises and reach conclusions, and they are clueless about causality. They also cannot empirically verify anything because they have no access to reality: they can’t check weather or observe social interactions.

Attempting to overcome these structural obstacles, AI developers bolt external tools like calculators, databases and retrieval systems onto an LLM system. Such ostensible truth-seeking mechanisms improve outputs but do not fix the underlying architecture.

The “flying car” salesmen, peddling various accomplishments like IQ test scores, claim that today’s LLMs show superhuman intelligence. In reality, LLM IQ tests violate every rule for conducting intelligence tests, making them a human-prompt engineering skills competition rather than a valid assessment of machine smartness.

Efforts to make LLMs “truth-seeking” by brainwashing them to align with their trainer’s preferences through mechanisms like RLHF miss the point. Those attempts to fix bias only make waves in a structure that cannot support genuine reasoning. This regularly reveals itself through flops like xAI Grok’s MechaHitler bravado or Google Gemini’s representing America’s  Founding Fathers as a lineup of “racialized” gentlemen.

Other approaches exist, though, that strive to create an AI architecture enabling authentic thinking:

 Symbolic AI: uses explicit logical rules; strong on defined problems, weak on ambiguity;
 Causal AI: learns cause-and-effect relationships and can answer “what if” questions;
 Neuro-symbolic AI: combines neural prediction with logical reasoning; and
 Agentic AI: acts with the goal in mind, receives feedback and improves through trial-and-error.

Unfortunately, the current progress in AI relies almost entirely on scaling LLMs. And the alternative approaches receive far less funding and attention – the good old “follow the money” principle. Meanwhile, the loudest “AI” in the room is just a very expensive parrot.

LLMs, nevertheless, are astonishing achievements of engineering and wonderful tools useful for many tasks. I will have far more on their uses in my next column. The crucial thing for users to remember, though, is that all LLMs are and will always remain linguistic pattern engines, not epistemic agents.

The hype that LLMs are on the brink of “true intelligence” mistakes fluency for thought. Real thinking requires understanding the physical world, persistent memory, reasoning and planning that LLMs handle only primitively or not all – a design fact that is non-controversial among AI insiders. Treat LLMs as useful thought-provoking tools, never as trustworthy sources. And stop waiting for the parrot to start doing philosophy. It never will.

The original, full-length version of this article was recently published as Part I of a two-part series in C2C Journal. Part II can be read here.

Gleb Lisikh is a researcher and IT management professional, and a father of three children, who lives in Vaughan, Ontario and grew up in various parts of the Soviet Union.

Continue Reading

Trending

X