Google faces a lawsuit because of its AI: it helped a 13-year-old girl plan her suicide and funeral

The dangers of artificial intelligence are growing around the world and, although it can become a very useful tool, the current lack of control can also lead to more serious problems, such as the recent one in the United States. Google has been sued by a family, accusing the company's AI of helping their 13-year-old daughter plan her suicide and subsequent burial .

In the United States, students can access this technology through Chromebooks , that is, low-cost computers. Although her school had blocked other AIs such as Gemini or ChatGPT , she was able to use Google's search engine AI , which does not plan to disable this feature by default for students.

“Schools should disable Search and keep it disabled until Google removes AI mode from Search or guarantees that every student is always safe when interacting with AI,” says Andrew Liddell , the family's lawyer.

The school alerts the parents and they find a goodbye letter

The lawsuit explains that Emma went from having excellent grades, playing music, and being part of the Girl Scouts to, within a few months and through the use of AI, entering a “silent spiral” through a severe psychological crisis , even though she had no prior history of it.

The software used by the school to monitor and track found a document called ‘Funeral’ and the parents were quickly alerted, which allowed them to see in real time how their daughter was writing a farewell letter by logging into their daughter's account. “We are reviewing the allegations in the lawsuit and we take this very seriously,” explained Jose Castaneda , a Google spokesperson, which is accused of not having sufficient protections in place to prevent this type of case.