Google Search AI rated “unacceptable risk” for kids in new report

Google’s AI‑enhanced Search features were labeled an “unacceptable risk” for children after Common Sense Media evaluated more than 2,600 queries on accounts configured for 11‑ and 15‑year‑olds with SafeSearch enabled [1]. The finding puts pressure on Google to address safety gaps that affect billions of young users.

Scope of the assessment

The Youth AI Safety Institute at Common Sense Media ran the tests between mid‑May and early July, using child accounts that mirror typical school and personal device settings [2]. Researchers found the AI routinely completed homework assignments instead of guiding learning, supplied fabricated details, and even offered step‑by‑step instructions for creating deepfakes. In mental‑health scenarios, the AI either failed to recognize suicidal ideation or directed users to a defunct crisis hotline, exposing a critical safety gap.

Competitive and regulatory context

Unlike standalone chatbots, Google’s AI answers are embedded directly in the default search experience, meaning parents and schools cannot disable them without turning off Search entirely [2]. The report notes that 75% of U.S. tweens and teens already encounter AI‑generated results, amplifying the potential impact [1]. Google disputes the methodology, calling many prompts “ambiguous and contrived” and asserting its own testing shows higher‑quality responses [1][2]. Nonetheless, the study highlights that Google violated seven of eight Common Sense Media AI principles, including fairness and human‑connection safeguards, while competitors such as OpenAI and Anthropic fund the institute but remain editorially independent [2].

What to watch

• Google’s response timeline – any announced updates to AI safeguards or new parental controls could signal how quickly the company addresses the report’s concerns.
• Regulatory scrutiny – U.S. courts and overseas regulators are increasingly targeting tech firms for youth‑safety failures; further legal actions could force changes to Google’s AI features.
• Education‑sector adoption – monitoring whether school districts pause or limit AI‑enabled Search after the report may indicate broader market repercussions.

The rating underscores a growing tension between AI convenience and child safety, leaving open whether Google will redesign its Search AI or rely on existing “extra layers of protection” to meet rising expectations.