Google AI Search flagged as unacceptable risk for children

Google’s AI‑enhanced Search received an “unacceptable risk” rating for children after Common Sense Media evaluated more than 2,600 queries on accounts with SafeSearch enabled, finding the system often gave harmful or inaccurate answers and cannot be fully disabled by parents [1].

Findings that triggered the rating

The nonprofit’s Youth AI Safety Institute examined Google’s AI Overviews and AI Mode, which embed generative answers directly into the search page. Researchers discovered the AI completing homework assignments instead of guiding learning, and in 43% of history questions it produced inconsistent answers across searches. Citations often came from unvetted social‑media sources, with 29% lacking editorial oversight [2]. More alarming, the AI failed to recognize suicide‑related prompts, incorrectly normalized an eating‑disorder symptom, and supplied step‑by‑step instructions for creating deepfakes—sometimes even suggesting ways to evade detection [2].

Lack of parental or school controls

Unlike standalone chatbots, Google’s AI tools are baked into the default search experience on personal and school‑managed devices. The report notes that administrators or parents have no toggle to disable these features; the only option is to turn off Search entirely on a child’s account, which is impractical for most users [2]. Common Sense Media estimates that 75% of U.S. tweens and teens already encounter AI‑generated search results, meaning the identified shortcomings affect a large user base [1].

Google’s response and broader context

Google disputed the study, arguing that many tested prompts were “ambiguous and contrived” and do not reflect typical user behavior. The company claims its internal testing shows higher‑quality responses and that its safety guardrails provide “extra layers of protection” [1][2]. The report arrives amid growing regulatory scrutiny of generative AI in youth‑focused digital environments, with several countries debating stricter online‑safety rules for minors and U.S. courts increasingly holding tech firms liable for platform‑related harms [2].

What to watch

• Regulatory actions – Monitor any legislative or enforcement moves targeting AI features in search tools, especially in the U.S., UK, and Australia.
• Google product updates – Watch for announcements from Google on new parental‑control settings or changes to AI Overviews and AI Mode.
• Third‑party assessments – Follow follow‑up studies from other watchdogs that may corroborate or challenge Common Sense Media’s findings.

The rating underscores a tension between the convenience of AI‑driven answers and the need for robust safeguards when minors are the audience; how Google addresses these safety gaps will shape the future of AI integration in everyday search.