AI apps on Google Play have to limit distribution of inappropriate content, company says


No graphic content, nonconsensual deepfakes, or bad marketing.

A phone displays the Google logo and Google Play store logo.

Google is drawing boundaries around generative AI apps.
Credit: Yasin Baturhan Ergin / Anadolu via Getty Images

New Google Play guidelines are putting the cuffs on generative AI apps offering dubious tools, such as deepfake “undressing” apps and those producing graphic content.

The updated app store policy, announced Thursday, instructs generative AI apps and their developers to build-in precautions against offensive content, “including prohibited content listed under Google Play’s Inappropriate Content policies, content that may exploit or abuse children, and content that can deceive users or enable dishonest behaviors.”

Developers must also offer in-app flagging and reporting mechanisms for users stumbling across inappropriate content and “rigorously test” their AI models, TechCrunch reported.

The rules apply to apps that produce AI-generated content in “any combination of text, voice, and image prompt input.” This includes chatbots, image generators, and audio-spoofing apps using generative AI. The policies do not apply to apps that “merely host” AI content or those with AI “productivity tools,” such as summarizing features.

Mashable Light Speed

In May, Google announced it was devaluing AI-generated (or “synthetic”) porn results in its internal search rankings, attempting to dress a growing problem of nonconsensual, deepfake pornography. The company also banned advertising for websites that create, endorse, or compare deepfake pornography.

The move came after a wave of viral, celebrity-centric deepfakes circulated on X and Meta platforms, including graphic advertisements for an AI-powered undressing app that featured underage photos of actor Jenna Ortega. Google was already fielding thousands of complaints from victims of nonconsensual, sexualized deepfakes, many of whom filed Digital Media Copyright Act (DMCA) claims against websites populating their likenesses.

AI industry insiders have issued multiple warnings about the threat of misinformation and the nonconsensual use of people’s likenesses, including a recent open letter penned by OpenAI and Google DeepMind employees. The group noted the potential risk of “manipulation and misinformation” should AI advancements continue without regulation.

Google’s app store regulations follow a White House AI directive issued to tech companies last month. The announcement called on industry leaders to do more to prohibit the spread of deepfakes, with Google specifically heeding a call to curb apps that “create, facilitate, monetize, or disseminate image-based sexual abuse.” If you have been a victim of deepfake abuse, there are steps you can take; read more about how to get support.

Chase sits in front of a green framed window, wearing a cheetah print shirt and looking to her right. On the window's glass pane reads

Chase joined Mashable’s Social Good team in 2020, covering online stories about digital activism, climate justice, accessibility, and media representation. Her work also touches on how these conversations manifest in politics, popular culture, and fandom. Sometimes she’s very funny.

This newsletter may contain advertising, deals, or affiliate links. Subscribing to a newsletter indicates your consent to our Terms of Use and Privacy Policy. You may unsubscribe from the newsletters at any time.


Please enter your comment!
Please enter your name here

Share post:



More like this

Season 4 of The Boys Is an 8-Episode Beginning of the End

ALL GOOD THINGS must come to an end. Especially...

65 Best Golf Gifts for Father’s Day

Best White T-ShirtsBare-Knuckle BoxingHow to Count MacrosTrue Story of...

Exactly How Much Water You Should Drink to Lose Weight

WEIGHT LOSS CAN be meticulous. Having to think about...

How to Barbell Deadlift With Proper Form

THERE ARE FEW more direct expressions of strength than...