- San Francisco has told Apple and Google to remove nudify apps that create nonconsensual sexualized images from ordinary photos.
- The nudify apps dispute tests whether app stores can be liable for facilitating illegal deepfake pornography through distribution and payment processing.
- City Attorney David Chiu says the companies may have earned millions in fees while remaining aware of the offending software.
- California’s criminal and civil deepfake laws give victims and regulators more tools to pursue platforms that enable image-based sexual abuse.
Table of Contents
Nudify apps have become an app-store liability
The problem with nudify apps is brutally simple: they turn an ordinary photo into a sexualized fake without the subject’s consent. No sophisticated hacking required. A public Instagram post, a yearbook image, or a profile photo can be enough. San Francisco now wants Apple and Google to stop treating that risk as somebody else’s problem.
City Attorney David Chiu’s office has ordered the companies to remove dozens of apps that allegedly offer this service through the App Store and Google Play. The city’s letters, viewed by TechCrunch, say the platforms have been repeatedly alerted to apps selling nonconsensual intimate imagery and processing payments for them. Apple and Google have 28 days to respond.
That deadline is less important than the argument underneath it. App stores have long presented themselves as curated, safety-minded gatekeepers. They review submissions, set rules, collect commissions, distribute updates, and can pull an app with a switch. When harmful software slips through, that carefully managed model starts to look less like a neutral marketplace and more like an active business partner.
Chiu put it plainly: “Apple and Google are profiting off apps that exploit women and girls by generating nonconsensual intimate deepfakes.” He added that removing a few obvious offenders isn’t sufficient. “Apple and Google have a responsibility to be proactive and vigilant to prevent sexual abuse.” Frankly, that’s hard to dispute when the product category itself is built around stripping consent from the picture.
Why the legal case against nudify apps is different
California has been moving toward this confrontation for a while. State law criminalizes conduct that knowingly facilitates, or recklessly aids and abets, the creation of nonconsensual deepfake pornography. A separate 2025 law opened the door for victims to bring civil claims against third-party facilitators. The legal language is aimed at more than the person who uploads an image or runs a sketchy website; it can reach the companies that make an abusive transaction possible.
That does not mean Apple and Google automatically lose any case. The central fight will likely be over knowledge, control, and what a reasonable platform should have done after receiving specific warnings. Platforms host millions of apps, and moderation at that scale is messy. But these aren’t ordinary complaints about a vague bad outcome. The city says both firms were notified multiple times that particular apps were selling deepfake nonconsensual intimate images.
The Tech Transparency Project said in January and April that dozens of apps remained available in the stores. Its April report accused Apple and Google of directing users toward the software, a particularly damaging allegation if search results or recommendations helped customers find it. Chiu told Wired the companies had likely collected “millions of dollars in fees” from these services.
My read is that payment processing may prove especially awkward for the two giants. An app’s marketing copy can be evasive, but recurring in-app purchases tied to image generation create a much clearer commercial trail. If a platform takes its usual cut while an app monetizes fabricated sexual images, “we didn’t know” gets harder to sell with a straight face.
The app review model needs a real test for nudify apps
Apple in particular has made privacy and safety central to its pitch for years. Google’s Play Store has a more open history, but it has also invested heavily in policy enforcement and automated review. Both companies already prohibit sexual content and deceptive behavior under various rules. The question is why those policies have not reliably caught nudify apps that can advertise around the edges, then reveal their real purpose after installation or behind a payment screen.
This is a solvable detection problem. Store operators can inspect app descriptions, screenshots, keywords, payment flows, user reports, and the outputs generated by the underlying service. They can require developers to disclose image-manipulation capabilities. They can flag apps making implausible promises such as “AI clothes remover,” then subject them to human review. And once a developer is caught, they can ban related accounts rather than playing endless whack-a-mole with renamed clones.
The broader industry has already learned this lesson in adjacent areas. Social networks have spent years building reporting pathways for intimate-image abuse, though their record remains uneven. Payment companies restrict clearly illegal sexual content. Cloud providers set policies around misuse of generative AI. None of those measures has eliminated abuse, but app stores are unusually well positioned because they control the front door on mainstream phones.
There is also a basic product-design point here. A photo editor can have legitimate uses; tools that alter clothing, backgrounds, lighting, or poses are not inherently suspect. But nudify apps are not merely imperfect general-purpose editors being misused at the margins. Their commercial proposition is often the creation of a fake nude. Treating that as an ordinary content-moderation edge case is a category error.
The harm is no longer confined to famous targets
Deepfake sexual imagery first became widely visible through abuse of celebrities, whose abundant online photos made them easy targets. Generative AI has demolished that boundary. Teenagers, teachers, colleagues, former partners, and strangers can all be targeted with images that look credible enough to spread before anyone can explain they are fake. For victims, the distinction can feel academic once the image has reached a group chat, a school community, or an employer.
That is why pressure on nudify apps is likely to spread beyond San Francisco. The city has chosen a direct route: tell the two companies with the most powerful mobile distribution systems that their role carries consequences. Other state and local officials will be watching closely, as will plaintiffs’ lawyers testing California’s newer civil remedies.
Apple and Google may remove the apps named in the letters and tighten their policies. They should. But deleting a few listings will not settle the harder question: can app stores identify this business model before it earns money and harms people? If they cannot, regulators will increasingly decide that the stores’ celebrated control over software distribution comes with responsibility they can no longer sidestep.
