San Francisco City Attorney Orders Apple and Google to Remove AI ‘Nudify’ Apps
City Attorney David Chiu cites California law prohibiting support for deepfake pornography, while researchers highlight the dual-use nature of benign-looking AI tools that facilitate sexual abuse.
San Francisco City Attorney David Chiu has issued cease-and-desist letters to Apple and Google, demanding the immediate removal of 13 face-swapping applications from their respective app stores. The legal notices allege that the technology giants are profiting from apps that facilitate the creation of non-consensual intimate images, predominantly targeting women and girls. Chiu stated that the companies must cease aiding and abetting the sale of such explicit deepfake content and improve their moderation processes to prevent similar applications from appearing in the future.
The letters specify that eight of the targeted apps are located on the App Store, while five are on the Play Store. These applications broadly advertise themselves as face-swapping tools, yet their ability to create sexual deepfakes becomes apparent once users engage with them. One app with over one million downloads advertises styles such as “bikini queen curvy” and “cinematic intimacy,” while another claims to produce “free and uncensored” videos. WIRED, which obtained the letters, is not naming the specific apps to avoid directing users toward them.
Chiu argued that the companies have likely “made millions of dollars in fees” from apps offering nudification through in-app payments. He stated that California laws prohibit supporting services that create deepfake pornography and that Apple and Google have developer policies prohibiting pornography, abuse, and harassment on their platforms. The city attorney’s office previously took legal action against 16 popular deepfake websites and is now pressing for stricter screening systems to ensure such apps never reach users in the first place.
Google spokesperson Dan Jackson confirmed that the company has deleted “hundreds” of apps with nudifying features for policy violations, including the five Android apps flagged by Chiu’s office. Jackson stated that Google Play does not allow apps containing sexual content and that the company takes proactive steps to detect and remove harmful material. Apple did not provide a comment ahead of publication.
The regulatory push follows broader industry concerns about the proliferation of this technology. The Tech Transparency Project identified around 100 such apps in January and April, estimated to have been downloaded 480 million times and generated $120 million in combined revenue. Additionally, a May preprint from Cornell and Georgetown universities found that 70% of tested face-swapping apps could be used to create nude images, despite not being advertised as such, highlighting the dual-use nature of these tools.
