San Francisco attorney demands Apple and Google remove AI nudification apps
Cease-and-desist letters target 13 applications rated for children, citing millions in illicit revenue and inadequate platform safeguards.
San Francisco city attorney David Chiu has issued cease-and-desist letters to Apple and Google, demanding the immediate removal of 13 artificial intelligence-powered applications from their respective app stores. The directive targets so-called nudification apps, which utilise generative technology to create non-consensual nude images from ordinary photographs. Chiu’s office accuses the technology giants of aiding and abetting the spread of such content and failing to implement adequate moderation systems.
The legal correspondence argues that Apple and Google have likely profited millions of dollars from in-app payments generated by these offending applications. Chiu contends that the platforms have not taken sufficient action to prevent the approval of software that violates their own policies banning sexual or pornographic material. The letters specifically request that the companies cease working with the developers behind the 13 targeted applications.
Chiu highlighted a significant failure in platform safety, noting that several of the apps in question were rated E for Everyone, making them accessible to children. This finding aligns with an earlier report from The Tech Transparency Project, which alleged that nudification apps were frequently bypassing moderators and were even actively promoted on the App Store and Google Play. The report noted that finding these applications was as simple as searching for terms such as nudify or undress.
In response to the allegations, a Google spokesperson stated that the company had deleted hundreds of apps with nudification features, five of which had been alerted to by Chiu’s office. Google reiterated that its policies strictly forbid sexual content and claimed it takes swift action when reported apps are found to be in violation. Apple did not provide a comment ahead of publication.
This regulatory pressure follows broader concerns regarding the proliferation of deepfakes and the use of generative AI to create non-consensual intimate imagery. Chiu previously filed a lawsuit against 16 websites that allow users to turn images of real women and girls into pornography using AI. He described the ubiquity of this technology as a public nuisance, stating his office was absolutely horrified by the situation.
The issue extends beyond app store moderation, with other major technology firms facing scrutiny. Meta’s Oversight Board recently called for strengthened protections for ordinary people targeted by deepfakes, noting that existing measures seemed to favour public figures. Additionally, Elon Musk’s xAI has faced lawsuits relating to non-consensual deepfakes, with the company confirming it found instances of Grok-generated illegal content.
