How the Take It Down Act Will Impact Platforms Handling NCII Compliance

The Take It Down Act (TIDA) goes into full effect today, creating new federal obligations for platforms that host user-generated content. The law prohibits distribution of non-consensual intimate imagery (NCII), including AI-manipulated intimate images, and requires covered platforms to remove reported material within 48 hours. Major social platforms already deploy the moderation systems needed to comply with these requirements, but many smaller platforms will face significant operational and technical challenges — particularly around identifying and removing duplicate versions of reported content.

Compliance with the Take It Down Act

The FTC has released a range of guidance on TIDA compliance, including sending letters reminding major platforms of their obligations. But here are the basics:

• Platforms Covered. The Take It Down Act applies to public-facing websites, platforms, and mobile applications that “provide a forum for user generated content” and which make NCII available in their “regular course of business.” The FTC believes this applies to a wide range of platforms, including social media, marketplaces, and games. There is no minimum platform size to be subject to the TIDA.
• NCII Definition. The TIDA defines NCII broadly, and includes both real images and those manipulated digitally.
• Reporting Requirements. Platforms are required to enable users to report NCII across the surfaces of their applications. It is not clear exactly how the FTC will interpret that obligation, but FTC fact sheets suggest that it might mean “having a way for users to submit a removal request directly” from different platform surfaces, such as videos, comments, and photos.
• Obligation to Remove Original and Copies. Platforms are required to remove reported NCII within 48 hours, as well as "make reasonable efforts to identify and remove any known identical copies,” of the material. The FTC suggests using hash-matching to fulfill this requirement. The FTC has also established a website where consumers can report platforms that fail to meet the removal requirements.

NCII is not a new problem, but it has been exacerbated by the widespread availability of “undressing” applications and other AI tools for image manipulation. A wide range of expert organizations, including StopNCII.org, SWGFL, The Tech Coalition, the National Center for Missing and Exploited Children, and the Internet Watch Foundation, have done important work for years to confront NCII and other sexually abusive material online. These groups will continue to be critical resources moving forward, particularly for enabling cross-platform approaches to NCII.

Practical Enforcement Concerns Regarding TIDA

The TIDA reflects clear policy intent, but it fails to define some technical concepts clearly, which will likely lead to implementation ambiguity and eventual legal debate. For example, it does not specify what it means to identify an “identical” image. To a lawmaker, this language may seem self-evident, and the FTC clearly understands it to mean using hash matching of some kind. But internet safety professionals will wonder whether this requires searching for exact file matches using a tool like md5 or visually identical (or very similar) images that are actually different files. That would require using a perceptual hashing algorithm like PhotoDNA or PDQ. Lawmakers almost certainly intend the latter, but that will create ambiguity about implementation that will likely eventually require the FTC or courts to specify what constitutes an “identical” perceptual hash match. Such an approach will open the door to government-sanctioned false negatives and false positive removals that will heighten the concerns civil libertarians have with the law.

Additionally, nefarious actors will likely attempt to weaponize the TIDA’s reporting requirements. Trust & Safety mechanisms, including user reports, are often exploited to attack platforms and their users. There is no reason to think that TIDA reporting will be different. If TIDA reporting is widely used, the FTC will need new mechanisms to sort through such reports to separate genuine reports of harm from malicious reporting designed to attack specific platforms.

The Take It Down Act addresses a problem that is far more widespread than anyone of good conscience can accept. But implementation is likely to be a challenge for smaller platforms. Moreover, increased regulation of content online will force lawmakers, regulators, and courts to wrestle with the details of digital safety in ways that have previously been limited to Trust & Safety professionals.