Hot AI News

Perceptual hashing: How platforms spot known abuse images without viewing them

Perceptual hashing gives an image a compact fingerprint so platforms can find copies of known illegal images, even after resizing. It cannot spot new material, which is why classifiers are being added.

Perceptual hashing: How platforms spot known abuse images without viewing them
Image: TechCrunch

Perceptual hashing turns an image into a short fingerprint, called a hash, that stays nearly the same when the picture is resized or lightly edited. A platform can then compare fingerprints instead of looking at the pictures themselves.

The best known system is Microsoft’s PhotoDNA. It is widely used to find copies of child sexual abuse material (CSAM) that has already been identified.

Finding a copy of one photo among billions

Picture a library with a card catalog. Each card describes a book in a few lines. A librarian who is told to find one specific book does not read every shelf. They look for the matching card.

Hashing works the same way. Experts and child protection groups confirm that an image is illegal. Its fingerprint goes into a shared list. When a user uploads a photo, the service computes the fingerprint and checks the list. A match flags the upload for action and reporting.

No one has to view the new upload to make that first check. The system only asks whether this fingerprint is on the list.

Advertisements

Why an ordinary hash is not enough

A monitor compares two sizes of the same coastal photograph, with different rows of colored fingerprint blocks beneath them.
Photo illustration: Aiholics

Computing platforms have long used cryptographic hashes such as MD5 and SHA-1. They produce a fixed-length string from a file. Change one pixel and the string changes completely.

That is useful for proving a file was not altered, but weak for finding copies. Anyone can resize, crop or recompress an image and defeat an exact match.

A perceptual hash is built to behave differently. Similar-looking images get similar fingerprints, so a match is a small distance between two fingerprints, not exact equality.

How PhotoDNA builds its fingerprint

Microsoft Research and Dartmouth College professor Hany Farid began developing PhotoDNA in 2009, according to Wikipedia’s summary of the project. Microsoft’s own page says the company partnered with Dartmouth that year.

The exact design has never been publicly disclosed by its authors. A leaked software library appeared on GitHub in 2021. Wikipedia describes the original method as converting the image to black and white, dividing it into squares and quantifying the shading of each square. The goal was to resist changes such as resizing and minor color shifts. Since 2015, similar methods have also been applied to individual video frames.

Microsoft states that a hash is not reversible, so it cannot be used to recreate the image. It also says PhotoDNA is not facial recognition and cannot identify a person or object in a picture.

Who uses it, and where the new OpenAI tools fit

Microsoft says it provides PhotoDNA free to qualified organizations, including technology companies, developers and nonprofits. It also donated the technology to the National Center for Missing and Exploited Children (NCMEC). Wikipedia lists services such as Gmail, Twitter, Facebook, Adobe, Reddit and Discord among its users.

NCMEC keeps hash lists in several formats, according to the same source: PhotoDNA, PDQ, MD5 and SHA-1. Other groups, including the UK’s Internet Watch Foundation, maintain lists of their own.

OpenAI’s child safety package, covered in our report on the new classifier and ChatGPT’s 24-hour bans, targets the gap that hashing leaves. According to TechCrunch, the new classifier looks for visual signs of abuse in images that have not been seen before, unlike hash matching against known material.

Advertisements

The blind spot: It only knows what it has seen

Hash matching has one hard limit. It finds copies of images that are already on a list. Microsoft says plainly that it does not identify new or unknown content.

A photo taken yesterday has no fingerprint on any list, so a hash check passes it. The same is true for images made by generative AI, which are new files by definition. A matching system cannot flag them until someone reviews them and adds them to a list.

Hash lists also depend on human review. Each entry exists because a person or organization confirmed the original image. That keeps false matches rare in principle, but it means the system is only as complete as the lists behind it.

Access is also controlled. Wikipedia notes that platforms and law enforcement apply through Microsoft or sublicensing organizations such as the Technology Coalition. The design of PhotoDNA is kept private, partly because a published recipe could help people craft images that evade it.

What to watch as classifiers join the toolbox

Classifiers and hashes answer different questions. A hash asks whether this is a copy of something known. A classifier asks whether this image looks like abuse. The second question can catch new material, but it is a prediction, so it can be wrong in both directions.

OpenAI says its classifier, offered to approved organizations, helps prioritize images for human review. It reported that in early partner tests the classifier flagged 48% fewer images while finding the same number of abusive ones. Those figures are the company’s own and have not been independently verified.

Expect platforms to layer the two approaches. Hashes handle the large volume of known material quickly and cheaply. Classifiers triage what is new, and trained people make the final call. Things to watch are independent evaluations of classifier error rates, and how quickly newly confirmed images, including AI-generated ones, enter the shared hash lists.

Sign Up for the Daily AI Pulse

One email a day. All the stories that matter.

Daniel ReedAI Research, Safety & Ethics Analyst

Daniel Reed currently works as an AI Research, Safety & Ethics Analyst at Aiholics, writing about how changes in artificial intelligence are affecting and will affect scholarship,…

Leave a Reply

Your email address will not be published. Required fields are marked *

You may also like