Who Reviews Flagged Photos at Google, and What Their Review Is Worth.

For many suspected CSAM and-related crimes, Google’s automated screening system only triggers a “review” by a person. Specifically, Google uses trained human reviewers to double-check all flags (or a large percentage of them) and it only transmits photos to U.S. federal authorities after human review is completed. Similar to how law enforcement officers use photo screening software to process large-scale seizures, Google’s algorithmic screening software does the first round of review and then passes the flagged content to human reviewers to verify.

But what can we say about Google’s human reviewers? We can say both a lot and very little.

First, Google likely does not employ the photo reviewers as direct employees. Like virtually all online platforms, Google outsources content moderation. This means that Google’s staff members have probably been replaced with contractors, sub-contractors, and perhaps the third-party staffing firm employees. The photo reviewers are likely private contractors with limited knowledge of the files they review, and their reviews are highly influenced by Google’s policies and procedures.

Second, Google’s human reviewers are private citizens. The Fourth Amendment, which protects against unreasonable searches and seizures, does not bind private citizens unless they act as agents or accomplices of law enforcement. If Google’s photo review is carried out as a private business function (i.e. if a human reviewer determines that some content should not be sold or shared on Google’s platform due to a violation of Google’s Terms of Service), then the Fourth Amendment does not apply to Google’s human reviewers.

This means that, although you have a constitutional right against unauthorized searches by the government, you do not have a constitutional right against unauthorized searches by private computer systems.

Q: How does Google decide which photos to review?

According to Google’s transparency report, Google uses “a combination of perceptual hashing and other tools” to identify and report CSAM. The first tool mentioned, perceptual hashing, is a standard method used by online services today. This is because files that contain the same visual content can have completely different hash values. It is impossible to keep a complete, permanent database of all the same images because images can be resized, cropped, recolored, or have their data changed in other ways. Perceptual hashing helps search companies like Google keep track of images as they are uploaded. For example, the tool PhotoDNA converts each image into a perceptual hash that survives common modifications and resizing, and this allows companies to store a database of all known child pornography that is constantly updated by law enforcement.

Google mentions “other tools” and refers to its Content Safety API, which uses “machine learning classifiers” to “identify and flag content never previously catalogued.” Classifiers find images that have characteristics similar to known child pornography, even if the images are unique. These classifiers express probability, and the fact that a classifier identified a picture as “likely” CSAM is not enough to qualify it for reporting to the National Center for Missing & Exploited Children (NCMEC). This means that if the human reviewer confirms the classifier’s determination, then it is human review that allows the picture to be reported.

In practice, what does “review” mean? This is a critical question for Google’s human reviewers. Some reports leave Google without any employee actually opening the file. In these cases, the report is based on a match with PhotoDNA. In other cases, the employee must open the file to review the photo and apply one of six potential labels: prepubescent, pubescent, under-age, and other, as well as three specific CSAM categories (child sexual abuse, child sexual exploitation, and child nudity) that distinguish based on age.

The labels are either based on the age of the subject or the act portrayed. Reviewers determine the subject’s age based on their visual appearance, not birth records or medical evidence. If the review requires a label, the human reviewer is required to first apply a label, like “prepubescent” or “pubescent,” and then “appropriate industry-recognized labels.” These labels classify the photos as apparent CSAM, and they do not provide any legal determination about whether they represent prohibited content or depict underage victims.

Google includes this information about the photo review process on the CyberTipline form that is used for reporting images. On the form, Google marks whether a human saw the photo and if it is “certain” the photo represents child sexual abuse material. This means that some reports from Google are based entirely on machine learning, and some reports from Google are based on a human’s “certain” visual assessment.

Q: What happens after Google sends a CyberTipline report?

Once Google compiles all the information it has gathered, it sends the reports to NCMEC (the National Center for Missing & Exploited Children). The report includes the user’s IP address and the flagged image’s metadata. U.S.C. § 2258A makes reporting to NCMEC mandatory. Specifically, section 2258A(a) says:

“If a provider of electronic communication service or remote computing service (or the parent company of such provider) is aware that an electronic communication, a transmission, or an image, in whole or in part, constitutes child pornography, such provider shall

“(1) respond as soon as reasonably possible to the child sexual abuse material (CSAM), with respect to which information has been made available. This includes forwarding a report to the National Center for Missing and Exploited Children (NCMEC) that contains all the available and useful information.”

The reporting duty imposed by section 2258A does not create a duty to affirmatively search for CSAM or otherwise monitor users. Specifically, section 2258A(f) states:

“Nothing in this section shall be construed to require a provider to, (1) monitor any user, subscriber, or customer of that provider; (2) monitor the content of any communication of any person described in paragraph (1); or (3) affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b).”

The scope and frequency of the reporting duty are also limited. A provider must report violations of the federal statutes “as soon as reasonably possible” after receiving actual knowledge of the crime. The reports themselves are very similar in form. And, although section 2258A(b) says a report must identify the subject of the child pornography, it says the provider may do so “to the extent” that “it is possible.” Most of the requirements in section 2258A(b) are optional for providers, and the provider is not required to forward the flagged image or a thumbnail.

NCMEC is a private nonprofit organization that works with the U.S. federal government. NCMEC has a CyberTipline and is responsible for forwarding reports from electronic service providers and others to law enforcement agencies. As noted above, in the U.S. District Court case of United States v. Ackerman, 831 F.3d 1292 (10th Cir. 2016), NCMEC had to respond to a class action lawsuit after opening files that electronic service providers had not reviewed and forwarded to NCMEC for dissemination to law enforcement agencies. Today, NCMEC continues to forward these reports and works in coordination with federal authorities and other jurisdictions.

Finally, under section 2258A, providers that knowingly and willfully fail to report violations of the child pornography laws are subject to a fine of not more than $600,000 for an initial failure, or $850,000 if the provider has at least 100,000,000 monthly active users. For a second or subsequent failure, the fine is not more than $850,000, or $1,000,000 for a provider with at least 100,000,000 monthly active users.

Q: Can the police open my flagged photo without a warrant?

As a general rule, the police may open your flagged photo without a warrant. In United States v. Jacobsen, 466 U.S. 109 (1984), the Supreme Court explained that the government may do “nothing more than replicate the private search” without violating the Fourth Amendment. But Jacobsen also noted that this means the government can’t “inspect any items that were not already within the scope of the private search.”

The extent to which the Fourth Amendment protects users of electronic services is a matter of considerable discussion in courts around the country. A few cases illustrate the split on the issue. In United States v. Ackerman, 831 F.3d 1292 (10th Cir. 2016), an employee of the National Center for Missing & Exploited Children (NCMEC) opened and viewed the contents of four attachments that America Online (AOL) had automatically flagged and reported to NCMEC after its Image Detection and Filtering Process matched the hash value of one of them. The Court of Appeals for the Tenth Circuit treated NCMEC as a government entity and held that the employee’s act of opening and viewing the attachments constituted a search that required a warrant. The court reasoned that opening the attachments exceeded the scope of AOL’s private search, because AOL’s automated system had only “opened” the files in the sense of analyzing their hash values without actually viewing the images.

The Court of Appeals for the Fifth Circuit reached a different result in United States v. Reddick, 900 F.3d 636 (5th Cir. 2018). Like in the Tenth Circuit, Reddick’s provider had flagged three files as appearing to be CSAM. The files were then routed to NCMEC, which used the accompanying IP address information to forward the CyberTips to the Corpus Christi Police Department without opening them. Law enforcement officers then opened the files. Here, too, the evidence contained in the flagged files was the basis for the warrant that a local police detective obtained to search Reddick's home. Reddick filed a motion to suppress the flagged files, asserting that the opening of those files by NCMEC and the government was unconstitutional. But the Fifth Circuit found that by opening the flagged files, the government “replicated” the search conducted by the provider and NCMEC. To the Fifth Circuit, the act of flagging a file for human review already constituted opening it, and opening it after it was flagged didn’t require a warrant.

More recently, in United States v. Wilson, 13 F.4th 961 (9th Cir. 2021), the Ninth Circuit suppressed four images because no Google employee had viewed them. Similar to the fact pattern in Ackerman, the government relied on a private search to justify opening the files. But the Ninth Circuit agreed that “opening the files constitutes a search.” Wilson explains: “No one at Google had opened or viewed Wilson's email attachments; its report was based on an automated assessment that the images Wilson uploaded were the same as images other Google employees had earlier viewed and classified as child pornography.”

Thus, current circuit case law is split over whether opening a flagged file that had not been opened by a human reviewer is a replication of a private search or is an independent search. This split has the potential to create an uneven enforcement landscape, particularly because many users of electronic services that conduct hash-based scanning are in states within the Fifth and Ninth Circuits.

The issue of whether any restriction applies to how law enforcement opens a flagged photo turns on whether the private party’s actions were carried out with the “knowledge and acquiescence of the government, with the private party intending to assist the government,” a standard adopted by the Fourth Circuit.

If any of this describes your situation, it is worth talking it through with counsel. Spodek Law Group can be reached at 212-300-5196.

Q: What does a Google reviewer’s label actually prove?

If the government is relying on a hash match, that hash match does not prove the actual contents of the file. Instead, a hash match can only prove “sameness” to some other file. For this “sameness” to mean anything, a human must have judged the initial file’s contents. And, unless a human judged the initial file’s contents, then a hash match doesn’t really prove anything.

Also, because hash databases are private and unpublished, a defense expert cannot independently verify that a claimed hash match is accurate.

If the evidence against you contains content from a hash match, then there are several other issues you need to consider. First, the government must prove that an actual child appears in the image. This means that the content of the image should be reviewed by a defense expert, the image should be authenticated under Rule 901, and the image’s contents should be analyzed under Rule 702. As a matter of procedure, all digital images must be authenticated before they can be admitted as evidence in court. Similarly, hashing, classifiers, forensic attribution, and other forms of expert evidence must be examined in accordance with the Rule 702.

While the government uses CSAM labels to build cases, having a CSAM label from a private entity like Google is not an element of 18 U.S.C. § 2252A. In order to meet the burden of proof for this federal offense, the government must establish the actual contents of the image and the circumstances of your possession of the image.

When the government charges someone with knowingly possessing an image of a child, federal prosecutors typically focus on proving “possession” by showing the image is in your account. But “possession” of an image means “control” of the image, and having control over an image is a critical factor in determining whether a defendant knew about the content in an image. It is easy for law enforcement to say, “Here is the image in the defendant’s Google account.” But account ownership is not proof of possession. Having a Google account is not proof of access, and having access is not proof of possession.

If you are facing charges for knowingly possessing CSAM, your defense lawyer needs to use both evidence and the law to counter the government’s evidence. The government often tells defendants that fighting a CSAM charge is an uphill battle, but this is only true if you are fighting on the government’s terms.

Q: What records show what Google and the police actually did?

The presence of an image in the CyberTipline report (or the provider’s failure to include the image) is only one piece of the puzzle. The specific reason for the image being sent is often the only other piece of information in the CyberTipline report. That specific field is sometimes left blank, and it may sometimes be completed inaccurately by the service provider. It is for this reason that defense counsel will need to subpoena the service provider’s internal escalation logs, rather than relying exclusively on the information available in the CyberTipline report. A provider’s internal escalation log typically contains a lot more information than the CyberTipline report because it provides a detailed record of the process that took place before the report was sent.

For example, a service provider’s detection logs can show whether the provider’s system flagged the image via a hash match or via a classifier. If the report was based on a hash match, then the hash value itself can be used to compare the files reported by the provider with any copies the government has in its possession. If the flagged file was an unknown unique, the provider’s internal notes can clarify whether there is evidence of child sexual abuse, sexual exploitation, or nudity, and the provider’s logs can help a defense expert determine whether a human reviewed the file or if the human’s review was an automatic approval of a classifiers’ determination.

From a defense perspective, there are also several other pieces of information that can help establish (or fail to establish) the elements of the offense. One is an account-access log. An account-access log can help show whether the content was uploaded by a user or if the presence of a flagged file in the user’s account was a result of an automated synchronization event. This is a critical distinction because knowingly possessing CSAM is an intentional offense.

If the files being been stored at Google’s headquarters or some other cloud storage facility, then it is important to determine if law enforcement requested that the files be opened prior to being sent to the government. To do this, a defense attorney should subpoena communications between the law enforcement officers and the platform. Many officers encourage platforms to scan users’ accounts as a way of collecting and presenting evidence that might be seen as less intrusive than performing an independent search. The communication logs, the subject of the warrant (or subpoena), and the images the police have in their possession can help establish the timeline of the private and government’s involvement, and this can help determine when a warrant was required.

And, if any human at Google reviewed the image before sending it, their notes may contain information regarding uncertainty, escalation to a supervisor or compliance team, and other details about the reason for applying the label of “apparent child sexual abuse material.”

Q: What should I do if the police contact me?

If you are the target of a CSAM or child pornography investigation, it is important to understand the Fourth Amendment. Generally, the Fourth Amendment requires that law enforcement obtain a warrant signed by a judge before conducting a search of a suspect’s home, device, account, and any other items. However, one of the most widely used exceptions to the Fourth Amendment warrant requirement is consent. In many cases, law enforcement officers can walk into a home or convince someone to give them access to a phone or computer.

While the Fifth Amendment grants you the right against self-incrimination, it is not possible to enforce this right with 100% accuracy without any prior knowledge. If you are being questioned, you have the right to decline to answer, to speak with counsel, and to terminate the questioning.

If you are charged with CSAM possession (or being identified as a “suspected” CSAM user), this is an extremely serious matter. If you need to defend yourself against an allegation of child exploitation or sexual abuse of children, you will need a highly experienced criminal defense attorney. However, for several reasons, a lawyer should not claim that they are the best lawyer. According to ABA Model Rule 7.1, lawyers cannot make “false or misleading” statements. This includes statements that cannot be factually verified. In fact, many state bar associations adopted some version of this rule because attorneys often made promises about how a client could expect their cases to go. Also, the U.S. courts do not certify lawyers as the “most successful child pornography defense attorney,” and the American Bar Association does not certify lawyers’ success rates.

If you speak to a CSAM defense lawyer, remember that the attorney-client privilege covers all of the lawyer’s knowledge of the facts at hand. This means that your defense lawyer will not disclose your case, the identity of your alleged victim(s), or your past behavior to anyone unless you give written permission to do so. This applies to cases involving child pornography and other crimes as well.

Contact a Federal Criminal Defense Attorney

Nothing here is legal advice, and the details of your case matter. Todd Spodek and Spodek Law Group take federal criminal and white collar cases nationwide, from offices in New York, Brooklyn, Queens and Los Angeles. You can reach the firm at 212-300-5196.