Hash-based tools for content moderation
Hash-based tools for content moderation
A method for detecting illicit content on social media platforms
TL;DR
This post explores how hash-based detection tools work and how they are used for content moderation on internet platforms.
In a previous post I explained what content moderation is and given an overview of how it generally works across different internet platforms:
Content moderation is where an internet platforms reviews content or communications on its service to identify that which is either illegal or breaches its terms of service (ToS). In the social media context, user-generated content (UGC) will be the type of content that these practices will be applied to.
A more comprehensive explanation can be found in the post below:
Hash-based detection tools form one of several mechanisms that internet platforms could use to identify content that is illicit (i.e., content that breaches the ToS or is otherwise illegal) and thus form part of their content moderation systems. These tools convert images that are being uploaded to a platform into a hash value and then compares that hash value to other hash values of content that has been previously identified as illicit. If there is a sufficient match, then the content can be flagged as potentially illicit and dealt with accordingly by the platform.
Such tools are used for detecting a variety of illicit content, including copyright-infringing content, terrorist content and even child sexual abuse material (CSAM). The post is therefore relevant to the EU's proposed Online CSA Regulation currently going through the legislative process.
Some helpful sources, including books and technical papers, are provided at the end of this post. Feel free to leave a comment below and subscribe if you like the content.
Hash-based detection tools
The main steps for hash-based detection are the following:
Image pre-processing - certain adjustments are made to the image being uploaded to the platform to aid the detection process.
Feature detection - the key features of the image are identified and condensed into a numerical form.
Hash value generation - the key features are converted into a hash value using a hash function.
Hash value matching - the hash value is compared to other hash values of illicit content to compare how similar the values are; if there is sufficient similarity, then the content being uploaded is flagged as being potentially illicit.
1. Image pre-processing
When computers work with images, they do so by processing the image as a set of numbers. This is done by essentially converting any given image into a grid of pixels, the smallest possible element of an image.
Take the cat image below:
In order for a computer to display this image, it would need to render it from the set of pixels that make up the image. In fact, what the computer will see when processing the image is the following:
[[131 103 63]
[135 107 67]
[138 110 70]
...
[55 49 53]
[52 46 50]
[49 43 47]] The above is an array of all the pixels in the cat image. Pixel arrays consist of rows of values representing each pixel in the image. In this case, the cat image has a dimension of 1254 by 837 pixels, meaning that the image contains a total of 1,049,598 pixels (1254 multiplied by 837). This means that the pixel array will have 1,049,598 rows of values. Each of these pixels are made up of red, green and blue colour channels, hence each row in the pixel array will contain three sets of values representing the level of the RGB channels. For example, for the first row, the red channel is 131, the green channel is 103 and the blue channel is 63. When the computer renders the image, it is doing so by combining the RGB values in each pixel together which will generate the cat image above.
For hash-based detection, the essential aim is to identify the key features of the image and then produce a hash value from these features. To make it easier to achieve this, hash-based tools typically do not process the raw image. Instead, it will make certain adjustments to the image to make the hashing process more efficient.
One adjustment that will typically be made to the image is to turn it into a black and white image as shown below:
When making this adjustment to the image, the pixel array will look slightly different:
[107 111 114 ... 51 48 45]Now, instead of having RGB channels represented by three sets of values, each pixel in the image is represented as a single value between 0 and 255 where 0 is black and 255 is white. Thus, the last value in the pixel array above for the black and white cat image is 45, meaning that this particular pixel is quite close to the colour black. Putting all these pixels together allows the computer to render the black and white image above.
The next adjustment that is made is to resize the image. This is because, in the real-world environment in which the hash-based detection tool will be deployed, the images being uploaded to the platform will vary in size. As such, for consistency in the detection process, the detection tool will resize each image to a particular size. For instance, the cat image below has been condensed down from 1254 by 837 pixels to 360 by 360 pixels (giving it a total of 129,600 pixels):
The final adjustment typically made by hash-based detection tools is to highlight the edges in the image. The edges of an image are those parts where there is a sharp contrast between pixels, usually the brightness of the pixel. When this adjustment is made, it will essentially look like a tracing of the outline of the main objects in the image, like below:
These various adjustments make up the pre-processing applied to images by hash-based detection tools before it then goes on to detect the key features of the image. By taking out its colour, resizing it and exposing the edges, the tool is narrowing a complex set of data down to the most essential components so as to make feature detection more straightforward and quicker.
2. Feature detection
Image feature detection is all about taking an image and identifying its most significant elements. This could be its shape, colour, patterns or other elements that are visually prominent.
There are variety of methods that can be used to identify the features of an image. One complex method is to use a convolutional neural network, which is a deep learning model that uses layers of neurons to process the image and find its most significant patterns.
Hash-based detection tools can use this method (as was the case with Apple's proposed NeuralHash, more on this later). However, there are other methods that are not as computationally intensive. This includes the algorithms developed by OpenCV (Open Source Computer Vision Library), which is a library of open source computer vision and machine learning software.
The feature detection algorithm from OpenCV explored in this post is called Oriented FAST and Rotated BRIEF (ORB). This algorithm is made up two parts:
Features from Accelerated and Segments Test (FAST) Keypoint Detector
Binary Robust Independent Elementary Features (BRIEF)
FAST Keypoint Detector
FAST is a feature detection technique focused on detecting corners in an image. It does this by comparing different pixels together to identify patterns that may resemble a corner. More specifically, for each pixel in the given image, FAST will identify 16 pixels surrounding a central pixel and compare that central pixel to surrounding pixels 1, 5, 9 and 13, which is shown in the diagram below:
When comparing the pixels, FAST is looking at whether surrounding pixels 1, 5, 9 or 13 are either brighter or darker than the central pixel. If at least three of these surrounding pixels are brighter or darker beyond a certain threshold, then that central pixel is identified as a corner (or a keypoint).
ORB uses a specific technique called 'oriented FAST'. In simple terms, this means that ORB uses an augmented version of FAST that is capable of identifying keypoints at different orientations. This helps to make the feature detection process more robust.
BRIEF
After keypoints have been detected using oriented FAST, BRIEF is used to generate what are known as 'descriptors.' The purpose of these descriptors is to capture certain information about the keypoints that have been identified in the image. In other words, the algorithm is using certain information about the keypoint to describe it and distinguish it from other keypoints that have been identified.
BRIEF describes the keypoint using both the specific pixel identified as a corner (point) and its surrounding pixels (patch). It does this by taking each pixel in the patch and comparing them to each other by their intensity. These comparisons are then recorded as boolean values inside a vector. For example, if the first point of the patch has a higher intensity than the second point, then this is recorded as 1 (meaning 'True'). Otherwise, it recorded as 0 (i.e., 'False'). This results the keypoints being described as a feature vector consisting of strings of 0's and 1's, which would look like something below:
[[011010000110]
[111010101101]
[111110100010]
...
[001011101011]
[010011101111]
[010101111001]]ORB uses a version of BRIEF that is 'rotated'. This means that the orientation information that is collected from oriented FAST is used to rotate the descriptors. In other words, the feature vector generated by rotated BRIEF will contain information about the keypoint at different orientations, again for the purpose of making feature detection more robust.
ORB
When oriented FAST and rotated BRIEF are combined together to detect the features of the cat image above, a total of 15552 features are identified. These can be mapped back on to the input image to show the location of each of the identified features:
These features shown on the cat image are derived from an array of the features that ORB generates, which will look something like below:
[245., 83., 104., ..., 132., 139., 105.]3. Hash value generation
In a hash-based detection tool, these features will be used to produce a hash value using a hash function.
What is hashing?
A hash function is an algorithm that takes any input (such as an image) of any size and from this produces a seemingly random output.
The output of a hash function (i.e., what they produce from the input) is known as a hash value (or a hash or a digest). A hash value is a string of alphanumeric characters that will look like the following:
f63e68ac0bf052ae923c03f5b12aedc6cca49874c1c9b0ccf3f39b662d1f487b(A side-note: hash values can be represented as alphanumeric characters, also known as a hexadecimal, which uses a combination of any number from 0 to 1 and any letter between a and z. The hash value could also be shown in a binary form using 0's and 1's. However, the hexadecimal encoding is slightly more readable and takes up less space.)
Hash values have three important properties:
They are always the same length: no matter the hash function, the hash value is always the same size. For example, SHA-256 (Secure Hash Algorithm), which is a popular hash function, always generates a hash value with 64 alphanumeric characters that are 256 bits in size. If using SHA-512, then the hash values produced will have more characters and be 512 bits in size.
They are deterministic: the hash value will always be the same for each input. In other words, each input will have its own unique hash value.
They cannot be reversed-engineered: it is usually very difficult (if not close to impossible) to use the hash value to recreate the original input. This is why hash functions are often described as 'one-way' algorithms.
These three properties are crucial for hash-based detection tools. That hash values are always the same length and size no matter the input makes them easier to store and to compare to each other (more on this below). That hash values are deterministic allows each piece of content being processed by the tool to be uniquely identified. Not being able to reverse-engineer hash values is also helpful from a privacy/data protection perspective (in terms of ensuring confidentiality).
4. Hash value matching
The final step in the hash-based detection process is comparing the hash value of the uploaded content to a database of hash values from content previously identified as illicit. For this there are two important functions at play here, which are perceptual hash functions and hamming distance functions.
Perceptual hash functions
SHA-256 is an example of a cryptographic hash function, which are hash functions that are typically used for security-based purposes, such as converting pass phrases into cryptographic keys. However, these type of hash functions are not actually useful for the purpose of hash-based detection tools for content moderation. This is because of something called 'the avalanche effect'; this is where a slight change in the input results in a completely different hash value. The reason that this is not helpful for content moderation is because the aim of content moderation is to identify pieces of content that are similar to previously identified illicit content. If the hash function used for this produces very different hash values for content that is similar to each other, this will be difficult to detect this similarity by comparing the hash values as the values themselves will not be very similar.
Thus, instead of using a cryptographic hash function, hash-based detection tools will use perceptual hash functions. These are algorithms that are designed to produce very similar hash values for inputs that are very similar and very different hash values for inputs that are different. Accordingly, such functions are far more useful for content moderation as the hash values can be reliably used to determine whether content being uploaded to a platform is similar or dissimilar to illicit content that needs to be flagged.
Hamming distance functions
Hamming distance is a method for measuring how much two strings of equal length differ from each other. This is achieved by looking at each position in the given strings and determining whether those positions are the same or different, and then providing the total number of positions in the two strings that are different. Hence, the lower that number, the closer the two strings are and vice versa.
For example, take the following two strings:
string_1 = 101010
string_2 = 111000If applying a hamming distance function to these two strings, the hamming distance would be 3. This is because the two strings differ in three positions.
A hash-based detection tool can use a hamming distance function to identify hash values belonging to content that is illicit; it can be used to compare the hash values of content being uploaded to a platform to a database of hash values of previously identified illicit content. If the hamming distance is below a particular threshold, then it can be flagged as potentially illicit.
Putting it all together
The diagram below shows the whole process applied by a hash-based detection tool comprising of (i) pre-processing, (ii) feature detection, (iii) hash value generation and (iv) hash value matching:
Real-world examples of hash-based detection
A few examples of hash-based detection tools include:
Microsoft PhotoDNA: This is a tool developed in 2009 by Microsoft Research and Henry Farid, a professor from Dartmouth University. Microsoft makes PhotoDNA available "for free to qualified organizations including technology companies, developers, and non-profit organizations for the purpose of combatting child exploitation" and has also provided the tool to law enforcement.
Facebook PDQ and TMK+PDQF: In 2019, Facebook open-sourced hash-based detection tools that it used for detecting harmful content on its platform like "child exploitation, terrorist propaganda, or graphic violence." PDQ can detect such content in images while TMK+PDQF can be used for videos.
Apple NeuralHash: In 2021 Apple proposed a number of technical measures designed for enhancing child safety on its services. Among these was its CSAM detection tool called NeuralHash, which was a hash-based detection tool applied to images uploaded to iCloud Photos. Apple had planned to deploy and run the tool on people's devices (thus constituting a form of client-side scanning) but withdrew such plans due to public backlash. The technical summary for NeuralHash still provides an interesting example of using hash-based detection tools to detect CSAM using a client-side scanning approach.
Sources
Technical Papers
Edward Rosten and Tom Drummond, Machine learning for high-speed corner detection (2011)
Michael Calonder et al, BRIEF: Binary Robust Independent Elementary Features (2010)
Ethan Rublee et al, ORB: an efficient alternative to SIFT OR SURF (2011)
Books
David Wong, Real-World Cryptography (Manning Publications 2021)
François Chollet, Deep Learning with Python (2nd edn, Manning Publications 2021)
Other Sources