Image Watermarking for Content Provenance

As part of our mission to embed trust in digital media, Trufo has released an open-source watermark benchmark library. Detailed instructions, for developers and researchers, on how to use the library can be found in the README file in the repository.

#Why a New Standard?

Digital watermarking has been around for half a century, and there is a substantial collection of literature on various techniques and technologies, as well as on the assessment of said methods. In the past few years, however, the raison d’être of digital watermarking has evolved significantly.

The culprit is the advancement of content generation capabilities. On one hand, the availability of phone cameras and the omnipresence of social media means that the main content generators are no longer just a handful of companies but rather the general public at large. On the other hand, the rapid development of generative AI capabilities means that digital content will be increasingly easy to create, mimic, or alter.

Existing evaluations for watermarks are tailored to the old priorities. While they are designed well for their purpose, many of the tested properties are not all that relevant. Here are a few examples:

• When assessing robustness, two common tests are the addition of (1) Gaussian noise and (2) salt & pepper noise. In practice, (2) most often arises in the analog-to-digital conversion, and is rare nowadays due to the widespread use of error-correction in data transmission. And while (1) may naturally arise at very small levels from color space rounding, in practice color filters (including simple gamma correction) are more common and more impactful.
• Many concerns are raised over security, and in particular, over secrecy. Oftentimes, security is dependent solely on the secrecy of the specific methods and parameters of the watermark, which does not scale well with mass adoption. The data that is carried is likewise restricted to a private indication of ownership, with the occasional inclusion of an ID to track down an offending user in the case of a copyright violation.
• PSNR is probably the most common way of assessing perceptibility. It is simple, and it is great because it is simple (we use it too). However, dig a little deeper and it is not all that representative of how human visual perception works.
• Overall, the focus is on preventing adversarial attacks in the context of controlled distribution. If a cable company wants to send a paywalled movie to a DVR console in a known AV encoding, they do not need to worry about the content being resized and compressed when sent over text or email, but they do need to worry about a hacker trying to pirate the content.

The watermark benchmark library Trufo is releasing instead focuses on addressing the modern problem. This problem includes:

• Distinguishing generated (AI) content from genuine (camera, human) content.
• Mitigating misinformation, by increasing the difficulty of creating and reducing the impact of receiving.
• Allowing the average creator to assert their content rights.

There are four primary characteristics associated with a good watermark for these purposes. This library serves as an effective tool to evaluate two of them: durability and invisibility.

#The Dataset

Currently, only images are supported. There are four image datasets.

• IMG_1: ten images, across a range of sizes, styles, and formats. Attributions are in the dataset mirror.
• IMG_VOC: 132 medium (HD-) images.
• IMG_BIG: 17 large (4K+) images.
• IMG_ART: 47 artistic works.

The latter three are compiled by Angela Tan, CS@Princeton ’25. The VOC and BIG selections are balanced across natural landscapes, urban landscapes, and people-centric photos.

#Evaluating Durability

The focus here, at least for the initial release of the library, is on benign content alterations. The first reason is that watermarks, and in particular cryptographic watermarks, serve as a mark of authenticity, so users are generally incentivized to keep them on. The second reason is that benign alterations, such as file downsizing and screenshotting, constitute the lion’s share of content alterations in practice.

A number of image edits are evaluated. Due to the large number of possible edits and the large number of test images, some degree of reproducible randomness is added to obtain a reasonable balance. The full list of edits includes:

• Simple exporting: lossless (PNG) and lossy (JPEG, 95Q).
• Various JPEG compression levels: 99Q, 90Q, 80Q, …, 10Q.
• Cropping: one side (2), all sides (2), random selections (4).
• Resizing: enlarge (2), shrink (2), aspect change (2), fixed sizes (2).
• Rotation: 90-degrees + reflections (3), fixed small angles (4), random angle (1).
• Filters: brightening (2), saturation + hue (2), grayscale (1), negative (1), blurring (1), sharpening (1), discretizing (1).
• Alterations: box overlay (2), lines overlay (2), lines removal (2), text addition (2).
• Composites: “post” ~ filter + alter + compress (2), “share” ~ resize + alter + compress (2), “screenshot” ~ resize + crop (2).

There are a total of 60 tests. Smaller test selection settings are also available (2 tests, 12 tests), and so is a false positive test setting.

#Evaluating Invisibility

Three separate metrics are used. We have found that the composite score is usually quite representative of true visibility.

The first is PSNR, a log-scale measure of the average magnitude of the noise (i.e. the watermark) added to an image. The second is SSIM, which factors luminance, contrast, and structure into a similarity metric, calculated over a number of windows covering an image; a transformation is applied in the analysis to place it onto a 0–100 scale like PSNR.

The third is an in-house metric, which we call PCPA (perceptibility A). The first step is to calculate a local perceptibility score. Because the human eye resolves luminosity at a higher precision than color, a larger blend parameter is used for the color channels. The scores across the three color channels are then summed, and a weighted mean is taken across the entire image; to achieve a reasonable balance between omnipresent light signals and localized heavy signals, a power factor of 1.5 is used in the mean. Furthermore, because small images and large images are different, to both humans and computers, the computed scores of the raw image and of a resized version are combined, and the value is placed onto a 0–100 scale.

The computation is neither simple nor elegant, unfortunately, but it does the job well enough.

#How Do Trufo’s Watermarks Stack Up?

Using the benchmark library, we evaluated Trufo’s watermarking capabilities. For comparison, we also evaluated the popular invisible watermark library, which has over 1000 stars on GitHub and is officially used by established generative AI companies such as Stable Diffusion.

Three primary metrics are measured:

• Durability: the proportion of edited images from which the watermark payload was recoverable without errors, over the 60 pseudo-random tests described above.
• Invisibility: an even blend of PSNR, SSIM (scaled to 0–100), and PCPA.
• Scalability: the amount of time, in milliseconds divided by the image size, the watermark encoding and decoding processes require.

These metrics are developed based on first principles, and have not been calibrated or manipulated to benefit Trufo’s watermarks. The IMG_1 dataset (10 images) was treated as a training set. The IMG_VOC dataset (132 images) was treated as a blind test set: the evaluation was run once and only once.

Note that Trufo’s watermarks do come with a suite of useful functionalities, such as content provenance and fuzzy authentication, that are not tested in the evaluation.

Trufo’s watermarks outperformed the reference implementation by a few orders of magnitude.

For a deeper dive, here is the summary table of results.

There are a couple of analytical takeaways here:

• The reference implementations slow down significantly for larger image sizes (IMG_1 contains two ~4K images; IMG_VOC contains none). Trufo’s watermarks do not; in particular, the heavy watermark that is substantially slower on small images becomes substantially faster on large images (by 2x and 8x versus DwtDct and DwtDctSvd, respectively; RivaGAN crashed the kernel).
• The PCPA metric is very friendly towards DwtDct and very unfriendly towards DwtDctSvd. From a quick human inspection, DwtDct is indeed far less visible than DwtDctSvd. On the flip side, Trufo’s medium and heavy watermarks have similar invisibility scores, but in practice the heavy version does tend to be more noticeable.
• The AI-only model (reference RivaGAN) requires a large amount of compute. With a hearty setup of H100 graphics cards and proper GPU optimization, the watermark decoding time can probably be reduced to a practical level, but it is tough. Trufo’s decoding time, in contrast, is consistently quick.

There is still plenty of room for research and improvement in the space of digital watermarking. Stay tuned for updates from Trufo.