A Bit Off-Color
Sunday, 20 April 2014
FotoForensics just hit another major milestone yesterday. It received its 500,000th unique image upload! That's a half-million pictures! There were a bunch of things that I had wanted to release to celebrate this achievement, but most of it isn't done yet. (Either you guys uploaded too fast, or I am programming too slow.) So I enabled the one feature that is currently ready: Digest. It's listed as an analyzer option and there is also a tutorial for it.
A digest is a basic summary that can be used to identify a specific image file. Usually it's a complex hash, but it can be as simple as the picture's dimensions and file size. The digest lets the analyst know that they are analyzing the correct file. If the file is altered, then the digest changes. With cryptographic digests, such as MD5 or SHA1, a modification as small as 1 bit is detectable. (You will know that the file is different, but not what was changed, who changed it, or when it was changed.)
For people who analyze files for a living, this makes perfect sense. Recording the cryptographic hash value and image attributes are pretty much the first thing you do. And if you are handed a file to evaluate, the first thing you are supposed to do is make sure the digest information matches. The last thing an analyst needs is to waste hours evaluating the wrong file.
These digests also go toward tamper detection and chain of evidence handling. If the digest ever changes, then you know something is wrong.
Of course, a lot of people just go by what the picture shows. For example, I can show you a picture of some Easter Eggs.
Since we are all looking at the same picture, it is easy enough for us to talk about it. For example, there are three eggs. The left egg is the largest and the front egg is leaning on its side. And from front to back, they are colored purple, yellow, and blue.
What? You do see purple, yellow, and blue, right? Uh... you don't? What colors do you see? Let's make sure that we are talking about the right file...
The easiest way to describe this is to walk into an electronics store and look at the wall of TVs. Every TV may be displaying the same show, but the colors will all be a little different. Red on one TV will be vibrant, while another TV may be dull or even a little purplish. With ICC color profiles, the video signal can be shifted to a uniform color space and then converted to colors that better match the monitor. This way, red on your screen will look like red on my screen.
As it turns out, there are two common versions for ICC Profiles. Version 2 is the most widely adopted, but version 4 is also supported by some applications. If the ICC Profile contains version 2 and version 4 information, then the application will use whatever it supports. If your browser supports version 4, then it will use that color correction information. If it only supports version 2, then it will apply the version 2 information. And if it does not support color corrections, then it will display the basic picture.
There are a couple of web sites that offer pictures designed to identify your application's level of compatibility. There is the ICC Test with four pictures, the same test at Microsoft, and a cool car photo at PetaPixel.
The egg picture is my own extreme version of this same color management test. It contains a custom ICC Profile that screws up the colors based on which color management is used.
Here's the test picture again:
And here's what it looks like based on your browser's support:
However, I didn't stop there. A few vendors, like Microsoft, have created their own proprietary extensions to the ICC Profile format. If Microsoft sees their custom extension, then they will use it instead of the version 2 or version 4 information. My egg picture has one of these extensions. I only tested it on Windows 8.1, but it should work on any Windows Vista or later system. The biggest issue is that Microsoft is inconsistent in their application. The same file will look different under Internet Explorer compared to Microsoft Photo Viewer.
Unfortunately, I cannot find any documentation about this custom Microsoft extension. I really have no idea what type of transformation it is performing. I only know that Microsoft uses their extension and renders pictures differently based on the application.
Without the Microsoft extension, Internet Explorer and Microsoft Photo Viewer will render the ICC Profile version 4 image.
On every operating system and application, a color management system (CMS) is used to apply the ICC Profile. There are a few open source CMS systems, but Microsoft, Apple, and Adobe each have their own systems. With each system comes compatibility issues and computational differences. For example, Apple's CMS on 10.5.8 mostly looks like the ICC version 2 image, except that Apple's blue egg looks a little more purplish. The purplish value may be due to my monitor's own color profile, so your picture may look different if you use a Mac.
When loading the picture into Adobe Photoshop CS5 for the Mac, I am prompted to "keep" the current profile, "convert" it to Adobe's color space, or "discard" the profile information. Each option creates very different results:
But even minor changes made by the file format can result in significant differences. Here is the original egg picture saved twice: the left is a PNG and the right is a JPEG made from the PNG. Below each picture is what it looks like when colorized using the open source Little Color Management System (LCMS):
LCMS supports ICC version 4, so the result on the PNG is as expected. However, JPEG is a lossy file format and can cause pixels to shift in value a little. These shifts appear as a new color due to a little numerical error in LCMS. (If LCMS has a little numerical error that causes some purple on the background, then Adobe must have a ton of error since they have a solid purple background!) Then again, since no two applications seem to render the exact same image, who is to say which one is "incorrect"?
The problems with color transformations do not end with the ICC Profile; ICC Profiles are not the only way to store transformation information. For example, the PNG file format also supports its own color profile information in cHRM and gAMA tags. If a PNG contains both internal color information and an ICC Profile (like my egg example), then the application should use whatever it can. If it supports ICCv4, then use that information, otherwise use ICCv2, else use the PNG color information. And if all else fails, render the picture without any color corrections. Here's what the image looks like if your application only supports the PNG color information:
For example, applying an ICC Profile is explicitly a color transformation. In general, each pixel has the opportunity to shift a little in the color space. Since pixels can change values, any shifting results in a higher error level potential. Applying an ICC Profile or PNG color transform will alter the results from an error level analysis.
Color transformations can also impact signal-to-noise ratios, light direction, and even cloned-region detection.
When performing any type of digital forensics, you want to minimize the amount of transformation made when trying to render the picture. (At FotoForensics, we explicitly do not apply any color transformations. This way, we do not evaluate any artifacts that could be created during the color transform.)
The result is pretty clear: I have one picture (a PNG) that yields NINE different color sets! (Ten if you convert it to JPEG and use LCMS to render it.) The colors that you see are strictly dependent on the specific program that you use to view the image. Even something as minor as calibrating your video driver or patching your software could alter how the image is displayed.
You may be evaluating one file, but it can appear at least ten different ways!
All of this comes back to the digest. You and I are likely seeing two different sets of egg colors. So how do we know that we are looking at the same file? The answer is: check the digests.
My egg picture is a PNG that is 295x224 and uses 3 color channels. The file size is 33,492 bytes. It has an MD5 checksum of b513db97c45848fae40ef55c5e0fdfd5, and the SHA1 checksum is e53cae13e50d5556b6f2021000ff7fbe171561f8. If your file has all of these same attributes, then we are looking at the same file. Now we can start asking why the picture looks different if it is the exact same file.
And remember: it does not take much to alter the picture. If someone uploads this egg picture to Facebook, Twitter, or some other social media site, then the digest will probably change. And a change means that you will probably be seeing different colors.
Update 2014-04-21: Added notes for people who understand ICC Profiles and want more information about the cause of the image differences. Identified why Adobe looks so weird, and clarified why Microsoft displays two different images that are both technically correct.
A digest is a basic summary that can be used to identify a specific image file. Usually it's a complex hash, but it can be as simple as the picture's dimensions and file size. The digest lets the analyst know that they are analyzing the correct file. If the file is altered, then the digest changes. With cryptographic digests, such as MD5 or SHA1, a modification as small as 1 bit is detectable. (You will know that the file is different, but not what was changed, who changed it, or when it was changed.)
For people who analyze files for a living, this makes perfect sense. Recording the cryptographic hash value and image attributes are pretty much the first thing you do. And if you are handed a file to evaluate, the first thing you are supposed to do is make sure the digest information matches. The last thing an analyst needs is to waste hours evaluating the wrong file.
These digests also go toward tamper detection and chain of evidence handling. If the digest ever changes, then you know something is wrong.
Of course, a lot of people just go by what the picture shows. For example, I can show you a picture of some Easter Eggs.
Since we are all looking at the same picture, it is easy enough for us to talk about it. For example, there are three eggs. The left egg is the largest and the front egg is leaning on its side. And from front to back, they are colored purple, yellow, and blue.
What? You do see purple, yellow, and blue, right? Uh... you don't? What colors do you see? Let's make sure that we are talking about the right file...
Colorful Eggs
The colors seen when a picture is displayed are not limited to the pixels in the image. Most common picture formats also support the addition of a color profile. The International Color Consortium defines the common color profile format (ICC Profiles). The profiles allow the application to shift the colors based on the display.The easiest way to describe this is to walk into an electronics store and look at the wall of TVs. Every TV may be displaying the same show, but the colors will all be a little different. Red on one TV will be vibrant, while another TV may be dull or even a little purplish. With ICC color profiles, the video signal can be shifted to a uniform color space and then converted to colors that better match the monitor. This way, red on your screen will look like red on my screen.
As it turns out, there are two common versions for ICC Profiles. Version 2 is the most widely adopted, but version 4 is also supported by some applications. If the ICC Profile contains version 2 and version 4 information, then the application will use whatever it supports. If your browser supports version 4, then it will use that color correction information. If it only supports version 2, then it will apply the version 2 information. And if it does not support color corrections, then it will display the basic picture.
There are a couple of web sites that offer pictures designed to identify your application's level of compatibility. There is the ICC Test with four pictures, the same test at Microsoft, and a cool car photo at PetaPixel.
The egg picture is my own extreme version of this same color management test. It contains a custom ICC Profile that screws up the colors based on which color management is used.
Here's the test picture again:
And here's what it looks like based on your browser's support:
No color profile support (most mobile devices) | |
ICC Profile version 2 support (most versions of Firefox and Chrome) | |
ICC Profile version 4 support |
However, I didn't stop there. A few vendors, like Microsoft, have created their own proprietary extensions to the ICC Profile format. If Microsoft sees their custom extension, then they will use it instead of the version 2 or version 4 information. My egg picture has one of these extensions. I only tested it on Windows 8.1, but it should work on any Windows Vista or later system. The biggest issue is that Microsoft is inconsistent in their application. The same file will look different under Internet Explorer compared to Microsoft Photo Viewer.
Microsoft's proprietary ICC Profile extension support with Microsoft Photo Viewer on Windows 8.1 | |
Internet Explorer on Windows 8.1 displaying the PNG. If the image is saved as a JPEG, then Internet Explorer will display the same thing as Microsoft Photo Viewer. |
Without the Microsoft extension, Internet Explorer and Microsoft Photo Viewer will render the ICC Profile version 4 image.
On every operating system and application, a color management system (CMS) is used to apply the ICC Profile. There are a few open source CMS systems, but Microsoft, Apple, and Adobe each have their own systems. With each system comes compatibility issues and computational differences. For example, Apple's CMS on 10.5.8 mostly looks like the ICC version 2 image, except that Apple's blue egg looks a little more purplish. The purplish value may be due to my monitor's own color profile, so your picture may look different if you use a Mac.
Apple CMS |
When loading the picture into Adobe Photoshop CS5 for the Mac, I am prompted to "keep" the current profile, "convert" it to Adobe's color space, or "discard" the profile information. Each option creates very different results:
Adobe CS5 for Mac with "convert" profile | |
Adobe CS5 for Mac with "kept" profile. (For people who actually understand ICC Profiles: Adobe appears to use the absolute colorimetric intent instead of the perceptual intent.) | |
Adobe CS5 for Mac with "discard" profile (the result is the same as using no profile) |
But even minor changes made by the file format can result in significant differences. Here is the original egg picture saved twice: the left is a PNG and the right is a JPEG made from the PNG. Below each picture is what it looks like when colorized using the open source Little Color Management System (LCMS):
The problems with color transformations do not end with the ICC Profile; ICC Profiles are not the only way to store transformation information. For example, the PNG file format also supports its own color profile information in cHRM and gAMA tags. If a PNG contains both internal color information and an ICC Profile (like my egg example), then the application should use whatever it can. If it supports ICCv4, then use that information, otherwise use ICCv2, else use the PNG color information. And if all else fails, render the picture without any color corrections. Here's what the image looks like if your application only supports the PNG color information:
Internet Explorer on Windows 8.1 will apply PNG color information if it exists. This is what you see when you apply the PNG's embedded gamma and chromaticities. | |
Apple's Quicktime player for 10.5.8 only applies the embedded PNG gamma correction and not the embedded PNG chromaticities. |
Impacting Forensics
While my egg example is an extreme case, it demonstrates that different applications perform different color transformations. When doing digital photo forensics, the application that you use may alter the data that you are evaluating. This can dramatically impact test results.For example, applying an ICC Profile is explicitly a color transformation. In general, each pixel has the opportunity to shift a little in the color space. Since pixels can change values, any shifting results in a higher error level potential. Applying an ICC Profile or PNG color transform will alter the results from an error level analysis.
Color transformations can also impact signal-to-noise ratios, light direction, and even cloned-region detection.
When performing any type of digital forensics, you want to minimize the amount of transformation made when trying to render the picture. (At FotoForensics, we explicitly do not apply any color transformations. This way, we do not evaluate any artifacts that could be created during the color transform.)
Digesting Colors
It took me a couple of days, but this one egg picture contains a complex set of color profiles. Each profile makes the picture look different. The color that you see all depends on the tools that you use. With a little more time, I can easily add in at least 8 more transformation tests into this one picture. Currently, it contains an ICC Profile with both version 2 and version 4 information, plus a Microsoft-specific ICC Profile extension and a PNG color correction.The result is pretty clear: I have one picture (a PNG) that yields NINE different color sets! (Ten if you convert it to JPEG and use LCMS to render it.) The colors that you see are strictly dependent on the specific program that you use to view the image. Even something as minor as calibrating your video driver or patching your software could alter how the image is displayed.
You may be evaluating one file, but it can appear at least ten different ways!
All of this comes back to the digest. You and I are likely seeing two different sets of egg colors. So how do we know that we are looking at the same file? The answer is: check the digests.
My egg picture is a PNG that is 295x224 and uses 3 color channels. The file size is 33,492 bytes. It has an MD5 checksum of b513db97c45848fae40ef55c5e0fdfd5, and the SHA1 checksum is e53cae13e50d5556b6f2021000ff7fbe171561f8. If your file has all of these same attributes, then we are looking at the same file. Now we can start asking why the picture looks different if it is the exact same file.
And remember: it does not take much to alter the picture. If someone uploads this egg picture to Facebook, Twitter, or some other social media site, then the digest will probably change. And a change means that you will probably be seeing different colors.
Update 2014-04-21: Added notes for people who understand ICC Profiles and want more information about the cause of the image differences. Identified why Adobe looks so weird, and clarified why Microsoft displays two different images that are both technically correct.
Pale Moon (a Mozilla fork) shows the image like the Adobe CS5 for Mac somewhere between the 2 profiles (convert/kept) on my Windows 7 box (it defaults to perceptual).