The Science and Pseudoscience of ICC Profiles and Scanning Colour Negative Film

Take 2 On Disentangling Colour Negative Film and ICC Profiles

Or, the longest titled blog post I have yet made.

In my previous post Scanning Colour Negative Film Using ICC Profiles I discussed my experiences in using ICC profiles in conjunction with negative film scanning. This lead to a lot of questions and much more research.

In this episode we will embark on a journey into the depths of colour management and go where no colour negative film-shooting photographer has gone before. Well, I bet they have but I can’t find anything else like it on the internet…

 

NOTE: If your browser is not able to render files tagged with AdobeRGB profile correctly you will not see a lot of what is discussed below. All the example scans have been converted to AdobeRGB space.

Kinds of Profiles

ICC Profiles come in two basic flavours.

The first kind are composed of three RGB primaries located in device independent terms in colour space. Accompanying these three primary locations are either gamma curves or what are called “shaper” curves (curves with irregular shape).

The second kind are composed of lookup tables that translate input colour values into output values by means of a large “lookup table”. For values that fall in between the values in the table the output value is assigned based on interpolation from the two nearest values. These tables can allow extremely non-linear relationships between the input and output values and are of highest value in the realm of printing. For example, the input might be RGB and the output may be to a printer with ten different coloured ink pigments.

The behaviour of most photographic systems is largely describable by logarithmic “gamma” curves. The linear portion of the response curve of colour negative film, for instance, is described this way. Similarly, digital camera and scanner sensors are linear over most of their range and therefore describable with such a curve. In an ideal world we would need nothing more but ideal devices cost a lot of money.

Problems With PCS Lookup Type Profiles

In the realm of cheap scanners, cheap LCDs and cheap digital cameras the behaviour of the final system is often a significant deviation from the linear ideal. For this reason a whole industry has grown up around profiling screens and scanners and even cameras using calibrated targets. The most popular of these being the IT87 target.

Typically, one will calibrate one’s scanner and/or screen which will usually result in the creation of a large PCS lookup table type profile. The application of this profile will generally significantly enhance the ability of the device to reproduce accurately the same sort of subject material in the particular parameters in which the measurement was made. So, for example, you might scan an Ektachrome target and use that profile for Kodak slide films, you might scan and profile with a Velvia target and use that profile for Velvia scans, and so on. To get good results all your subsequent scans should be made with the same settings as were used to generate the profile. Furthermore, if anything on your later slides is brighter or darker than the extremes of the target they will be clipped. Not much use for colour negative film.

Profiling monitors is relatively straightforwards because the light source is fixed and the software can generate every level of brightness of which  the device is capable of. Scanners profiled with a lookup table (cLUT) profile will be described by that profile only in terms of that scanner exposure and that film. This is because if we assume that the reason the profile improves output is because it compensates for the non-linear behaviour of the device that device at a different exposure level will require a different compensation because the same colour patches on the target will now be further up or down the response curves of the scanner’s sensor.

Gamma Matrix Profiles

In a gamma plus matrix profile it is assumed that the response of the system being described is either linear (e.g. gamma 1.0) or is describable with an idealized logarithmic curve. It should be obvious that the behaviour of such a profile should be much more predictable than a lookup table profile. On the other hand, it will necessarily not be able to acheive the accuracy in reproducing a particular target on a particular device in a particular set of circumstances that a lookup table would be able to. And this gap will be larger the more non-linear or the more irregular the response curve of the device is (or in rough terms, the closer it is to a “consumer-level” device).

In exchange, however, the predictability of the basic gamma+matrix profile allows it to be much more universally applicable. Its description of the device in terms of the location in colour space of its R, G and B illuminants and their idealised response curves is just as applicable at any exposure level and (if we consider a scanner) with any film.

Examples

Above are the shipped profiles from the manufacturer for my Epson V700 flatbed scanner and my Nikon Coolscan 9000ED film scanner. The response described by the scanner is plotted as a 3D shape in colour space with the white cross at the top describing the brightest colours and the black cross at the bottom the darkest.

Above is the profile describing the AdobeRGB(1998) colour space. The profile locates the three RGB primaries and describes a 2.2 gamma for each. It also sets a white point (not shown in these two screenshots). Photoshop has a profile editor that shows the profile in a different way as seen below:

AdobeRGB is a very popular colour space although something of an oddball amongst colour spaces in that it uses the D65 white point (most use D50) and in that it combines the NTSC and PAL standards in the location of its illuminants. It was an especially important colour space in the early days of colour management in that it was big enough to encompass most film-generated colours (by which is largely meant slide film) and it was small enough not to be “wasteful” in terms of 8bit colour files. I’m not, at the moment, going to go into why that consideration is historically relevant.

Why Profiles Matter For Colour Negative Film Part I

If we look back on the previous posts I have made on scanning colour negative film you can see that the first hurdle to overcome is capturing the film’s curves accurately in the scan. For this we favour a “RAW” linear scan because we don’t want the film’s curves combined with the curves of another colour space and we certainly don’t want them put through the sort of non-linear transform that applying a cLUT profile would perform to the data. But the file we output will have a colour profile applied nonetheless. This is because even in absence of a profile Photoshop (and many other graphics programs) will render the untagged file in reference to its “working profile” which is something you set in the application’s preferences. If you then assign this profile to your file you at least ensure that other software will render it with the same appearance but what if this is not the correct appearance?

I scanned the same piece of film on both of my scanners. In each case I took the raw linear scan and performed a default ColorPerfect conversion using the upper right square on the colour target strip for a white balance. My working space was set to AdobeRGB and I applied the AdobeRGB profile.

Nikon Coolscan 9000ED

Epson Perfection V700

What do you see above? Well, there are a number of differences between the output of the two scanners. But perhaps the most important thing you might see (putting aside for the moment the question of correctness) is different colours. If you are at all familiar with the colour patches on a Kodak Q13 card as shown here you might particularly notice the almost luminous Red and Magenta squares in the version from the Nikon scanner.

Why are these two so different? Each scanner should have been capable of measuring the amount of Red Green and Blue in the film almost equally well. These values have then been treated the same by the same ColorPerfect software and given the same colour space (AdobeRGB). Why the difference? The answer becomes more clear if we profile the scanners. I used another picture taken on the same Kodak Portra 400 negative film of the same Kodak Q13 colour target. I scanned it on each scanner. I took the linear scan and inverted it at default settings in ColorPerfect only this time I chose to keep the output gamma linear as well. I used this scan to generate a Gamma+Matrix profile using the ArgyllCMS utilities. This profile contained information about the calculated location of the RGB primaries of each scanner in regards to a negative film target and also gamma and white point information. Since all that I’m interested in, here, is the location of the primaries I took these profiles into Photoshop and applied standard D50 white points and standard gammas (I did a 1.0 and a 2.2 gamma version of each). The results are below.

Nikon Coolscan 9000ED

Epson Perfection V700

Now the reason is clearer. The colour patches are more shifted on the Coolscan 9000ED scan because the gamut of the scanner is less similar to the gamut of AdobeRGB colour space which is the colour space we assigned to both files. In fact, the colour gamut of the V700 is quite similar to that of AdobeRGB. We might therefore expect that if we assign these colour spaces to the appropriate scans the change in the Coolscan scan would be noticeable and the change in the V700 almost unnoticeable. Remember, these profiles differ from a standard colour space ONLY in the location of the primaries. In fact, this is what we will see:

Nikon Coolscan 9000ED

Epson Perfection V700

The two scans still differ. In particular, the Coolscan scan is less saturated. The V700 scan has much poorer control of densities with blocked up shadows and the plain background is varying in hue towards the edges of the frame more severely than the Coolscan. However, hopefully you can see that the colours are now more similar between the two. The Coolscan scan has lost its magenta cast and the magenta and red patches on the target have calmed down. The V700 scan is much less changed as expected since the V700 profile is much more similar to the previously assigned AdobeRGB profile than the Coolscan profile is.

Manufacturer Profiles

Both of these scanners ship with PCS profiles from the manufacturer. These profiles were generated at the manufacturer and will have profiled an ideal example of the model. So, on the one hand, these profiles were made using tools which are probably out of reach for the home user. On the other hand, they are profiling some prototype model of the device which may differ from the one on our desk in some way due to production differences. These profiles are “lookup table” type profiles but have been produced by the manufacturer to be as universally applicable as possible because they are intended to be used “blindly” by the manufacturer’s scanner software when performing a scan. When the manufacturer’s scanning software is used this profile is applied to the output before the scan is converted into the colour space chosen by the user in the software’s settings.

Epson provide two profiles. One for scanned film and one for reflective scans (since the scanner is also capable of scanning non-transparent material).

Nikon provides a suite of profiles dedicated to separate scanning tasks. These profiles are disabled for use by other software as delivered but it is simple to re-enable them as I discussed in the previous article Scanning Colour Negative Film Using ICC Profiles.

It is imperative that we not convert to these profiles before we invert the scan because we need ColorPerfect to operate on the raw scanner data (assigning a profile tells the computer how to render the RGB data but does not change it, converting to a profile alters the RGB data). After inversion, however, it is useful and interesting to see what the results are of applying the profiles. Here is our subject again but this time with the manufacturer’s profile applied after the inversion.

Nikon Coolscan 9000ED

Epson Perfection V700

Notice that this is the most natural our model has looked for either scanner so far. The scans still differ but now we see a big improvement in the value separations in the darker areas of the V700 scan. Again, the V700 is the weaker scanner. Its sensor is the least linear and so it is getting the most “help” from the non-linear “lookup table” profile.

Conclusion

So, what is the conclusion based on these findings? I think there are two.

1. For the best experience evaluating colour of linear scans inverted using a deterministic system like the Photoshop DonzRGB action or the ColorPerfect software it is best to assign a profile that specifies the illuminant locations for your scanner. This is ideally determined using a pigment coloured target like the Kodak Q13 or ColorChecker target. However, I have also generated a profile using a photograph of an IT87 reflective target (using its accompanying data file) and the results were almost identical to the results I obtained using my Q13.

2. Once the scan has been appropriately inverted and processed to taste it is worth trying out what applying the manufacturer’s profile to the scan looks like. In my experience it is often beneficial and improves the appearance of the picture. Your eyes are the best judge.

Examples

The following pictures are scans from my Coolscan 9000ED. I present them first in AdobeRGB, secondly in the colour space of the scanner as revealed by profiling and lastly with the manufacturer’s profile assigned.

AdobeRGB:

Scanner colourspace:

Nikon Profile: