The Science and Pseudoscience of ICC Profiles and Scanning Colour Negative Film

Take 2 On Disentangling Colour Negative Film and ICC Profiles

Or, the longest titled blog post I have yet made.

Screen shot 2014-04-09 at 18.55.11

In my previous post Scanning Colour Negative Film Using ICC Profiles I discussed my experiences in using ICC profiles in conjunction with negative film scanning. This lead to a lot of questions and much more research.

In this episode we will embark on a journey into the depths of colour management and go where no colour negative film-shooting photographer has gone before. Well, I bet they have but I can’t find anything else like it on the internet…

 

NOTE: If your browser is not able to render files tagged with AdobeRGB profile correctly you will not see a lot of what is discussed below. All the example scans have been converted to AdobeRGB space.

Kinds of Profiles

ICC Profiles come in two basic flavours.

The first kind are composed of three RGB primaries located in device independent terms in colour space. Accompanying these three primary locations are either gamma curves or what are called “shaper” curves (curves with irregular shape).

The second kind are composed of lookup tables that translate input colour values into output values by means of a large “lookup table”. For values that fall in between the values in the table the output value is assigned based on interpolation from the two nearest values. These tables can allow extremely non-linear relationships between the input and output values and are of highest value in the realm of printing. For example, the input might be RGB and the output may be to a printer with ten different coloured ink pigments.

The behaviour of most photographic systems is largely describable by logarithmic “gamma” curves. The linear portion of the response curve of colour negative film, for instance, is described this way. Similarly, digital camera and scanner sensors are linear over most of their range and therefore describable with such a curve. In an ideal world we would need nothing more but ideal devices cost a lot of money.

Problems With PCS Lookup Type Profiles

In the realm of cheap scanners, cheap LCDs and cheap digital cameras the behaviour of the final system is often a significant deviation from the linear ideal. For this reason a whole industry has grown up around profiling screens and scanners and even cameras using calibrated targets. The most popular of these being the IT87 target.

Typically, one will calibrate one’s scanner and/or screen which will usually result in the creation of a large PCS lookup table type profile. The application of this profile will generally significantly enhance the ability of the device to reproduce accurately the same sort of subject material in the particular parameters in which the measurement was made. So, for example, you might scan an Ektachrome target and use that profile for Kodak slide films, you might scan and profile with a Velvia target and use that profile for Velvia scans, and so on. To get good results all your subsequent scans should be made with the same settings as were used to generate the profile. Furthermore, if anything on your later slides is brighter or darker than the extremes of the target they will be clipped. Not much use for colour negative film.

Profiling monitors is relatively straightforwards because the light source is fixed and the software can generate every level of brightness of which  the device is capable of. Scanners profiled with a lookup table (cLUT) profile will be described by that profile only in terms of that scanner exposure and that film. This is because if we assume that the reason the profile improves output is because it compensates for the non-linear behaviour of the device that device at a different exposure level will require a different compensation because the same colour patches on the target will now be further up or down the response curves of the scanner’s sensor.

Gamma Matrix Profiles

In a gamma plus matrix profile it is assumed that the response of the system being described is either linear (e.g. gamma 1.0) or is describable with an idealized logarithmic curve. It should be obvious that the behaviour of such a profile should be much more predictable than a lookup table profile. On the other hand, it will necessarily not be able to acheive the accuracy in reproducing a particular target on a particular device in a particular set of circumstances that a lookup table would be able to. And this gap will be larger the more non-linear or the more irregular the response curve of the device is (or in rough terms, the closer it is to a “consumer-level” device).

In exchange, however, the predictability of the basic gamma+matrix profile allows it to be much more universally applicable. Its description of the device in terms of the location in colour space of its R, G and B illuminants and their idealised response curves is just as applicable at any exposure level and (if we consider a scanner) with any film.

Examples

Screen shot 2014-04-09 at 18.54.30 Screen shot 2014-04-09 at 18.55.11

Above are the shipped profiles from the manufacturer for my Epson V700 flatbed scanner and my Nikon Coolscan 9000ED film scanner. The response described by the scanner is plotted as a 3D shape in colour space with the white cross at the top describing the brightest colours and the black cross at the bottom the darkest.

Screen shot 2014-04-09 at 19.35.07 Screen shot 2014-04-09 at 19.35.12

Above is the profile describing the AdobeRGB(1998) colour space. The profile locates the three RGB primaries and describes a 2.2 gamma for each. It also sets a white point (not shown in these two screenshots). Photoshop has a profile editor that shows the profile in a different way as seen below:

Screen shot 2014-04-09 at 19.42.01

AdobeRGB is a very popular colour space although something of an oddball amongst colour spaces in that it uses the D65 white point (most use D50) and in that it combines the NTSC and PAL standards in the location of its illuminants. It was an especially important colour space in the early days of colour management in that it was big enough to encompass most film-generated colours (by which is largely meant slide film) and it was small enough not to be “wasteful” in terms of 8bit colour files. I’m not, at the moment, going to go into why that consideration is historically relevant.

Why Profiles Matter For Colour Negative Film Part I

If we look back on the previous posts I have made on scanning colour negative film you can see that the first hurdle to overcome is capturing the film’s curves accurately in the scan. For this we favour a “RAW” linear scan because we don’t want the film’s curves combined with the curves of another colour space and we certainly don’t want them put through the sort of non-linear transform that applying a cLUT profile would perform to the data. But the file we output will have a colour profile applied nonetheless. This is because even in absence of a profile Photoshop (and many other graphics programs) will render the untagged file in reference to its “working profile” which is something you set in the application’s preferences. If you then assign this profile to your file you at least ensure that other software will render it with the same appearance but what if this is not the correct appearance?

I scanned the same piece of film on both of my scanners. In each case I took the raw linear scan and performed a default ColorPerfect conversion using the upper right square on the colour target strip for a white balance. My working space was set to AdobeRGB and I applied the AdobeRGB profile.

CP_QARoll151_NP37_cln_LS9k_AdbRGB

Nikon Coolscan 9000ED

CP_QARoll151_NP37_cln_V700T_AdbRGB_1-0_AdbRGBCnv

Epson Perfection V700

What do you see above? Well, there are a number of differences between the output of the two scanners. But perhaps the most important thing you might see (putting aside for the moment the question of correctness) is different colours. If you are at all familiar with the colour patches on a Kodak Q13 card as shown here you might particularly notice the almost luminous Red and Magenta squares in the version from the Nikon scanner.

Why are these two so different? Each scanner should have been capable of measuring the amount of Red Green and Blue in the film almost equally well. These values have then been treated the same by the same ColorPerfect software and given the same colour space (AdobeRGB). Why the difference? The answer becomes more clear if we profile the scanners. I used another picture taken on the same Kodak Portra 400 negative film of the same Kodak Q13 colour target. I scanned it on each scanner. I took the linear scan and inverted it at default settings in ColorPerfect only this time I chose to keep the output gamma linear as well. I used this scan to generate a Gamma+Matrix profile using the ArgyllCMS utilities. This profile contained information about the calculated location of the RGB primaries of each scanner in regards to a negative film target and also gamma and white point information. Since all that I’m interested in, here, is the location of the primaries I took these profiles into Photoshop and applied standard D50 white points and standard gammas (I did a 1.0 and a 2.2 gamma version of each). The results are below.

Screen shot 2014-04-08 at 10.13.20 AM

Nikon Coolscan 9000ED

Screen shot 2014-04-09 at 18.45.06

Epson Perfection V700

Now the reason is clearer. The colour patches are more shifted on the Coolscan 9000ED scan because the gamut of the scanner is less similar to the gamut of AdobeRGB colour space which is the colour space we assigned to both files. In fact, the colour gamut of the V700 is quite similar to that of AdobeRGB. We might therefore expect that if we assign these colour spaces to the appropriate scans the change in the Coolscan scan would be noticeable and the change in the V700 almost unnoticeable. Remember, these profiles differ from a standard colour space ONLY in the location of the primaries. In fact, this is what we will see:

CP_QARoll151_NP37_cln_LS9k_AdbRGB_LS9kHyb_AdbRGBCnv

Nikon Coolscan 9000ED

CP_QARoll151_NP37_cln_V700T_V700P400Q13RGBT_AdbRGBCnv

Epson Perfection V700

The two scans still differ. In particular, the Coolscan scan is less saturated. The V700 scan has much poorer control of densities with blocked up shadows and the plain background is varying in hue towards the edges of the frame more severely than the Coolscan. However, hopefully you can see that the colours are now more similar between the two. The Coolscan scan has lost its magenta cast and the magenta and red patches on the target have calmed down. The V700 scan is much less changed as expected since the V700 profile is much more similar to the previously assigned AdobeRGB profile than the Coolscan profile is.

Manufacturer Profiles

Both of these scanners ship with PCS profiles from the manufacturer. These profiles were generated at the manufacturer and will have profiled an ideal example of the model. So, on the one hand, these profiles were made using tools which are probably out of reach for the home user. On the other hand, they are profiling some prototype model of the device which may differ from the one on our desk in some way due to production differences. These profiles are “lookup table” type profiles but have been produced by the manufacturer to be as universally applicable as possible because they are intended to be used “blindly” by the manufacturer’s scanner software when performing a scan. When the manufacturer’s scanning software is used this profile is applied to the output before the scan is converted into the colour space chosen by the user in the software’s settings.

Epson provide two profiles. One for scanned film and one for reflective scans (since the scanner is also capable of scanning non-transparent material).

Nikon provides a suite of profiles dedicated to separate scanning tasks. These profiles are disabled for use by other software as delivered but it is simple to re-enable them as I discussed in the previous article Scanning Colour Negative Film Using ICC Profiles.

It is imperative that we not convert to these profiles before we invert the scan because we need ColorPerfect to operate on the raw scanner data (assigning a profile tells the computer how to render the RGB data but does not change it, converting to a profile alters the RGB data). After inversion, however, it is useful and interesting to see what the results are of applying the profiles. Here is our subject again but this time with the manufacturer’s profile applied after the inversion.

CP_QARoll151_NP37_cln_LS9k_AdbRGB_NkLS9k_R_AdbRGBCnv

Nikon Coolscan 9000ED

CP_QARoll151_NP37_cln_V700T_EpsV700_AdbRGBCnv

Epson Perfection V700

Notice that this is the most natural our model has looked for either scanner so far. The scans still differ but now we see a big improvement in the value separations in the darker areas of the V700 scan. Again, the V700 is the weaker scanner. Its sensor is the least linear and so it is getting the most “help” from the non-linear “lookup table” profile.

Conclusion

So, what is the conclusion based on these findings? I think there are two.

1. For the best experience evaluating colour of linear scans inverted using a deterministic system like the Photoshop DonzRGB action or the ColorPerfect software it is best to assign a profile that specifies the illuminant locations for your scanner. This is ideally determined using a pigment coloured target like the Kodak Q13 or ColorChecker target. However, I have also generated a profile using a photograph of an IT87 reflective target (using its accompanying data file) and the results were almost identical to the results I obtained using my Q13.

2. Once the scan has been appropriately inverted and processed to taste it is worth trying out what applying the manufacturer’s profile to the scan looks like. In my experience it is often beneficial and improves the appearance of the picture. Your eyes are the best judge.

Examples

The following pictures are scans from my Coolscan 9000ED. I present them first in AdobeRGB, secondly in the colour space of the scanner as revealed by profiling and lastly with the manufacturer’s profile assigned.

AdobeRGB:

CP_UKRoll021_NP05_AdbRGB CP_UKRoll021_NP08_AdbRGB CP_UKRoll021_NP27_AdbRGBScanner colourspace:

CP_UKRoll021_NP05_LS9kQ13P400_AdbRGBCnv CP_UKRoll021_NP08_LS9kQ13P400_AdbRGBCnv CP_UKRoll021_NP27_LS9kQ13P400_AdbRGBCnv

Nikon Profile:

CP_UKRoll021_NP05_NkLS9k_R_AdbRGBCnv CP_UKRoll021_NP08_NkLS9k_R_AdbRGBCnv CP_UKRoll021_NP27_NkLS9k_R_AdbRGBCnv

 

Advertisements

7 thoughts on “The Science and Pseudoscience of ICC Profiles and Scanning Colour Negative Film

  1. Hi Sam,

    What a great series of tutorials. Thanks for clarifying this last idea about icc profiles. May I ask one question: you state that assigning a scanner (Nikon or Epson) profile after conversion in ColorPerfect. When opening the untagged raw scan for the first time in Photoshop, you still need to assign your working color space (e.g. AdobeRGB) don’t you? Then you do the conversion and then you assign the scanner profile. Is this what you mean and does this double profiling still maintain the color integrity?

    Thanks for your help and all the work you have put into this series!

    Take care
    Sebastian

    • Sebastian,

      Assigning is non-destructive. You can assign the Nikon profile initially if you like. As long as you don’t convert to another profile you have not affected the RGB data in the file, only the way it will be rendered on the screen. As an alternative, you can assign a profile that merely assigns the primaries as I mention in the second article. In fact, this is what I recommend. To initially assign the simple matrix profile and then, when finished working with the file, try assigning the Nikon profile to see if you see an improvement. Until you convert to another profile you can assign as many profiles as you like without affecting the file’s data.

      Sam

  2. Great Sam. Thanks for your reply. This makes absolutely sense to me. So assigning AdobeRGB prior inversion and trying an Epson or Nikon profile after inversion should do the trick. Of course only if I like what I see.

    Thanks
    Sebastian

  3. I’m considering whether or not to order an IT87 reflective target from Wolf Faust, to photograph for profiling my scanner (Coolscan LS-50) as you mention doing in your conclusions. I was thinking, though: wouldn’t that method amount to profiling the film’s development as well as the scanner’s response? I have to wonder, then, how much the profile might be sabotaged by variations in film development from one batch to the next. Have you given any thought to this, or witnessed any such problems / absence of problems with your own profiling by photograph-a-target method?

    • So here’s the “devil’s bargain”: You are basically profiling a system (as you say). In other words, everything between the paper target and your positive digital image (the film, the development, the scanner, the scan settings and the negative to positive inversion workflow). Therefore, and this is what a lot of these articles detail, if you want a meaningful result you need to nail these all down. Same lab (or consistent development technique), same scanner, locked scanner settings, locked inversion workflow with the same settings each time.

      It is also for this reason that I ended up favouring the simpler curve+matrix profiles (in fact, I typically simplify these further in Photoshop so that I am left with only the primary colour locations for the scanner. I create a “gamma 2.2” version and a “gamma 1.0” version). Again, for reasons I stil do not fully understand, if you do have a Nikon scanner it is worth the small trouble to make modified copies of the Nikon “_N” version of the factory cLUT profile. Try applying this profile to your scans after inverting them. I continue to be impressed how often it does wonders to make the reds and greens particularly look more natural. If you like your picture with the Nikon profile applied you can “bake it in” by then converting to a standard profile. Conversion means that the file retains the same visual appearance but the values are re-written to give the same visual look with a different (for example AdobeRGB or sRGB) profile applied. You must not do any profile conversions until you are finished the process of inverting your negative scan back to a positive image since conversions actually rewrite the image data (applying profiles just affects how the image is displayed and doesn’t alter the image data — you can and should apply an appropriate profile before inversion to help you judge the result).

      Now, to the heart of your question. I have actually been quite pleased at the consistency of results I get. In the end, I didn’t see a need to have one profile per film (ColorPerfect plugin is already taking care of the differing gammas per colour channel when I choose the correct film profile at inversion). In my testing I found that the primary colour locations were fairly consistent from film to film and even target to target (IT87 vs Q13). What made the big difference was the scanner’s sensor/light source combination. For example, the profile from my Coolscan and my V700 are quite different.

  4. Hi. I’ve been following all your posts regarding profiling and scanning, very interesting, great stuff 🙂

    I do not own a Nikon nor Epson scanner but a Reflecta Pro Scan 7200. I’m quite happy with it for the budget. I’m also using VueScan to produce linear scans and ColorPerfect to invert them. Great software that I bought right away.

    I’m wondering if you could tell me how to find the “factory profiles” for the Reflecta. I don’t see any icc/icm for it anywhere in the system, maybe they hace a different/propietary file extension?

    Thanks for the help.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s