Shadow/Highlight Detail in Photoshop -- Part I

Article and Photography by Ron Bigelow

www.ronbigelow.com

Photoshop CS or Photoshop CS2 Used in this Tutorial

Dealing with detail in the shadows and highlights is a basic part of photography. Photographs with blocked up shadows or blown out highlights are simply considered unacceptable. Yet, our equipment doesn't always want to cooperate in the endeavor to maintain this detail. Both color film and digital sensors have dynamic ranges far narrower than those of Mother Nature, which causes problems with detail in the shadows, highlights, or both. For digital, this is further exacerbated by the fact that the response of digital sensors to light is completely different than that of the human eye. In essence, the digital sensor places the most detail where the eye is the least sensitive and the least detail where the eye is the most sensitive.

This leaves the poor digital photographer (whether using a digital camera or scanning film) with the need to extract the shadow and highlight detail out of an image that was created by a device that was not necessarily optimized for that detail. Therefore, the means used to pull out that shadow and highlight detail become very important to the quality of the final image.

There are many ways to extract shadow and highlight detail. The problem is that they are not all equally effective at producing high quality results. Some of the techniques of enhancing shadow and highlight detail produce superior results compared to the results of other techniques. The problem is that not all photographers are aware of the options available. Even if they are, they may not necessarily know which methods are the best and why they produce superior results.

The purpose of this article is to review techniques for enhancing shadow and highlight detail, and the theory behind the techniques, in order to understand the strengths and weaknesses of each technique. In order to understand these strengths and weaknesses, five factors need to be considered.

The theory behind these factors will be presented first. Then, each of the shadow and highlight techniques will be presented individually and analyzed with respect to these factors.

Shadow and Highlight Levels

One of the biggest problems facing the enhancement of shadow detail is that there is not a lot of detail in the shadows to begin with. The problem is that most digital camera sensors are linear devices. What that means is that when the amount of light that reaches a sensor is doubled, the output of the sensor is doubled. Conversely, when the amount of light is cut in half, the output of the sensor is halved. This may not sound like a big deal; however, it causes major problems for the shadows as shown in Figure 1. The purpose of Figure 1 is to show how the tonal values of a file are distributed from the shadows to the highlights. For our purposes, we will assume that we have an image that was taken from a camera with a dynamic range of five stops and that the image was shot as a JPEG. The data for JPEG files starts off with 256 tones (before application of any tonal curves) and these values are spread from the shadows to the highlights.

Now, suppose that a pixel from this camera was exposed until it could accept no more light (this would be from an area in the scene being photographed that was very bright). In the case of our five stop dynamic range camera, the pixel would receive five stops of light. At this point, the pixel would be full. Pixel A in Figure 1 shows such a pixel at full capacity. This pixel would be able to render the full 256 tones (shades).

Now, suppose that pixel B in Figure 1 received its light from an area in the scene that was somewhat darker. In this example, the pixel would be given four stops of light (half as much light as pixel A). Since sensors are linear, and Pixel B got only half as much light, it would be able to render only half as many tones. Thus, the data from pixel B would be able to render only 128 tones. Since pixel A’s five stop exposure rendered 256 tones, and pixel B’s four stop exposure rendered only 128, the fifth stop of light was responsible for rendering the other 128 tones. In other words, the brightest stop of dynamic range (the fifth stop) used up half of all the available tones.

The procedure repeats itself with pixel C. The light is reduced by one more stop so that the pixel receives three stops of light. Since pixel C receives only half as much light as pixel B, it would be able to render only half as many tones. Accordingly, pixel C would render 64 tones. Since pixel B’s four stop exposure rendered 128 tones, and pixel C’s three stop exposure rendered only 64 tones, the fourth stop of light was responsible for rendering the other 64 tones. In other words, the second brightest stop of dynamic range (the fourth stop) used up one fourth of all the available tones.

Pixels D and E in Figure 1 show that, as we continue to work down the dynamic range, the camera is capable of rendering less and less tones. Eventually, one stop of light is reached. This last stop of light is capable of rendering only 16 tones. A summary of how these tones are distributed is shown in Table 1.

Figure 1: Tonal Values vs. Stops of Light for JPEG
Table 1: Distribution of Tonal Values for a Five Stop Dynamic Range Image (JPEG File) Prior to Application of Tonal Curves
Light Level
Tonal Values
Notes
5 Stops
128
Highlights
4 Stops
64
Three quarter tones
3 Stops
32
Mid tones
2 Stops
16
Quarter tones
1 Stop
16
Shadows

Here comes the bad news. Table 1 shows that when the camera only gets one stop of light, there are only sixteen tones. The problem is that this lowest stop is where the shadows reside. In other words, the shadows only get sixteen of the 256 tones.

Now, the story doesn't stop there. Before the camera is done with the image, there will be some manipulation of the data in the file (the application of a tonal curve, which will be briefly covered later). By the time the camera has finished with the JPEG file, the tonal distribution will look something like that displayed in Figure 2.

Table 2: One Possible Distribution of Tonal Values for a Five Stop Dynamic Range Image (JPEG File) After Application of a Tonal Curve
Light Level
Tonal Values
Notes
5 Stops
69
Highlights
4 Stops
50
Three quarter tones
3 Stops
37
Mid tones
2 Stops
27
Quarter tones
1 Stop
20
Shadows

While the final distribution of the tones has changed somewhat (the number of total tones has decreased to 203 and four more tones have been added to the shadows), the moral of the story pretty much stays the same: the shadows contain very few tones.

In short, the shadows of an image have a limited amount of tones and therefore a limited amount of detail. So the endeavor to enhance shadow detail starts off with the reality that there isn't a lot of detail in the shadows to begin with.

On the other hand, the highlights are a lot easier to handle. A look back at Table 2 shows that the highlights have almost three and a half times as many tones as the shadows. This means that there is much more detail in the highlights with which to work.

Noise

Another big issue with enhancing shadow detail is noise. All digital sensors and scanners have noise. At the image level, noise manifests itself by random detail that distracts from the real detail in the image. The real issue for digital images is the signal to noise ratio (SNR), which is the ratio of the signal the pixels get to the noise that is generated during the exposure. The higher the SNR, the better the image quality. The lower the SNR, the lower the image quality.

There are several types of noise in digital cameras and scanners. These various types of noise can be grouped into two main categories: constant noise and variable noise. Constant noise (primarily dark current and readout noise) is noise that remains relatively fixed; it does not vary with the amount of signal that the sensor receives. Variable noise varies with the amount of signal (the greater the signal that the sensor receives, the greater the variable noise). However, variable noise increases more slowly than the signal (the increase in variable noise is approximately equal to the square root of the increase in signal). These facts do not bode well for the shadows.

Table 3 shows a simulation of the SNR for shadows compared to mid-tones and highlights.

Table 3: Simulation of SNR
  Shadows Mid-Tones Highlights
Signal
50 Units
8,750 Units
35,000 Units
Constant Noise
20 Units
20 Units
20 Units
Variable Noise
7 Units
94 Units
187 Units
SNR
1.85
76.75
169.08

Notes

1. The signal is for a pixel with a full well capacity of 35,000 photons. The mid-tones are assumed to be two stops below full well capacity. The shadows are assumed to be slightly above the noise floor.

2. The constant noise is assumed to be 20 photons.

3. The variable noise is calculated as the square root of the signal.

Table 3 clearly shows that the shadows have a very poor signal to noise ratio. In fact, in this simulation, the highlights have a SNR over ninety times greater than the shadows. This is a major problem for the shadows due to the way many of the shadow enhancement techniques work. A large percentage of the shadow enhancement techniques work by amplifying the signal in the shadows in order to lighten the shadows and bring out the shadow detail. The only problem is that the techniques can not amplify the signal without amplifying the noise. When this noise was buried deep in the dark shadows, it was probably not very noticeable. However, once the shadows have been lightened, the noise will likely become more noticeable and may degrade the quality of the image.

 Again, things are much better when dealing with the highlights. The highlights have a relatively good SNR compared to the rest of the image. In addition, enhancing the highlights does not require the signal to be amplified as is the case with the shadows.

Tonal Stretching/Compression

Figure 2: Tonal Distribution before and after Editing (Aimed at Shadows)

Often, when images are edited, the tones get stretched and/or compressed. This is particularly true when enhancing shadow and highlight detail. Figure 2 shows a graph of tones before and after editing. The tones run from 0 (the darkest value) to 255 (the lightest value). The horizontal axis shows the original tones, and the vertical axis shows the tones after the image was edited. You can think of this in terms of a Curves graph. The blue line shows the data before editing. This line lies perfectly along the diagonal. The red line shows the data after editing. It can be seen that the tones have increased (e.g., the image has been lightened). In particular, the lower tones have increased significantly on a percentage basis. This chart shows the tonal effects of a typical edit that would be done to lighten shadows.

Figure 3 gives the numeric values of the first twenty-six tones (zero to twenty-five), taken from Figure 2, both before and after editing. In other words, Figure 3 shows what happens to the shadow values when the editing demonstrated in Figure 2 is performed. For example, the original value of zero stays at zero. However, the original value of ten has been increased to thirty-one by the editing, and the original value of twenty-five has been increased to fifty-six.

Figure 3: Shadow Tonal Values before and after Editing

The numbers in Figure 3 illustrate several important points about the effects of the editing that was done in Figure 2. First, the tones have, in fact, been lightened. For instance, the original value of fifteen has been more than doubled to forty. Second, the contrast has been increased. This can be seen by examining what happens when we go from one tone to the next. For example, with the original tones, going from a tone of one to a tone of two gave a one unit increase in tone. However, after editing, the tonal value of one became a value of seven, and the tonal value of two became a value of eleven. Now the difference between these two tones has become four units -- this results in an increase in contrast. Third, there are now more tonal levels in the shadows. Before editing there were twenty-six levels (zero to twenty-five) shown in Figure 3. After editing, there are fifty-seven levels (zero to fifty-six).

Great, the tones have been lightened, the contrast has been increased, and there are more tonal levels in the shadows. So much for the good news. Now for the bad! The first thing that needs to be noticed is that all those new tonal levels that were added to the shadows are empty (no pixels have those tonal values). In the previous paragraph, it was mentioned that after editing, moving from a tonal value of seven to the next tonal value of eleven was an increase of four units (which increased the contrast), but three of those tonal levels are empty (tonal levels eight, nine, and ten). There is now a much bigger gap between adjacent tones. In essence, the tones have been stretched apart creating gaps between the tones (the empty tonal levels may become populated if further editing is performed).

These tonal gaps are demonstrated in Figures 4 and 5. Figure 4 shows the histogram of an image before any image editing was performed. The histogram is smooth with no tonal gaps visible. In an effort to lighten the shadows, an edit was performed on the image. Figure 5 shows the same image after the editing was performed. The histogram has become very jagged and clearly displays a large number of tonal gaps (on the left and middle of the histogram; the spikes on the right will be explained in a bit).

Figure 4: Histogram before Image Editing
Figure 5: Histogram after Image Editing
Generally, in digital images, the tones blend together seamlessly. As a viewer's eyes move around the image, the tones seem to merge into each other. However tonal gaps caused by tonal stretching can cause major problems. If the image editing causes the tonal gaps to become large enough, they will become noticeable in the image. Instead of smooth tonal transitions, bands will appear where the image moves from one tone to the next. This is referred to as banding (i.e., posterization) and is often most conspicuous in areas of little detail such as featureless skies or bland shadows. An example of banding can be seen in Figure 6.
Figure 6: Banding
Figure 7: Highlight Tonal Values before and after Editing

So, stretching the tones can have some undesirable effects, but what about the other end of the graph shown in Figure 2: the highlights. Figure 7 gives the numeric values of the last twenty-six tones, taken from Figure 2, both before and after editing. In other words, Figure 7 shows what happens to the highlight values when the editing demonstrated in Figure 2 is performed. For example, the original value of 230 has been increased to 238 and the original value of 245 has been increased to 248.

An examination of Figure 7 reveals a disquieting problem. What used to be separate tones before editing have been compressed into the same tone after editing. The original tones 247 and 248 have both become 250 after editing. Similarly, the original tones 250 and 251 have both become 252 after editing. This compression results in a loss of image detail. This compression of the tones into fewer tonal spaces can be seen on the right side of the histogram in Figure 5 where some of the pixels can be seen "piling up" in the remaining tonal levels resulting in upward spikes in the histogram.

This compression of tonal values and the resultant degradation in image quality is due to something called quantization error. When image editing is performed, Photoshop runs the digital numbers (e.g., tones) through formulas to determine the new numbers. However, the new numbers have to be rounded off to the nearest digital number (e.g., a new tone of 157.43 would be rounded to 157). The information that is rounded off is thrown away forever. Thus, information is lost in the rounding process. This loss of information due to the rounding process is called quantization error.

Figure 8: Tonal Distribution before and after Editing (Aimed at Highlights)
The analysis in this section showed how one type of editing step to lighten the shadows, such as that shown in Figure 2, would stretch the shadows and compress the highlights. On the other hand, the issue of tonal stretching and compression is not limited only to edits aimed at the shadows. Figure 8 shows the tones, before and after editing, for an edit aimed at bringing out the detail in the highlights. This edit stretches the highlight tones and compresses the shadow tones (at one place on the tonal distribution, six shadow tones get compressed into one).

Clipping

Figure 9: Raw Converter Histogram before Editing
Figure 10: Raw Converter Histogram after Editing (Clipping has Occurred)

Some image editing can cause data to be clipped (i.e., lost).

Figure 9 and 10 show a case of data clipping that occurred during the editing of an image in a raw converter. In Figure 9, the right side of the histogram shows that some highlight data has already been clipped (the red spike). However, this is a specular highlight (e.g., it is supposed to be clipped). However, there is some clipping of the shadows. There is actually data in the shadows that can be reclaimed in the raw converter. Figure 10 shows the histogram after an edit was performed. The left side of the histogram shows that additional, usable shadow detail has been recovered. However, the right side shows that a large amount of the highlight detail has now been clipped. If this image is converted with these settings, that highlight detail will be lost in the converted image (raw files are not affected by editing, so the raw file will not lose the data). Thus, that clipped data will no longer be available in the converted image.

Data Lost to Tonal Curves

Figure 11: Image before Application of Tonal Curve

When a digital image is taken, a certain amount of information is captured. However, this information is not in a format that is directly usable. It needs to be processed in order for that information to become a usable image. However, there are different ways in which that information can be processed. One thing that is common to all these ways of processing the information is that tonal curves are used. Without the application of tonal curves, the data is so dark that it is barely recognizable. Figure 11 shows an image before the application of a tonal curve.

The problem is that many of the tonal curves that are used clip the data. This will result in lost detail in the shadows, the highlights, or both. Figures 12 and 13 show the effect of two different tonal curves on the highlight detail of the spray from a wave. In Figure 12, the highlight detail is mostly lost. By selecting a different tonal curve, the highlight detail was recovered as shown in Figure 13.

The problem is that the most commonly used shadow and highlight recovery techniques do not allow a photographer to access the data that was clipped due to the application of a tonal curve. However, some of the more advanced shadow and highlight recovery techniques do allow a photographer to access this data.

 

Figure 12: First Tonal Curve Causes Loss of Highlight Detail
Figure 13: Second Tonal Curve Reveals Highlight Detail

Articles

Shadow/Highlight Detail -- Part II