Raw Processing Discussion

This area is for the discussion of what's new, what's on your mind, and general photographic topics. A place to meet, make comments on this site, and get the latest community news.

Moderators: rjlittlefield, ChrisR, Chris S., Pau

ray_parkhurst
Posts: 3416
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Raw Processing Discussion

Post by ray_parkhurst »

I've been perplexed for quite a while regarding some of the claims made by folks who advocate raw processing, and am wondering what I am missing. Hopefully the collective brain trust here can help me understand.

Brief background...I have been shooting with the A7Rm4 for a while now, and have tried both raw and jpg workflows, and found that I can get similar results from either through judicious choices of in-camera jpg processing. I am worried that I am doing something wrong. It wouldn't be the first time!

Now, stacking does improve with more bit depth, but that's a different subject. What I am most concerned about is the almost universal assertion in the various online refs that blown highlight details are recoverable with raw, but lost with jpg. Some folks claim that highlight details can be recovered even if the entire image is white! So, is it true? And if so, how is it done?

I understand the concept of dynamic range, and how raw processing can be applied to recover shadow details. In fact this is the area I concentrate on with in-camera adjustments for jpg output that allows reasonable equivalency between jpgs and raw->jpg outputs. It is the claims of highlight detail recovery that makes no sense to me. From what I understand, once a highlight is blown, it is not recoverable.

aphi
Posts: 12
Joined: Wed Mar 15, 2023 6:09 am

Re: Raw Processing Discussion

Post by aphi »

Blown highlights are not recoverable, since the pixels are saturated.

The in-camera tone processing will generally compress and move things around such that e.g. a 50% signal (one stop down from saturation/blown out) becomes white. In that case you would find recoverable highlight detail in the raw file, compared to the jpg. Shoot RAW+JPEG and compare the raw with a flat/neutral tone curve to the JPEG.

ray_parkhurst
Posts: 3416
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Re: Raw Processing Discussion

Post by ray_parkhurst »

aphi wrote:
Wed Mar 22, 2023 5:09 pm
Blown highlights are not recoverable, since the pixels are saturated.

The in-camera tone processing will generally compress and move things around such that e.g. a 50% signal (one stop down from saturation/blown out) becomes white. In that case you would find recoverable highlight detail in the raw file, compared to the jpg. Shoot RAW+JPEG and compare the raw with a flat/neutral tone curve to the JPEG.
Hi aphi, and welcome to the forum. I'm glad to have you reply to my post!

Not sure what you mean about in-camera tone processing. It sounds like you're saying cameras have an "extra bit" at the top, but I don't think this is true. Or looking at it the other way, one bit would be "lost" if it was displayed as white when it is not really so. Please explain your thoughts further.

Prior to starting this thread, I did something similar to what you suggested. I shot an over-exposed image raw+jpg, and then processed both to recover highlights as much as I was able. Here is the original image (jpg) followed by recovered files. Others may be better at this but after doing this I was comfortable starting this thread:

No processing:
.
DSC00010(1)_2_2.JPG
.
processed raw file:
.
RAW_2.JPG
.
processed jpg file:
.
JPG_2.JPG
There are some subtle differences between the raw and jpg processed files, but basically most of the over-exposed pixels remain over-exposed in both. If anything I was able to recover more pixels from the jpg than I could from the raw!

Lou Jost
Posts: 5945
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:

Re: Raw Processing Discussion

Post by Lou Jost »

In my experience there is more dynamic range in the RAW file than can be displayed in the jpg. With the RAW file you can choose which parts of the tone curve you want to "print". But I can't say for sure whether you can duplicate this with some unusual custom jpg settings.
There is also more freedom to make changes in color balance in the RAW editor versus the jpg.

ray_parkhurst
Posts: 3416
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Re: Raw Processing Discussion

Post by ray_parkhurst »

Lou Jost wrote:
Wed Mar 22, 2023 7:38 pm
In my experience there is more dynamic range in the RAW file than can be displayed in the jpg. With the RAW file you can choose which parts of the tone curve you want to "print". But I can't say for sure whether you can duplicate this with some unusual custom jpg settings.
There is also more freedom to make changes in color balance in the RAW editor versus the jpg.
Yes, no doubt the 14+ bit raw file has more dynamic range than the 8-bit jpg. The jpg editor I use can white balance to a reference similar to raw, so that's not a limitation. I often shoot toned coins on a reference background so I can ensure the final (jpg) output is properly white balanced. I find the camera often introduces errors that need correcting in post.

Perhaps what I'm experiencing is that Sony's jpg engine is really good? I've noticed that I see no false colors like I did with the Canon, even at smaller apertures. That was another unexpected benefit to the A7Rm4.

Anyway, is there any contrary evidence that if an image is over-exposed, and that even if the image is raw, the blown highlights cannot be recovered? ie does raw allow recovery of blown highlights? It seems a silly question, but I am not exaggerating that the vast majority of folks out there believe (adamantly) that you can recover blown highlights with raw processing.

aphi
Posts: 12
Joined: Wed Mar 15, 2023 6:09 am

Re: Raw Processing Discussion

Post by aphi »

ray_parkhurst wrote:
Wed Mar 22, 2023 5:23 pm
aphi wrote:
Wed Mar 22, 2023 5:09 pm
Blown highlights are not recoverable, since the pixels are saturated.

The in-camera tone processing will generally compress and move things around such that e.g. a 50% signal (one stop down from saturation/blown out) becomes white. In that case you would find recoverable highlight detail in the raw file, compared to the jpg. Shoot RAW+JPEG and compare the raw with a flat/neutral tone curve to the JPEG.
Hi aphi, and welcome to the forum. I'm glad to have you reply to my post!

Not sure what you mean about in-camera tone processing. It sounds like you're saying cameras have an "extra bit" at the top, but I don't think this is true. Or looking at it the other way, one bit would be "lost" if it was displayed as white when it is not really so. Please explain your thoughts further.

Prior to starting this thread, I did something similar to what you suggested. I shot an over-exposed image raw+jpg, and then processed both to recover highlights as much as I was able. Here is the original image (jpg) followed by recovered files. Others may be better at this but after doing this I was comfortable starting this thread:
Apologies, I worded that poorly - I didn't mean the tone mapping tries to chop the brightest stop off on principle, but that it might end up doing that depending on the image. And if it does so, then you'll end up with more recoverable highlights in the raw.

Here's what Nikon's JPEG engine does with a particular scene:
1 jpg.jpg
This is the automatically matched tone curve in RT, which is not perfect but close enough. About half a stop from the raw is just hard-clipped:
1 tonecurve.png
The result RT generates with this tone curve is relatively close to the OOC JPEG:
1 rt matched.jpg
As expected, significant detail may be recovered that simply is not there in the JPEG:
1 linear.jpg
Another scene gives a totally different tone curve:
2 other tone curve.png
And indeed there is no real additional detail to be had in the raw for this one.

Lou Jost
Posts: 5945
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:

Re: Raw Processing Discussion

Post by Lou Jost »

"Anyway, is there any contrary evidence that if an image is over-exposed, and that even if the image is raw, the blown highlights cannot be recovered? ie does raw allow recovery of blown highlights?"

Of course there are limits Ray. I have many such examples. But in general, I can recover more highlights or shadows from RAW imnages than from jpgs.

ray_parkhurst
Posts: 3416
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Re: Raw Processing Discussion

Post by ray_parkhurst »

Lou Jost wrote:
Thu Mar 23, 2023 5:34 am
Of course there are limits Ray. I have many such examples. But in general, I can recover more highlights...
As long as the pixels were not overexposed, then you can always remap them to a lower level. You can even map the overexposed ones to a lower level. But what I'm saying is you can't map anything brighter down because the data has been clipped. I think these are the limits you refer to, correct? In reality though, the most-exposed pixels in the raw file should be relatively positioned vs the most-exposed pixels in the jpg, unless the jpg engine did something weird to them during compression or recovery (eg aphi's example). So perhaps the reason highlight recovery in raw is generally more effective than in jpg is that the tools offered for raw processing are more geared toward that end? If you look at my example above, there is actually more highlight detail recovered in the jpg than in the raw, though as I said I am not expert at raw image highlight recovery. I just used simple histogram tools in both (identical processing, close as I could) to map the highlights to lower levels.

Beatsy
Posts: 2105
Joined: Fri Jul 05, 2013 3:10 am
Location: Malvern, UK

Re: Raw Processing Discussion

Post by Beatsy »

Blown highlights and blocked shadows are blown and blocked full stop - no matter how many bits of dynamic range you have. By definition, they are not recoverable. So good exposure when taking the shot is still paramount. Expose to the right but don't clip highlights etc.

There is one small caveat in that histograms shown on the back of the camera use 8-bit (jpeg) source data. So there actually *is* a little bit of leeway at the bright end if you're shooting RAW. That is, something shown as (only just) "blown" on the histogram (or displayed with potentially over-exposed areas of the image blinking) will have real recoverable detail available when processing the RAW file in post. Not much, but sometimes enough to save the pic, where the same pic shot in jpeg would be unrecoverable for sure.

My main bias towards RAW revolves around the leeway to stretch contrast more before banding kicks in. Most useful in microscopy (low contrast subjects) and for putting more life into flat skies in landscapes etc. Jpeg (8-bit) quickly breaks smooth gradients into visible bands and posterises (or quantises) colours if stretched too far. Higher bit-depth RAW has much more tolerance for heavy contrast stretching while still keeping the output smooth and natural-looking.

Post Reply Previous topicNext topic