Questions concerning the workflow when stacking

A forum to ask questions, post setups, and generally discuss anything having to do with photomacrography and photomicroscopy.

Moderators: rjlittlefield, ChrisR, Chris S., Pau

Planapo
Posts: 1581
Joined: Tue Nov 07, 2006 2:33 am
Location: Germany, in the United States of Europe

Questions concerning the workflow when stacking

Post by Planapo »

What sort of files do you use for stacking? JPEGs only? I haven´t tried any other as I figure my computer wouldn´t have enough memory anyway, but would it be possible or recommendable to stack any other type of files than JPEGs?

And then concerning the workflow to obtain maximum quality: Should one just convert the RAWs to JPEGs, and then do the stacking, and do all the post-processing afterwards to the one JPEG that results?
Or would it be better to do (some) post-processing to the single frames (uniform treatment done to all of the frames in a batch process in order to not to make it any harder for the stacking software, I´d guess) before doing the stacking? For example some first USM to make it easier for the stacking software to locate the parts that are in focus?

Do you have any experience of what would be the best approach?

Thanks in advance for your input, guys.

--Betty

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Betty,

For almost all purposes, JPEGs straight from the camera will be fine.

In theory, the very best quality would come from shooting raw and processing with a 16-bits-per-color workflow.

Probably the most important advantage in shooting raw is great flexibility in correcting color after the shoot. 16-bit workflow also offers slightly lower noise and significantly better tolerance for underexposure and very dark shadows.

However, these are not large advantages for stacking because your stacks will be shot under completely controlled conditions. You can custom white balance in the camera, shoot at lowest ISO setting for least noise, and provide fill light to avoid dark shadows.

In addition, shooting raw may be slower, possibly a lot slower, depending on what camera, memory cards, and computer interface you have.

As far as I know, full 16-bit workflow is only available with Helicon Focus. TuFuse supports 16-bit, but if you need registration, you'll probably be going through CombineZ*, and CombineZ* only does 8-bit.

For what it's worth, I think that all the stacks I've ever posted for show were shot as JPEGs. Many of them were even at the "low quality" setting. That's because with my ancient Canon 300D, using "high quality" slows down shooting quite a bit due to larger files, and when I tested, I found that for many setups, I couldn't tell any difference in the output. I don't recommend using low quality because sometimes it does matter, and predicting when takes a lot of experience. But it does illustrate that if you get good lighting, the file format is not a big concern.

As for processing after the shooting, before the stacking, I don't do any routinely. There are postings in the forum archives discussing how some cases of "stacking mush" can be avoided by pre-sharpening. However, there are more recent postings discussing that those same cases can now be handled without pre-sharpening through the use of either HF's "Method B" or the pyramid algorithms of TuFuse and CombineZP. TuFuse still has some problems with contrast buildup blowing out some pixels, in which case you might want to pre-process to reduce the contrast of TuFuse input. But other than that, I can't think what might be helpful.

Bottom line, by far the most important thing is to get good lighting. Do that and JPEGs will work fine. Mess it up, and raw/16-bit workflow might rescue the stack, but don't count on it.

I'll be very interested to hear if other people's experiences are different from mine.

--Rik

augusthouse
Posts: 1195
Joined: Sat Sep 16, 2006 1:39 am
Location: New South Wales Australia

Post by augusthouse »

Betty,

Could you remind me which camera you are currently using?

I would recommend that you become familiar working with RAW files as a matter of practice.

Converting the processed RAW files and exporting as .tif files is the workflow I employ. I am currently on the focus-stacking learning curve myself and have been seriously considering the not often mentioned benefits of manual compositing as opposed to stacking. (but that is a discussion that would lead us away from your original question).

I have read elsewhere in the forum where another member's camera allows the taking of a RAW file and a low-resolution jpeg at the same time. Working with the low-resolution jpeg files provides an indication if a series of slices will provide a worthwhile 'stack'.

Focus-stacking software is a component of the workflow; but not the primary component. It is a tool that requires good quality photographic material to work with as it attempts to fulfil its role.

Batch processing of multiple RAW files can be accomplished in such programs as Adobe Lightroom and PhaseOne Capture one (v4.1), among others.

Additional expense for additional software - yes; however, if you own a license for an earlier version of PhaseOne Capture One you are eligible to upgrade to the latest version for free. I purchased my license for about USD $24.00 and took advantage of their unique licensing policy. I recommend downloading the tutorial movie (132MB) as this will provide a good indication of the RAW processing and Batch Processing capabilities of the program. I mention this program in particular because of the entry-level cost as opposed to the more expensive Adobe Lightroom.

Link to PhaseOne Download page: (Version 4.1)
http://www.phaseone.com/Content/Downloads/CO4.aspx

PhaseOne Homepage
http://www.phaseone.com/


Craig
To use a classic quote from 'Antz' - "I almost know exactly what I'm doing!"

NikonUser
Posts: 2693
Joined: Thu Sep 04, 2008 2:03 am
Location: southern New Brunswick, Canada

Post by NikonUser »

My camera will shoot in JPEG, TIFF, and RAW.
At the best quality setting, JPEG gives 5.9 MB image, TIFF a 36.5 MB image, and RAW a 20.7 MB image.
I shoot with the intention that I might want to make a 13x19" print.
JPEG seems like too much compression, TIFF files take too long for the camera to process.
So I shoot RAW.

The full lateral image of this fly HERE as an insert:
60 RAW exposures, each at about 20MB;
stacked with Helicon Focus 4.1,
took 12 minutes to process (time to get a coffee, or take another set of photos);
gave me a 70MB TIFF image (saved).

Levels adjust, Curves adjust, USM sharpening in Photoshop CS2 (saved);
reduced to 800 pixels wide (obviously much less for this insert) and less than 200KB JPEG in Photoshop CS2 (posted).

This works OK for me and I am not sure that processing the RAW files to another format before stacking with HF would be of any benefit - but it is nice to have the option.
All my stacked images on this site have followed the same sequence. Possibly taking the original 60 exposures as JPEG's would work OK, I'm sure it would be much quicker processing in HF

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

NikonUser wrote:At the best quality setting, JPEG gives 5.9 MB image, TIFF a 36.5 MB image, and RAW a 20.7 MB image.
... JPEG seems like too much compression
I am curious what type of image these numbers reflect.

If I read correctly, your camera is 12.8 Mpixels. The RAW number makes perfect sense as probably 14 bits per pixel with a tiny bit of non-lossy compression. The TIFF number also makes perfect sense as 8 bits RGB (3 bytes) per pixel with no compression at all. (Your 70 MB TIFF file is presumably 16 bits per color.) The JPEG would also be 8 bits RGB, and therefore represents a compression factor of 6.5X, compared against the 8-bits TIFF.

I agree that 6.5X would be a large compression factor if the entire image is full of fine detail. But it's a perfectly reasonable number if this is a stack slice where most of the frame is out of focus. OOF stuff compresses really well.

A couple of examples from my Canon 300D may illustrate.

I just now shot a piece of sandpaper straight on so that it's all sharp. For that I got 9,165 KB RAW, 5,136 KB high quality JPEG, and 3,066 KB low quality JPEG. (Numbers reported by Windows Explorer.) But I also shot a typical slice of a moth head, and for that I got 5,882 KB RAW, 1,528 KB high quality JPEG, and 698 KB low quality JPEG.

Compared against a 18.8 MB TIFF (6.3 Mpixels), the high quality JPEGs are compressed by 3.2X for the detailed sandpaper frame and 12.3X for the stack frame. The corresponding low quality JPEGs are 6.1X and 27X (!).

With the RAW converter set for no sharpening and no noise reduction, and observing with the layer-and-click-between technique, I can't see any significant difference in resolution and I can only find compression artifacts by differencing the coarse and fine JPEGs and adding a strong levels adjustment.

But in doing this test, I did see one effect that I have not noticed before, pointing out a possible advantage of raw that's also not been mentioned in this thread.

When I initially ran the raw conversion, the eye of the moth ended up with significantly less color streaking than showed in the JPEGs. It turned out that the "Color Noise Reduction" parameter was set fairly high in the raw converter. Cranking that parameter back down to zero and re-converting made the raw about the same as the JPEG.

I initially assumed that the color streaking was due to real interference effects in the moth's eye. Now I have to also consider the possibility that it's a Moire effect that crept in during de-Bayering. In that case, being able to tune the raw conversion could have a noticeable effect on output quality, that could not be achieved by JPEG.

This is one example of a general advantage of shooting raw, namely that separate raw converters may be able to do a better job of conversion than the camera's algorithm can.

--Rik

NikonUser
Posts: 2693
Joined: Thu Sep 04, 2008 2:03 am
Location: southern New Brunswick, Canada

Post by NikonUser »

The file size data I gave earlier came from a 3rd party text. In the official Nikon D2Xs guide:
Image sensor 12.84 million pixels
Effective pixels 12.4 million
JPEG 5.9 MB
TIFF 36.5 MB
RAW 19.2 MB (not 20.7 as reported in the 3rd party book)

I'm not technically inclined but the only advantage I see in JPEG's is the quicker processing time. I run 2 computers so while HF is processing the RAW files on one I can browse photomacrography et al. on compter 2; so HF processing time is not that critical for me.

Harold Gough
Posts: 5786
Joined: Sun Mar 09, 2008 2:17 am
Location: Reading, Berkshire, England

Post by Harold Gough »

I don't know whether this is totally relevant here but alamy.com, who set just about the highest file standards for professional images for submission, require RGB JPEGs of quality of at least Photoshop level 10, with a minimum size of 48MB and uncompressed. "This means you should make your JPEG file from an 8 bit TIFF file that is at least 48MB"

That doesn't mean too much to me but it might be relevant to the issue of quality.

Harold
My images are a medium for sharing some of my experiences: they are not me.

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Harold Gough wrote:require RGB JPEGs of quality of at least Photoshop level 10, with a minimum size of 48MB and uncompressed. "This means you should make your JPEG file from an 8 bit TIFF file that is at least 48MB"
You might be amused by the following experiment I just ran.

I started with a 17-frame stitched panorama landscape -- gobs of detail everywhere you look -- rendered at resolution 8192 x 3481 pixels. That's 28.5 Mpixels. The TIFF file size is 93,188,352 bytes at 8 bits/color. I pulled it into Photoshop and saved it back out as JPEG level 10, exactly as specified by alamy.com. The resulting file is 13,137,952 bytes.

Divide it out, that's a JPEG compression factor of 7.09X, using alamy.com's recommended procedure.

What alamy.com is really saying amounts to "we want more pixels than a consumer camera will give, and of course we never want to see a JPEG compression artifact". More crudely, "no casual shooters need apply".

Regarding Betty's original question about workflow for stacking, I'll summarize like this:

You never lose quality by shooting raw, and you may gain a little. You will lose some time and space, which may or may not be bothersome depending on your circumstance. Test with your own equipment to see what matters to you. In either case, take the time to tune up your system to have good lighting with proper exposure and color balance. This matters more to JPEGs than it does to raw.

I will emphasize again that stacking has a different balance of costs & benefits from most other shooting . I shoot stacks as JPEG, but I shoot almost everything else as raw if the camera provides that.

--Rik

Ref: a slightly different version of the stitched landscape can be found here: http://janrik.net/PanoPostings/LupinesAndPineTrees/ . The original 17 frames were shot raw. I did run a comparison against JPEG-in-camera, shot at the same time and place. Raw provided significantly better shadow quality for that high contrast scene.

Planapo
Posts: 1581
Joined: Tue Nov 07, 2006 2:33 am
Location: Germany, in the United States of Europe

Post by Planapo »

Thank you for all the great info you´ve provided. :D
I´m a bit less lost in post-processing now, and some further steps up the learning-curve!

Craig, I currently use a Canon 400D and shoot exclusively Raw, neutral (with no further processing done by the camera itself).
As raw converter I use the Canon software that came with the camera, or now the latest version of Adobe Camera Raw (special thanks to DaveW!:D ) as plug-in for PSE 5.0.
Now I´ve just discovered that when for example trying to clone-out dust specks etc., PSE tells me that I´ve to convert the file from 16-bit back to 8-bit. Can´t judge if this will eventually loose me any quality, maybe conceivably in a large print only. But as I have the raw files I could later edit them again in 16-bit if necessary.

Rik, that slice of pine forest with the lupine undergrowth is beautiful! Makes me imagine insect humming, woodpecker sounds and the balsamic odour of resin terpenes!

Thanks again
--Betty

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Planapo wrote:Rik, that slice of pine forest with the lupine undergrowth is beautiful! Makes me imagine insect humming, woodpecker sounds and the balsamic odour of resin terpenes!
Yes, that was a beautiful time for that place. It was fairly late in the year for that elevation. All around, most of the lupines were gone by. Only in that one spot, very shaded on a north slope, were there still lots of flowers. I have gone back to that spot several times since then, and found it mundane every time -- either too early or too late. Apparently that was the perfect day...

Thanks for the feedback! :D

--Rik

lauriek
Posts: 2402
Joined: Sun Nov 25, 2007 6:57 am
Location: South East UK
Contact:

Post by lauriek »

A few cameras ago I tested RAW vs JPEG and started shooting RAW immediately, and never looked back.

Whether you should do this or not really depends on how good the JPEG 'engine' is in your camera. I suspect they are a lot better now than on my old E20!

As I understand it, although JPEGs are compressed on disk, when you load them up, they have to be uncompressed to be displayed, so although a bunch of TIFFs take up more space on disk, the time to process a stack of them shouldn't be any/much different for JPEGs at the same resolution. (RAW files do contain more info than either TIFFs or JPEGs though so stacking RAW files should take longer).

My normal workflow is to shoot RAW, process those into (for now 8bit) TIFFs, I stack those, then produce a TIFF stack output. From there I obviously convert down to a JPEG with minimal compression for web posting.

Would prefer to use 16 bit TIFFs but the disk space requirements would balloon and I already have problems in this area (my PC has 2.25tb of disk space and most of it is currently used up!)

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

lauriek wrote:RAW files do contain more info than either TIFFs or JPEGs though so stacking RAW files should take longer.
The programs don't know how to stack raw files directly. They just convert raw to uncompressed RGB as an internal processing step. It's the same idea as uncompressing JPEG, except raw conversion takes longer.

So in terms of overall time, it won't matter much whether you convert explicitly before stacking, or let the stacking program do the conversion for you.

This assumes that you only run the stack once, or allow the stacking program to cache the converted files for you. If you run the stack multiple times, and do NOT allow caching, then the raw files will get converted multiple times. That will definitely take longer.

--Rik

mgoodm3
Posts: 273
Joined: Mon Sep 08, 2008 8:50 am
Location: Southern OR

Post by mgoodm3 »

I shoot everything RAW and convert to TIF for stacking. Helicon focus does allow for RAW, but you had better get everything right in the RAW capture because Helicon uses the "as shot" settings for RAW conversion. Anything you do in Photoshop camera raw will not be acknowledged by Helicon.

You could take your finished stack back into PS CS3 as ACR allows for TIF and JPG images now. I would rather make all of the adjustments in 16 bit and prior to reconstruction of the image.

Therefore, I do all of my adjustments in ACR and export TIF files and then let Helicon work on it.


I'm sure that JPG does fine also, but again you gotta be sure everything is good before you hit helicon as adjusting a JPG is "destructive".

Phil Savoie
Posts: 19
Joined: Tue Feb 23, 2010 9:44 pm
Location: Bozeman Montana
Contact:

Latest wisdom for stacking workflow with ZSP

Post by Phil Savoie »

I found this thread when searching for stacking workflow, as it's a few years old I'm wondering what the current ideas are pertaining to workflow for Zerene Stacker. I'm shooting a Canon 7D in large jpeg and raw. The large jpegs on the 7D aren't as good as the raw obviously- are members using the camera jpegs to test stacks and then go back to the raw files and convert them to better/larger jpegs to input into ZS when they find a shot they like?

cheers, Phil
Phil Savoie

Craig Gerard
Posts: 2877
Joined: Sat May 01, 2010 1:51 am
Location: Australia

Post by Craig Gerard »

Phil,

Which RAW image processing software are you using?

The one I use allows for Export 'recipes'. I shoot in RAW and can then select to export in a variety of user nominated/configered 'recipes'. The main recipe is 16 bit TIFF; an alternate recipe is quality and resolution variable JPEG format (including QuickProof™); DNG is another option, etc.

My Canon 50D is set to capture only RAW files, although the option is available to also capture a jpeg, at a nominated resolution, at the same time; but I don't use this feature because of the reasons explained in the previous paragraph, however, there are occasions when this facility is useful.

When stacking, I tend to use 16 bit TIFF files; but 'QuickProof™' jpeg files are useful for 'test runs' when exploring the potential of a 'stack' or variations of a stack. Working with the low-resolution jpeg files provides an indication if a series of slices will provide a worthwhile 'stack'. If they do, then I send in the TIFF files.

Like Groucho Marx said, “Those are my principles, and if you don’t like them… well, I have others.”

Craig
To use a classic quote from 'Antz' - "I almost know exactly what I'm doing!"

Post Reply Previous topicNext topic