What is "stack-and-stitch", and how can I do it?

Here are summaries and links to discussions for Frequently Asked Questions.

Moderators: rjlittlefield, ChrisR, Chris S., Pau

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

What is "stack-and-stitch", and how can I do it?

Post by rjlittlefield »

"Stack-and-stitch" refers to shooting multiple focus stacks of the same subject and stitching them together side by side to cover a larger area.

This technique is most commonly used to create images that have very high resolution, far beyond what can be captured by the camera and lens from a single position. It can also be used to capture an especially wide angle of view, up to 360 degrees in the extreme case.

The primary challenge is to avoid "parallax errors". These occur when different stacks have different perspective, due to movement of the lens's entrance pupil. When this occurs, features of the subject that are at different depths will line up differently in different stacks, making it impossible to get a perfectly clean seam.

There are several distinctly different methods to overcome this challenge. They include:

1. Use a lens that focuses by internally moving elements that are behind the entrance pupil. These lenses are commonly found on compact cameras. See HERE for discussion. In this scheme, you shoot different stacks by rotating the camera around the lens's entrance pupil.

2. Hold the lens in a fixed position and focus by changing the lens-to-camera distance. This is typically done with a bellows rig, such as HERE or HERE. Again, rotate around the lens's entrance pupil.

3. Use "telecentric" optics. More precisely, use optics that are telecentric on the object side. These are unusual lens systems whose entrance pupils are effectively located at infinity. They stare straight forward, having zero angle of view and no change in magnification with distance. Stacks shot with telecentric optics naturally give orthographic (parallel) projection, so stacks for stitching can be shot just by shifting the camera and lens sideways, no rotation. (Rotating around the entrance pupil at infinity is equivalent to this shift.) This approach is convenient for scanning small specimens. Telecentric lenses designed for machine vision can be purchased for small cameras. For DSLRs, telecentric optics can be constructed by adding a carefully chosen auxiliary lens to an ordinary macro lens (see HERE). At higher magnifications, with some lenses, you can get the same effect by adding an aperture at a carefully chosen position (see HERE). Some microscope objectives are naturally telecentric, such as the Mitutoyo M Plan Apo 5X and all of the Nikon "toolmaker" objectives (TM series).

4. At sufficiently high magnification, you can just pretend that the optics are telecentric, by turning off the usual scale correction when stacking. This approach depends on shallow DOF to prevent radial streaking. It becomes viable when field size drops below 5 mm or so, depending on lens design and aperture. See HERE for an example. This approach is especially good when shooting through a microscope objective, where none of the other approaches are really viable.

5. If you're willing to either tolerate some stitching errors or to fix them up by retouching, then you can also use approximate solutions like focusing by moving the lens, and rotating around some average position of the entrance pupil.

Pretty much any stacking software and any panorama-stitching software can be used for these jobs. Stacking software includes Zerene Stacker, Helicon Focus, and CombineZP; stitching software includes Photoshop, PTgui, Microsoft ICE, hugin, and PTassembler.

A follow-up question is frequently asked: should I stack first, or stitch first? In most cases, it is better to stack first. The main reason is that the stitching process figures out how to align the tiles based on their image content, and that process is likely to make slightly different decisions for each stitch. If you stitch first, the variability in aligning tiles often results in warping the various focus layers in ways that cause smaller features to be misaligned during the focus stacking process, and those misalignments in turn cause localized smearing, ghosts, or echoes. In principle a similar problem can happen if you stack first, where variable alignment in stacking can cause parallax errors in the stitch. This latter problem seems less common, but in difficult cases you may have to try it both ways and see what works the best.

Questions? What more needs to be said or linked?

--Rik

Edit: added Photoshop to the list of stitching software. I've left Photoshop off the list for stacking only because of its recognized limitations in that area.
Edit: added mention of some microscope objectives that are naturally telecentric.
Edit: added advice to stack first.
Last edited by rjlittlefield on Tue Mar 14, 2017 11:37 am, edited 3 times in total.

DumbMarine
Posts: 58
Joined: Sat Jun 16, 2012 7:31 am
Location: South-West England

Post by DumbMarine »

Rik,

I dont know whether this would work or even contribute to the discussion here but could a (well set-up) panoramic head be used to rotate around the entrance pupil (front element of the objective) to reduce later work required to clean-up?

regards
Ian

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

could a (well set-up) panoramic head be used to rotate around the entrance pupil (front element of the objective)
In theory yes. In practice it would depend on how tradeoffs work out for the whole system. I think the best chance with this approach would be to use an infinity objective in front of an internal focusing tube lens so that nothing outside the lens would have to move.

By the way, the entrance pupil of an objective is often not its front element. As always, the entrance pupil is the apparent position of the limiting aperture. In many objectives that position is outside the lens, either in front or in back, courtesy the effect of refracting elements in front of the physical aperture. Often the entrance pupil is much farther away from the subject than the front element is. I have a couple of 10X objectives that are almost telecentric by themselves.

--Rik

UncleChip
Posts: 59
Joined: Wed Jan 11, 2023 1:56 pm
Location: Uk

Re: What is "stack-and-stitch", and how can I do it?

Post by UncleChip »

I have only done 2 stack and stitch images with the lowia 2.5/5x at 4x I had been contemplating the parallax and subsequent stitching issues, I set up moving the camera parallel to the subject, my next was moving around the entrance pupil but I was perplexed how to account for the depth and field of view,
Why is everything so complicated :shock:
I have had a quick read over your articles over the various links and now grasp the concept, it explained my problems and confirmed my theory that the only way to make the lens work would be with some serious photoshop manipulation,
The telecentric lens setup will work best for me as I have excellent linear movement in the XYZ axis

If you use any infinity objective would this work as a telecentric lens? You mention the Mitutoyo mplan 5x, would this be the same for the 10x Mitutoyo mplan?

And for my normal macro lenses I will need to determine the entrance pupil position then set my achromatic lens it’s focal length in front of said entrance pupil

Lou Jost
Posts: 5942
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:

Re: What is "stack-and-stitch", and how can I do it?

Post by Lou Jost »

There's one more method, which I think is the easiest and most perfect method for stitching. Use a lens with a large image circle and stack and stitch by moving the sensor across the single aerial image of the lens, which does not move in the x-y plane. These can be stitched essentially perfectly.

This can be done quite easily using a medium format lens and readily-available tilt/shift adapters for medium format lenses to 35mm sensors. There is usually a tripod collar on the adapter so the lens stays still and the camera gets shifted around.

I've just modified one of these adapters so it accepts enlarger lenses instead of medium format lenses. I'll eventually report on the results.

It can also be done with a large-format lens or enlarger lens or reversed camera lens and a view camera with a 35mm camera on the back standard.

Lou

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Re: What is "stack-and-stitch", and how can I do it?

Post by rjlittlefield »

UncleChip wrote:
Fri Mar 24, 2023 6:01 pm
If you use any infinity objective would this work as a telecentric lens? You mention the Mitutoyo mplan 5x, would this be the same for the 10x Mitutoyo mplan?
It really makes no difference whether the objective is finite or infinite. Those terms describe what happens behind the lens, not what happens in front of the lens where it matters for stack-and-stitch.

But conveniently, almost any high magnification objective, say 10X or above, can be treated as if it were telecentric, regardless of whether it actually is. This is because the depth of field is so shallow that the image scale cannot change much while any particular feature is sharp enough to matter. When stacking, just turn off scale adjustment. Turning off scale adjustment is a standard recommendation at high magnification, because with many subjects the adjustment is likely to be led astray by changes in appearance as objects go in and out of focus.

For what it's worth, I have just now checked my Mitutoyo M Plan Apo 10X NA 0.28. Scale ratio measures about 1.00009 per 10 microns of focus change. On my Canon R7 with 6960 pixels sensor width, stepping at 5 microns, that amounts to a maximum shift between adjacent frames of about 0.16 pixels at frame edge, when the scale change is ignored.
And for my normal macro lenses I will need to determine the entrance pupil position then set my achromatic lens it’s focal length in front of said entrance pupil
Correct.

--Rik

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Re: What is "stack-and-stitch", and how can I do it?

Post by rjlittlefield »

Lou Jost wrote:
Fri Mar 24, 2023 9:19 pm
There's one more method, which I think is the easiest and most perfect method for stitching. Use a lens with a large image circle and stack and stitch by moving the sensor across the single aerial image of the lens, which does not move in the x-y plane. These can be stitched essentially perfectly.

This can be done quite easily using a medium format lens and readily-available tilt/shift adapters for medium format lenses to 35mm sensors. There is usually a tripod collar on the adapter so the lens stays still and the camera gets shifted around.
That's a good point. If you have a lens that covers your whole subject at the resolution you need, but your sensor won't cover the image, then it's best to scan the image by moving the sensor around.

All the techniques mentioned in the original post are addressing a different problem, where you don't have a lens like that so you have to scan the subject.

--Rik

Lou Jost
Posts: 5942
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:

Re: What is "stack-and-stitch", and how can I do it?

Post by Lou Jost »

Some Printing Nikkors are good examples of such a lens. The Rayfact 95mm 2x has an image circle of 64mm. Much of it is wasted on FF if you don't shift the sensor around. You could get a 50% larger FOV by doing this.

Post Reply Previous topicNext topic