I have no educated understanding of how the various stacking software programs go about performing or attempting to perform their magic; but I keep thinking that they are going about it the long way.
How many 'states' can a pixel have? If we were to go back to 0 1, on and off, is it possible that a pixel in-focus could be classified as 'on' and a pixel out-of-focus 'off'?
If it were or is this simple then a blending mode called 'focus' used in a similar way to Photoshop's 'lighten', 'darken', 'multiply, 'screen', etc, modes may be an interesting approach to consider??
I realise there are many processes that need to be addressed apart from focus alone, but if the images were placed in layers and 'pin registered' with alignment?
This may seem like a naive question and it probably is; I offer no defence.
Enlighten me.
Craig
Stacking software question ON/OFF
Moderators: rjlittlefield, ChrisR, Chris S., Pau
-
augusthouse
- Posts: 1195
- Joined: Sat Sep 16, 2006 1:39 am
- Location: New South Wales Australia
Stacking software question ON/OFF
To use a classic quote from 'Antz' - "I almost know exactly what I'm doing!"
The way I understand it, an 8 bit pixel has 255 value in each Red, green and Blue channel so it's not simply on or off since that would make the pixel a 1 bit pixel.
I'm guessing the way focus stacking work is by tracking pixels across the range of images where there's a sharp contrast, keeping those while discarding soft gradient where possible.
It's good to know the limitation of the software so that we can create better stacks. I'm really interested in this too.
My question are, would sharp images stack better than softer ones?
Would leaving an overlapping area of focus between the images lets the software track and align them better and also gives better stack?
I'm guessing the way focus stacking work is by tracking pixels across the range of images where there's a sharp contrast, keeping those while discarding soft gradient where possible.
It's good to know the limitation of the software so that we can create better stacks. I'm really interested in this too.
My question are, would sharp images stack better than softer ones?
Would leaving an overlapping area of focus between the images lets the software track and align them better and also gives better stack?
-
Graham Stabler
- Posts: 209
- Joined: Thu Dec 20, 2007 11:22 am
- Location: Swindon, UK
You cannot tell if a single pixel in an image is in focus by looking only at that pixel, all you get is an intensity level, three for colour images. You must look at the variation of the image through that pixel. In focus parts of an image are generally much sharper, the spatial frequency is higher. That is one way to detect in focus information.
Graham
Graham
-
rjlittlefield
- Site Admin
- Posts: 23603
- Joined: Tue Aug 01, 2006 8:34 am
- Location: Richland, Washington State, USA
- Contact:
Every stacking algorithm I'm familiar with is based on the apparently simple concept of "use the info from the best-focused frame(s) at each pixel position".
The challenge is in actually deciding which frame(s) that is. As Graham points out, you can't look at just one pixel, you have to look around at some neighborhood. Make the neighborhood too big, and you get confused because it contains some parts that really are the best focused and other parts that aren't. Make it too small, and you get confused because you can't distinguish pixel noise from subject detail. Try to use heuristics like "neighboring pixels should be sharp in the same or neighboring frames", and you get messed up by depth discontinuities at the edges of overlapping structures.
The problem is surprisingly difficult. People who haven't tried to write a stacking code generally can't appreciate what all can go wrong when working with real-life images.
But to give a flavor of the task, imagine that you're charged with referring a ball game. It's a big game, and viewers are picky. You have 300 million close calls to make, and if you mess up 100 of them (one in 3 million), people will complain that you've fouled up the outcome. To add to the amusement, you have to make each of those calls while watching the game through a soda straw, because that's the largest field of view that you're given time to consider. That's the situation in stacking even 50 frames at 6 MP each.
Every software package I know does conceptually put the images in layers and "pin registers" them in proper alignment before starting to composite them. The alignment is challenging by itself. But there is lots of fun after that point too -- that's when the 300 million decisions occur.
P_T, yes, sharp images do stack better than softer ones with most algorithms. The pyramid approaches like in TuFuse and CombineZP tend to be scale-invariant, so they don't mess up as often when images are soft at the finest levels of detail.
And yes, it does help a lot to have overlapping focus, specifically because it makes the alignment process a lot more precise when you have some detail in similar focus in adjacent frames.
--Rik
The challenge is in actually deciding which frame(s) that is. As Graham points out, you can't look at just one pixel, you have to look around at some neighborhood. Make the neighborhood too big, and you get confused because it contains some parts that really are the best focused and other parts that aren't. Make it too small, and you get confused because you can't distinguish pixel noise from subject detail. Try to use heuristics like "neighboring pixels should be sharp in the same or neighboring frames", and you get messed up by depth discontinuities at the edges of overlapping structures.
The problem is surprisingly difficult. People who haven't tried to write a stacking code generally can't appreciate what all can go wrong when working with real-life images.
But to give a flavor of the task, imagine that you're charged with referring a ball game. It's a big game, and viewers are picky. You have 300 million close calls to make, and if you mess up 100 of them (one in 3 million), people will complain that you've fouled up the outcome. To add to the amusement, you have to make each of those calls while watching the game through a soda straw, because that's the largest field of view that you're given time to consider. That's the situation in stacking even 50 frames at 6 MP each.
Every software package I know does conceptually put the images in layers and "pin registers" them in proper alignment before starting to composite them. The alignment is challenging by itself. But there is lots of fun after that point too -- that's when the 300 million decisions occur.
P_T, yes, sharp images do stack better than softer ones with most algorithms. The pyramid approaches like in TuFuse and CombineZP tend to be scale-invariant, so they don't mess up as often when images are soft at the finest levels of detail.
And yes, it does help a lot to have overlapping focus, specifically because it makes the alignment process a lot more precise when you have some detail in similar focus in adjacent frames.
--Rik
I've worked with camera tracking softwares in CG and the way it works is by having me pick a selection of pixels in each frame so the software can use those as a point in virtual 3D space and recreate the camera positions.
I found the type of operation/tool that gets the user involved give the best result though time has to be sacrificed due to manual nature.
I realise picking an "anchor" point for each image manually would be a pain to do in 100+ images stack but if it can increase the accuracy in aligning 5-10 images shot handheld, I think it will be a much welcome option.
I found the type of operation/tool that gets the user involved give the best result though time has to be sacrificed due to manual nature.
I realise picking an "anchor" point for each image manually would be a pain to do in 100+ images stack but if it can increase the accuracy in aligning 5-10 images shot handheld, I think it will be a much welcome option.