A7r4 pixel shift. Rather a resolution eye-opener

Have questions about the equipment used for macro- or micro- photography? Post those questions in this forum.

Moderators: rjlittlefield, ChrisR, Chris S., Pau

Beatsy
Posts: 2105
Joined: Fri Jul 05, 2013 3:10 am
Location: Malvern, UK

A7r4 pixel shift. Rather a resolution eye-opener

Post by Beatsy »

I've recently been using Sony A7riv (61 megapixel full frame) on my main macro rig. Before that I'd used only A7rii (42mpix) and A9 (24mpix) in this context and had no plans to use riv for studio macro. I figured A7rii already captured all the resolution my lenses and objectives could deliver so why over-sample for no benefit (as I thought the riv would)? But after using 4-shot pixel shift on the microscope and seeing an enormous IQ increase from the extra colour resolution, I figured that might have some uses in studio macro too. Hence plugging the A7riv in for a quick look-see. While I was at it, I figured I might as well try 16-shot pixel shift too, even though I doubted it would work well.

Outcome? Well, I couldn't have been more wrong! Using pushed down Mitties on a 135mm tube lens and the camera in APS-C crop mode + pixel shift, the setup can squeeze 104 megapixels of detail onto APS-C with the 5x, 7.5x, 10x and 20x Mitties. At 50x things start to look a bit soft though (NA too low - and excluded that from the test anyway). I've attached a few of the tests consisting of a full APS-C frame and 100% crop for each of the 5x, 10x and 20x Mitties.

For quick routine stacks I think this is a pretty astonishing result in terms of captured resolution. I wonder what it's *really* equivalent to though? Although 4 times as many "positions" are sampled by moving the sensor half a pixel, it's still a whole pixel that captures the light at each half-pixel location (kind of sub-sampling neighbouring pixels). My guess is the outcome is much like what a heavy anti-aliasing filter would do. Although a virtual 240 megapixels are captured, spatially they would resolve less than that (but more than 61mpix). All guesses though, I have no idea if or how the pixel shift algorithm works, maybe it can squeeze extra information from the overlaps. I wish Sony would document such things. But whatever theory and numbers say - in practice, the outcome looks pretty darned impressive to me. I mean, extreme macro onto 104 megapixels in APS-C mode! Barely seems feasible. The laws of physics might be getting nervous soon :D

The test subject is a single 7.1mm wide rectangle cleaved from the middle of a silicon wafer full of otherwise mundane chips (almost fills the FoV of the 5x as shown in the first pic). The material is a bit tough to light for good contrast and the noise-like granular substrate is a real annoyance, but it does the job. Lights, diffusion and subject remained in the exact same positions for all three lens tests.

5x mitty pushed down to 3.4x plus 100% crop from the main image. Had to use a lot of jpeg compression on these, so quality not too good - but you'll get the idea.
PS16-apsc-5x-at-3.375x.jpg
PS16-apsc-5x-at-3.375x-crop.jpg
10x mitty pushed down to 6.75x - each pixel is covering about quarter of a micron on the subject!
PS16-apsc-10x-at-6.75.jpg
PS16-apsc-10x-at-6.75x-crop.jpg
And finally, 20x mitty at 13.5x - diffraction softening getting noticeable, but nothing a bit of standard sharpening won't fix.
PS16-apsc-20x-at-13.5x.jpg
PS16-apsc-20x-at-13.5x-crop.jpg

lothman
Posts: 958
Joined: Sat Feb 14, 2009 7:00 am
Location: Stuttgart/Germany

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by lothman »

I found similar improvement of Pixelshift of A7riv on the Laowa 2,5-5x macro lens

viewtopic.php?f=27&t=43097

But I have no clue how I could handle all those files when stacking.

Doppler9000
Posts: 111
Joined: Thu Oct 13, 2011 3:56 pm

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by Doppler9000 »

Here is an article on the topic by Jim Kasson.

https://blog.kasson.com/the-last-word/d ... esolution/

Lou Jost
Posts: 5944
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by Lou Jost »

Beautiful results, in line with what I have been seeing, first in MFT and now in Pentax and Panasonic FF. This isn't pushing against scientific limits. First, you have the Bayer filtering to deal with. Even just a four-image shift that does not increase the number of pixels (as in my Penatx K1) gives a very significant increase in real resolution, as you noted. Second, just because point-spread functions of nearby points overlap significantly (as when using the Rayleigh criterion), I think this does not mean that we cannot resolve them, because there is still a minimum between the maxima. Anything that reduces noise (as pixel-shifting does) should increase resolution. And it always helps to put more pixels under a feature. Three pixels is a good target, and more pixels should help (unless smaller pixels do not generate too much extra noise), though we eventually reach a point of diminishing returns. Rik and I have shown through tests that good lenses and objectives far outresolve our sensors:
https://www.photomacrography.net/forum/ ... 00#p178314
I estimated that an APS-C sensor should have at least 120Mp to do justice to a Mitutoyo 10x.

Regarding the actual real resolution of eight-shot pixel-shifted images, Olympus has thought about this and offers the output in two ways: as an 80Mp raw file with the pixel dimensions of the original pixel-shifted image, and also as a reduced 50Mp jpg which is claimed to efficiently match the "real" resolution of the image. I think this is probably a good estimate of how much real resolution there is in 8-shot shifted images, relative to the nominal resolution. Your 16-shot shift must do better than the eight-shot shift..... Very impressive!

RobertOToole
Posts: 2627
Joined: Thu Jan 17, 2013 9:34 pm
Location: United States
Contact:

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by RobertOToole »

Hi Beatsy,

Pixel Shift Multi Shoot mode is fantastic! In my mag.x 8x test with the A7RIV (here: https://www.photomacrography.net/forum/ ... 25&t=43165 + my site: https://www.closeuphotography.com/qiopt ... -objective) I posted some notes on pixel shift mode, pasting them below in case you or anyone missed them:


"On the Sony α7R IV 4 the 16-image Pixel Shift Multi Shoot (16PSMS) 240MP images are bigger spatially, 60 MP vs 240 MP, so this would be an advantage for printing, but they aren't really any sharper on a monitor when compared to 4-image Pixel Shift Multi Shoot (4PSMS) 60MP images.

The biggest gain with either Pixel Shift Multi Shoot mode, 4 or 16, is at least for me, higher color resolution and lack of demosaicing (or de-bayering) artifacts like false color and moiré, and less noise and more dynamic range. Mostly since there is no need for demosaicing (or de-bayering).

With PSMS there is a 1 EV reduction is noise, equiv. to giving it one more stop light, compared to non-PSMS images."

The most amazing gains from Pixel Shift Multi Shoot mode in my eyes? Less demosaicing artifacts which means less stair-stepped lines, and less noise/more dynamic range. I posted some examples in the test somewhere. Was going to post them here but I don't want to hijack your thread. :-k

Best,

Robert

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by rjlittlefield »

Doppler9000 wrote:
Wed Apr 07, 2021 4:58 pm
Here is an article on the topic by Jim Kasson.

https://blog.kasson.com/the-last-word/d ... esolution/
Unfortunately the body of that article has one significant flaw, and the subsequent discussion in comments is beyond my patience to try untangling.

As I read it, the analysis sounds OK. The problem is in the assumptions, specifically at the point where Kasson explicitly assumes that the sensor has 100% fill factor, and implicitly assumes that the sampling function is perfectly uniform. Given those assumptions, I agree that the MTF curve does drop to zero at one cycle every two pixels, and there's no way to undo that by shifting.

However, if the sensor does not have 100% fill factor, or in general if the sampling function is not perfectly uniform, then the MTF curve does not drop to zero at one cycle every two pixels, and after that the whole argument falls apart.

As an extreme example, consider the case where the sensor has only 50% fill factor, with a perfect hat sampling function inside that 50%. I think you'll have no trouble convincing yourself that in that situation, the multi-exposure shifted sampling is exactly the same as what you'd get with a single shot using pixels half the size and 100% fill factor. Resolution would double. Ironically, a sensor that is "perfect" at its native resolution (100% fill factor) can gain far less from shifting than a sensor whose sampling function is lumpy (<100% fill).

So, the article's conclusion that sensor shifting cannot give more resolution was preordained by its assumptions about the nature of the sensor.

I rather doubt that any manufacturer is going to tell us enough about their sensor to permit a really good theoretical analysis.

So my plan is to put a lot more faith in experimental results. If somebody claims to get more resolution from sub-pixel shifting, then I will certainly say "show me". But if the images support what they said, I'll be happy to accept that result. There's no violation of theory in getting more resolution from sub-pixel shifting.

--Rik

Chris S.
Site Admin
Posts: 4042
Joined: Sun Apr 05, 2009 9:55 pm
Location: Ohio, USA

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by Chris S. »

Steve,

Any chance we could see a comparison between an image without pixel shift, an image with 4-shot pixel shift, and an image with 16-shot pixel shift? Perhaps made with whichever objective shows the improvement most dramatically? Perhaps resized as necessary for easy comparison?

I fear that's asking a lot to prepare. This said, I can't get a sense of the improvement you're seeing from the "after" images alone--despite the fact that I've looked at my share of image stacks made with Mitutoyo objectives.

The level of improvement you Sony α7R IV users are describing is sobering for someone like me, who is invested in Nikon bodies, accessories, and workflow. That Sony body is going on two years old, and so far, one still hears of nothing similar on the Nikon horizon. Anything similar in the Nikon world is likely years away, or perhaps never. Though I loathe the thought of mixing systems, perhaps it needs to be considered.

--Chris S.

Beatsy
Posts: 2105
Joined: Fri Jul 05, 2013 3:10 am
Location: Malvern, UK

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by Beatsy »

Chris S. wrote:
Wed Apr 07, 2021 10:45 pm
Steve,

Any chance we could see a comparison between an image without pixel shift, an image with 4-shot pixel shift, and an image with 16-shot pixel shift? Perhaps made with whichever objective shows the improvement most dramatically? Perhaps resized as necessary for easy comparison?

I fear that's asking a lot to prepare. This said, I can't get a sense of the improvement you're seeing from the "after" images alone--despite the fact that I've looked at my share of image stacks made with Mitutoyo objectives.
...
Sorry Chris. this really was a quick, qualitative test. I just rattled off the three stacks, transferred the images from card to PC using Imaging Edge (as it combined the shifted shots into single tiffs) and later formatted the card without keeping the individual frames. I know what they look like, you see. My goal here was solely to convey my wonder and surprise at getting genuine 100 megapixel images off an APS-C sensor (well, FF, but in APS-C mode).

I did a comparison shot some time ago, when I first got the camera. I was impressed at the time, but it's not quite up to the improvement I saw this time. More likely the setup and my technique though. Anyway, you can see a nice hi-res comparison pic at https://www.dpreview.com/forums/thread/ ... t-63173757 (if you haven't seen it already). The comparison is similarly stark, but I got slightly better end results this time probably due to lighting.
Last edited by Beatsy on Thu Apr 08, 2021 2:36 am, edited 2 times in total.

Beatsy
Posts: 2105
Joined: Fri Jul 05, 2013 3:10 am
Location: Malvern, UK

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by Beatsy »

lothman wrote:
Wed Apr 07, 2021 3:12 pm
I found similar improvement of Pixelshift of A7riv on the Laowa 2,5-5x macro lens

viewtopic.php?f=27&t=43097

But I have no clue how I could handle all those files when stacking.
The size and amount of files can cause issues - particularly with anything but the shallowest stacks. I take the view that the final pixel-shifted frames are my "source images" - and anything captured before that is temporary and disposable (the 4 or 16 raw frames that make up each image).

I don't tether anymore, preferring to "sneaker net" a memory card over to my PC. I transfer the images by running the Imaging Edge software in this case. It takes the frames direct from the card and deposits the combined images on the PC (which also imports them into a Capture One session). I've never had any frame alignment problems (thanks to the stable rig, I guess) and Imaging Edge makes a perfect conversion every time - so I don't need to "attend" the process and the source frames are binned once I have the converted tiffs on disk. After that, the workflow is no different to normal stacking - albeit with larger images in the 16-shot case, or just nicer ones with the 4-shot :D

Beatsy
Posts: 2105
Joined: Fri Jul 05, 2013 3:10 am
Location: Malvern, UK

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by Beatsy »

RobertOToole wrote:
Wed Apr 07, 2021 6:29 pm
Hi Beatsy,
...The most amazing gains from Pixel Shift Multi Shoot mode in my eyes? Less demosaicing artifacts which means less stair-stepped lines, and less noise/more dynamic range. I posted some examples in the test somewhere. Was going to post them here but I don't want to hijack your thread. :-k
Thanks Robert, I have read some, but not all, of your pixel-shift posts - but will be going back to fix that shortly...

Please post pics if you have them - it's not hijacking at all. Happy to see anything that livens and deepens discussion!

Beatsy
Posts: 2105
Joined: Fri Jul 05, 2013 3:10 am
Location: Malvern, UK

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by Beatsy »

rjlittlefield wrote:
Wed Apr 07, 2021 7:50 pm
Doppler9000 wrote:
Wed Apr 07, 2021 4:58 pm
Here is an article on the topic by Jim Kasson.

https://blog.kasson.com/the-last-word/d ... esolution/
Unfortunately the body of that article has one significant flaw, and the subsequent discussion in comments is beyond my patience to try untangling.
...
So, the article's conclusion that sensor shifting cannot give more resolution was preordained by its assumptions about the nature of the sensor.

I rather doubt that any manufacturer is going to tell us enough about their sensor to permit a really good theoretical analysis.

So my plan is to put a lot more faith in experimental results. If somebody claims to get more resolution from sub-pixel shifting, then I will certainly say "show me". But if the images support what they said, I'll be happy to accept that result. There's no violation of theory in getting more resolution from sub-pixel shifting.
Thanks for that Rik (and Doppler9000 for the link). I tend to respect/believe what Jim Kasson says and pretty much swallow it whole. Useful to have the health warning - especially as it supports my direct experience being somewhat beyond what simplistic theories (and Jim Kasson) might dictate.

Cheers

Beatsy
Posts: 2105
Joined: Fri Jul 05, 2013 3:10 am
Location: Malvern, UK

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by Beatsy »

Lou Jost wrote:
Wed Apr 07, 2021 5:45 pm
... Rik and I have shown through tests that good lenses and objectives far outresolve our sensors:
https://www.photomacrography.net/forum/ ... 00#p178314
I estimated that an APS-C sensor should have at least 120Mp to do justice to a Mitutoyo 10x.
...
Hi Lou,

Hmm, it's easy enough to approach testing that limit by *not* pushing mag down, but still use APS-C. I have an old Hoya 200/3.5 sat here on the desk. I got it to use with the 24mpix A9 in full frame mode (with mitties - which worked well) but never thought to use it for crop mode. I think I'll repeat the test using that and see what happens. If it doesn't push the limits, then I have a Soligor 250/4.5 too. I think I remember that vignetted on FF, but again I didn't think to use it on APS-C. Less FoV of course, but nice to lose the extreme corner smush. Back later...

Lou Jost
Posts: 5944
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by Lou Jost »

Hi Beatsy, which 135mm lens did you use for this test? I remember you had a fancy 135mm (Samyang/Rokinon? or Zeiss?) which you rejected because of vignetting. In crop mode that wouldn't be a problem. I like my cheap Vivitar-Komine 135mm but my Sigma Art 135mm gives better results, apart from hard vignetting in the extreme corners on FF.

Beatsy
Posts: 2105
Joined: Fri Jul 05, 2013 3:10 am
Location: Malvern, UK

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by Beatsy »

Just tried 5x and 10x Mitties on a 200mm tube lens for nominal rated magnifications. Still shooting 16-shot pixel shifted, crop mode, same specimen, tweaked lights. Pics below. Note: no non-pixel-shifted comparisons as it's too onerous in the workflow. This was all just to see if the extra (virtual) resolution afforded by pixel shift would even be useable...

And it certainly is! I believe I see a bit more detail captured with the 200mm tube lens (at the cost of a smaller FoV) but it's well into diminishing returns territory. Clearly the lenses were already approaching the hairy edge when mag was pushed down on 100mpix (with 135mm tube lens). The extra mag pushes the performance envelope hard and softening shows up. It's still a very impressive result though. I never, ever thought I'd be able to capture 100-megapixel macros with decent detail in them. Now if only I could have "fully resolved" AND "fully sharp" at the same time. Life would be grand... :roll:

5x Mitty at 5x onto APS-C + 100% crop from the centre. Lots of jpg compression again, and looks a little worse (softer) than the first anyway, but still shows good, usefully resolved detail from pixel shift images.
testchip5x-pix16-apsc-stack19x4_1.jpg
testchip5x-pix16-apsc-stack19x4_1_1.jpg
10x Mitty at 10x which pretty much resolves all of the smallest details available on the chip. So I didn't bother doing a 20x version as it would be the same detail, twice the size, with just quarter the surface area in view.
testchip10x-pix10-apsc-stack13x4_1.jpg
testchip10x-pix10-apsc-stack13x4_1_1.jpg
Last edited by Beatsy on Thu Apr 08, 2021 12:19 pm, edited 2 times in total.

Beatsy
Posts: 2105
Joined: Fri Jul 05, 2013 3:10 am
Location: Malvern, UK

Re: A7r4 pixel shift. Rather a resolution eye-opener

Post by Beatsy »

Lou Jost wrote:
Thu Apr 08, 2021 7:54 am
Hi Beatsy, which 135mm lens did you use for this test? I remember you had a fancy 135mm (Samyang/Rokinon? or Zeiss?) which you rejected because of vignetting. In crop mode that wouldn't be a problem. I like my cheap Vivitar-Komine 135mm but my Sigma Art 135mm gives better results, apart from hard vignetting in the extreme corners on FF.
Same Vivitar/Komine as you have - from your recommendation many years ago.

Yes, I still have that Samyang - languishing somewhere. I bent a stacking rail with it too (weight + imbalanced). I've replaced it for normal togging with a 135/1.8 G-Master. Now *that* is a proper 135mm lens. It vignettes as a tube lens (yes, it had to be tried) but it's way too good for "normal use" to be used as a tube lens anyway. The Vivitar works very well as it is, no need to change...

Cheers

Post Reply Previous topicNext topic