Anatomy, A DIY Precision S & S System (First Light)

mawyatt · Post by **mawyatt** » Sat Dec 01, 2018 1:56 pm

While I was writing a note on the Nikon Remote Interface I had started a Stack and Stitch Session utilizing the Wemacro Vertical setup using a KR20 Z axis rail and a pair of KR15 for X and Y. The camera was a D850 with a DI5400 lens on a bellows (just grabbed it since it was available) with ~160mm extension from camera flange to DI5400 base. Vertical setup fixture was just sitting on a table and lighting was 3 Adorama manual (no TTL) strobes and a styrofoam cup around lens for some diffusion. D850 was set to 1 second exposure with EFCS and ESCS shutter mode. Strobes were triggered wireless and controlled wirelessly via Adorama R2 to set exposure. A Raspberry Pi 3B operated by VNC was the controlling computer.

I picked a chip that was available, very dirty though. It's about ~4mm by 3mm and has solder balls. The session was 9 long with 2000um steps in X and 1500um in Y. Step size in Z axis was 50um, and number of Z steps ranged from 27 to 35 at various X(Y) positions, for ~300 total. Timing was 1 second exposure, 5 second rail delay (settling), 1/4 second strobe delay and 50ms trigger pulse widths for camera trigger and strobe trigger.

Did a quick exposure setting with strobes (sorry, solder balls are way overexposed) and started the session without any intervention, totally "hands-off" and that included the remote Mac VNC...the session ran completely unattended as I intend to use the system. All positions of X and Y with Z stacks were collected hands-off.

Outputs processed with Zerene in both Aligned and Unaligned PMap, and Unaligned used with Photoshop Panorama under the "Automated" mode. BTW this stitched without any adjustments to any of the images, no cropping, nothing, and no adjustments in PS either! Never had that happen with PS, always had to crop, tweak and manipulate the stacked images to get them to stitch with PS....not this time!! The image has not had anything done other than stacked with Pmap and Stitched with PS, nothing else. Everything was done in JPEG, all inputs and outputs.

It's dirty with lots of debris, color is off (no white balance performed), needs cropping and lots of cleanup, not a good image at all, I haven't even tried to find the stitch marks yet as PS just finished.....but the S&S Setup worked flawlessly. The JPEG is ~60MB file size.

Anyway, here's "First Light" of the S&S System. I'll add some notes to this later.

Added edit. The file system I'm using to store the stack, stitch and timing parameters is just a simple text file. It uses a specified name and appends "_T" for example for the time, "_X", "_Y" and "_Z" for the X, Y and Z rail parameters. These files just contain numbers and arrays separated by line returns...about as simple as you can get!! You can create a custom session by just making a text file if you want, or use the main program to create & store the session. Here's an example of the Z axis files for this image session.

0.0
array('f', [4600.0, 4500.0, 4500.0, 4500.0, 4700.0, 4600.0, 5000.0, 4900.0, 4900.0])
array('f', [5900.0, 5900.0, 5900.0, 6200.0, 6200.0, 6100.0, 6300.0, 6300.0, 6200.0])
array('f', [1300.0, 1400.0, 1400.0, 1700.0, 1500.0, 1500.0, 1300.0, 1400.0, 1300.0])
array('f', [50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0])
array('b', [27, 29, 29, 35, 31, 31, 27, 29, 27])

Where:

Z Zero # Axis 0 Point
Z Start Array # Axis Start Point at X(x)Y(y)
Z End Array # Axis End Point
Z Range Array # Axis Range
Z Step Size Array # Axis Step Size
Z Step Numbers Array # Axis Number of Steps

So the first Z stack is X 1st at Y 1st stack starts at 4600, ends at 5900, has a range of 1300, step size of 50 and is 27 steps. The next to last Z stack is X 2rd at Y 3rd starts at 4900, ends at 6300, range of 1400, step size 50, and 29 steps.

Best,

ray_parkhurst · Post by **ray_parkhurst** » Sat Dec 01, 2018 9:09 pm

It's too small to tell much about IQ, ultimate resolution, etc. Can you upload it to EasyZoom or other for viewing?

Also, it's very tough to follow your description. I can't even figure out how many images are in X and Y. What are the panorama details?

From the jagged edge it seems there are significant XY misalignments vs the sensor. But it's hard to tell if PS has any issue with this looking at the small pic.

mawyatt · Post by **mawyatt** » Sat Dec 01, 2018 9:50 pm

Ray,

The image isn't very good, but just a "proof" the S&S system behaved as expected and didn't do anything unexpected or "bad". I don't have an account on that site anyway. I just played with the image a little, cropping it, tone mapping, color correction and a little cleaning up some of the debris (still lots of stuff, this chip was covered). I may put it on Nikon Image Space.

If I use it again I'll clean it, so it's at least more presentable and use the 5X Mitty instead of the DI5400. Also need to use a much better lighting setup, I only used a cup over the lens end.

The good result will be coming soon, if I can get some free time in the next week or so.

The basic parameters are/were X was moved from 0 to 4000 microns in 2000 microns steps, while Y was moved from 0 to 3000 microns in 1500 micron steps. At each X of Y position the Z axis was stacked with the parameters shown.

So the session went as Z stacked from 4600 to 5900 at X(0) and Y(0), then Z stacked from 4500 to 5900 at X(2000) Y(0), then 4500 to 5900 at X(4000) Y(0), then 4500 to 6200 at X(0) Y(1500) and so on until the end with 4900 to 6200 at X(4000) Y(3000). I could have used a finer step but didn't want to wait too long, but it actually went pretty fast since no time was wasted and the motors are setup for high velocity but precise positioning using velocity ramping up and down. I also don't waste extra time waiting for the motor to settle to position, because I pole the controller and begin the time delay as soon as the Z axis reaches position, so I'm assured the Z axis has reached destination but don't need to wait any longer. The axis are controlled so the Z axis is pulled up out of the way after a stack completion and the X and Y move, but all three can be moving at once but no chance of the lens hitting the subject.

All 3 axis returned to their exact starting and zero positions after the full session.

The blank image was very handy when stacking with Zerene, as each stacking session could be easily identified. In fact I can tell you exactly where each image was taken in precise XY and Z coordinates.

Here's what I looks like cleaned up a bit, but needs lots of work!!

Best,

mawyatt · Post by **mawyatt** » Sun Dec 02, 2018 8:27 am

ray_parkhurst wrote:It's too small to tell much about IQ, ultimate resolution, etc. Can you upload it to EasyZoom or other for viewing?

Also, it's very tough to follow your description. I can't even figure out how many images are in X and Y. What are the panorama details?

From the jagged edge it seems there are significant XY misalignments vs the sensor. But it's hard to tell if PS has any issue with this looking at the small pic.

Ray,

I've uploaded the unedited ~9000 by 13000 pixel image to Nikon Space, and the slightly cleaned up edited version, so you can take a look there.

http://img.gg/tnylrN5
http://img.gg/ty9aO1K

There was no initial alignment done to the fixture other than just look at things to "see" if they looked OK, so no precise X, Y nor Z alignment. The only thing I did prior to actually beginning the image capture was set the exposure (which I overexposed the solder balls

) with the strobe power. Also please remember this IS NOT an image for evaluation of subject, lens/camera, resolution, etc., ONLY FOR evaluation of characteristics of experimental S&S HARDWARE, SOFTWARE and ALGORITHMS. My only goal was to see if things didn't "crash and burn".

The high quality images will be coming soon, when I get some time.

This setup works very well, better than any setup I have, including by beloved Stackshot system! I'll still be adding features and whatnot to the code, but the hardware is pretty firm now, especially since I've figured out that darn Nikon trigger system (I think

).

If you don't want to mess around with Windows (or Macs) for remote desktops, I believe you can run the control Raspberry from another Raspberry since these are full fledged Linux machines.

I'll probably roll a couple more of these based the Raspberry Zero W and Raspberry Pi 3B+ that I have to use with different S&S setups (Thor) and maybe using larger X and Y rails (I have some Stackshot, Wemacro, MJKZZ extra rails) for more X and Y range (KR15 only have ~30mm range).

Anyway, get a Raspberry Pi (or Arduino) and start playing round, these are amazingly useful tools for us macro folks!!

Best,

ray_parkhurst · Post by **ray_parkhurst** » Sun Dec 02, 2018 8:48 am

mawyatt wrote:I've uploaded the unedited ~9000 by 13000 pixel image to Nikon Space, and the slightly cleaned up edited version, so you can take a look there.

http://img.gg/tnylrN5
http://img.gg/ty9aO1K

There was no initial alignment done to the fixture other than just look at things to "see" if they looked OK, so no precise X, Y nor Z alignment. The only thing I did prior to actually beginning the image capture was set the exposure (which I overexposed the solder balls ) with the strobe power. Also please remember this IS NOT an image for evaluation of subject, lens/camera, resolution, etc., ONLY FOR evaluation of characteristics of experimental S&S HARDWARE, SOFTWARE and ALGORITHMS. My only goal was to see if things didn't "crash and burn".

I did not mean to be overly critical of your first image from the machine. Mostly I was curious about the odd way it seems to have stitched. The edges of the uncropped image don't seem to be lined up in a way I'd expect a 3x3 panorama to stitch. I guess I'm just not familiar with how PS stitches.

mawyatt · Post by **mawyatt** » Sun Dec 02, 2018 10:48 am

ray_parkhurst wrote:
mawyatt wrote:I've uploaded the unedited ~9000 by 13000 pixel image to Nikon Space, and the slightly cleaned up edited version, so you can take a look there.

http://img.gg/tnylrN5
http://img.gg/ty9aO1K

There was no initial alignment done to the fixture other than just look at things to "see" if they looked OK, so no precise X, Y nor Z alignment. The only thing I did prior to actually beginning the image capture was set the exposure (which I overexposed the solder balls ) with the strobe power. Also please remember this IS NOT an image for evaluation of subject, lens/camera, resolution, etc., ONLY FOR evaluation of characteristics of experimental S&S HARDWARE, SOFTWARE and ALGORITHMS. My only goal was to see if things didn't "crash and burn".
I did not mean to be overly critical of your first image from the machine. Mostly I was curious about the odd way it seems to have stitched. The edges of the uncropped image don't seem to be lined up in a way I'd expect a 3x3 panorama to stitch. I guess I'm just not familiar with how PS stitches.

Ray,

Critical assessments are welcome, so that's fine. I don't know how PS works either, but I've never been able to get it to do even a halfway decent initial stitch without a lot of tile image manipulation, and even then more often than not PS would just choke.

One thing to remember this is not a uniform S&S as you might expect. I thought about this and decided I wanted each tile session to be independent of all other tiles. What this basically means is that each Z axis stack at each tile can have different start, end and even step sizes. So this isn't a uniform Z axis stack across all X and Y positions, although it can be if you want.

Why I did it this way was to save images of unwanted space. If you think of a square titled back with one corner in lower somewhat center (kind of like this and other chips image I do), then at the left and right upper and lower extremes lots of space is not required for the final rendering. If you are using high magnifications (small FOV), then most of that blank space isn't even in the camera frame, and even if a slight amount of the subject is, that portion of the subject is well covered in another tile section. So these areas are just wasting time to image. Some of the insects images have lots of "wasted" space around the subject, so when manual stitching I would assume folks would adjust the FOV to only cover the important areas and not the wasted space, so I've attempted to replicate this somewhat with allowing (if wanted) separate Z axis parameters at each X Y tile position.

I used the Minolta Di5400 scanner lens that Robert found, used it since it was already mounted to a bellows and D850 on a THK KR20, and all I had to do was slide it in the Wemacro Stand vertical ARCA clamp and plug in the motor, and as mentioned no attempts at precise alignment in any way.

I don't know how close this lens is to being telecentric, but my guess it's not very close. This would introduce some image variations that PS would have to deal with, and it was on automatic mode, so all image adjusting parameters were active. But all this was stuff I didn't care about at this time, since my goal was just to see if things worked as anticipated (I had my fingers crossed too

), and not have any user intervention, otherwise I would have considered it a failure (wouldn't have posted it either

) and embarrassing!!

I just ran PS Panorama with the Pmax tiles that were stacked with Alignment ON, got a cup of coffee and this is what PS produced. Uploaded to Nikon Image Space.

http://img.gg/3E9PoNx

Best,

ChrisR · Post by **ChrisR** » Sun Dec 02, 2018 11:28 am

Mike for me the N-I-S images don't zoom, they're not copyable and they're a bit smaller on my screen than here

mawyatt · Post by **mawyatt** » Sun Dec 02, 2018 11:39 am

ChrisR wrote:Mike for me the N-I-S images don't zoom, they're not copyable and they're a bit smaller on my screen than here

Chris,

I think you can download them, let me know if you can't.

Best,

rjlittlefield · Post by **rjlittlefield** » Sun Dec 02, 2018 1:35 pm

mawyatt wrote:I think you can download them, let me know if you can't.

I don't see any way to download. I'm not a member of N-I-S, if that matters.

--Rik

mawyatt · Post by **mawyatt** » Sun Dec 02, 2018 2:47 pm

All,

See if this works. Please tell me if this works?

http://img.gg/10oj9Jm

Download should be in upper left hand corner.

Best,

rjlittlefield · Post by **rjlittlefield** » Sun Dec 02, 2018 3:19 pm

Yes, I can download now, thanks.

I know that this is first-light, not for evaluation, etc., but I did notice one oddness about sharpness. Comparing with Zerene alignment on versus off, the tile at center right, the one showing "ATLASTC2", is a lot sharper one way but the tile at center left seems to be just the reverse. I have no idea what's going on there, just wanted to call it to your attention.

I'm also not completely sure which is aligned, which not, because of how the N-I-S browser interface truncates names on the download selection page.

--Rik

mawyatt · Post by **mawyatt** » Sun Dec 02, 2018 3:50 pm

rjlittlefield wrote:Yes, I can download now, thanks.

I know that this is first-light, not for evaluation, etc., but I did notice one oddness about sharpness. Comparing with Zerene alignment on versus off, the tile at center right, the one showing "ATLASTC2", is a lot sharper one way but the tile at center left seems to be just the reverse. I have no idea what's going on there, just wanted to call it to your attention.

I'm also not completely sure which is aligned, which not, because of how the N-I-S browser interface truncates names on the download selection page.

--Rik

Rik,

The one of the left is with Zerene Alignment ON, the next one to the right is OFF, as is the edited version.

Honestly don't know about the sharpness between different sections. I had lots of "other" things going on while this S&S session was taking place, so lots of possibilities. When I do a "proper" image for evaluation I'll/we'll be able to dig in further and investigate some IQ related things.

I didn't even pay attention to Zerene when it was stacking which I usually do. Just did a tile stack with Alignment ON then OFF, then on to the next tile. Heck, I didn't even look at the finished stacked images before I loading them into PS Panorama figuring it would bark at me about something or just choke as it usually does, then out popped an image...same for using non-aligned & aligned stacked tile images. I was and still quite surprised how well this went, everything just worked from beginning to end without any issues or hiccups!!

Just in case Sir Edsel Murphy is watching, "I am not complaining!!", so please stay away

This isn't about any usual member, just hope you know who "Murphy" is

He's been "helping" me for as long as I can remember

Murphy's law

Best,

mawyatt · Post by **mawyatt** » Tue Dec 11, 2018 11:18 pm

Finally got to getting a stacking session to really check things out over the past weekend.

As mentioned Murphy did get me, he didn't stay away

I had a long stack & stitch session going with 25 microns steps (Mitty 5X) and was near the end of a session after ~1700 images. The Raspberry Pi haunted and Python indicated a numerical error. After tracing and correcting the problem I started another session, these take about 3 hrs. I got an old wafer sliver from my somewhat failed attempt at severing a wafer along the silicon crystal preference orientation, the cut was straight except near one end where a sliver broke off and I just had it laying around (badly scratched and dirty), so decided to use this wafer sliver. This makes a extremely difficult subject when tilted because it has a large length to width ratio, and of course the repeated precise features.

The X was set to 16mm in 4mm steps (5), Y 13.75mm in 2.75mm steps (6) and Z was 25 micron steps at various ranges on some tiles (30 total). I've added some routines to allow easier setup on the X, Y and Z stack parameters and this made initial setup better. Nikon D500 with Mitty 5X and Raynox 150 tube lens was used and didn't use any light diffusion. After ~3hrs (almost 2K images) of hands off, no computer interface either, the session completed without an issue.

Stacked the selected tiles in Zerene, ~20 tiles in all, some are null tiles (no subject, so they weren't used) with all alignments OFF and Pmax.

PS completely folded when stitching attempted, and even PTgui couldn't handle this particularly subject, well at least I haven't been able to get any useful images yet (I'm sure it will I just need to spend more time on it!!).

So I finally gave up and used PS manual mode. First I printed out the images and aligned them so I had a hardcopy printed reference, then with PS (no alignments or image manipulation, just move the tiles around by hand in X and Y until things line up. Didn't use any edge blending or scale adjusts either, just manually line up in X and Y without any rotation. Things lined up without too much stitching artifacts and here's the result without anything done, no sharping, tone mapping, cleanup, just a simple crop. The lack of using any light diffusion is evident (bright spots), as is the condition of the sliver!! The PS file is ~1GB, JPEG is 111MB

Hopefully I'll get the stitching SW working and get a better result, then I'll probably clean it up some.

Best,

pbraub · Post by **pbraub** » Wed Dec 12, 2018 12:05 am

Hi Mike,

thank you for your detailed report on developing and building your system. I really enjoy reading along.

Have you tried ImageJ for stitching your images (especially the Grid/Collection stitching plugin). I have successfully stitched microscopy image up to 10 GB on my relatively modest machine (Xeon 1230v2, 32 GB Ram). One thing I do not know however is how RGB images are handeled.

Kind regards
Peter

mawyatt · Post by **mawyatt** » Wed Dec 12, 2018 7:16 am

pbraub wrote:Hi Mike,

thank you for your detailed report on developing and building your system. I really enjoy reading along.

Have you tried ImageJ for stitching your images (especially the Grid/Collection stitching plugin). I have successfully stitched microscopy image up to 10 GB on my relatively modest machine (Xeon 1230v2, 32 GB Ram). One thing I do not know however is how RGB images are handeled.

Kind regards
Peter

Peter,

Thanks Ill take a look at this.

Best,