Help with setting up a "machine vision" microscope system

Have questions about the equipment used for macro- or micro- photography? Post those questions in this forum.

Moderators: rjlittlefield, ChrisR, Chris S., Pau

bokemon
Posts: 20
Joined: Sun Jan 16, 2022 6:06 pm
Location: Silicon Valley, CA

Help with setting up a "machine vision" microscope system

Post by bokemon »

Hello folks, new guy here.
I'm designing a system which "effectively" does the same thing as imaging microchips at a long working distance. I.e. needs to image a flat surface, from > 1 inch away, 2 um resolution, and as large a field of view as possible so as to allow digital zooming instead of physically moving things.
After some reading of Robert OToole's blog, I think what could work is the highly respected Mitutoyo 5x objective (NA 0.14, 34mm working distance) plus Thorlabs ITL 200 tube lens. Illumination will be... not sure yet, maybe thru the objective?
There's an additional requirement, which is a fast readout of the sensor (especially in a limited Region Of Interest), so perhaps a regular photography camera won't work, and I have to resort to one of those camera modules with a USB 3 port.  For budget reasons, it'll probably be a APS-C or 4/3" sensor, maybe even smaller.

OK, so here's a few questions:
1) Since the image circle of this objective (~30mm) is larger than the sensor, how about I just use a different spacing on the extension tubes to try and get a lower magnification? Would there be negligible image degradation and the system still have the same resolution as referenced to the sample? (i.e. 2 um)
2) Assuming the resolution "at the sensor" is 2um x mag (5x) = 10 um, how many pixels per 10 um do I need to avoid loss of resolution? Also, this will be a color sensor, so doesn't it actually have lower resolution due to Bayer matrix stuff?
3) Haven't yet looked much into how the lighting will be done, but if thru the objective, what needs to be done, and is the BD Plan objective a better choice?

Any suggestions on better equipment, or how to better implement this would be appreciated. I know there's a Mitutoyo HR objective, but that's on the expensive side, plus the working distance is shorter, so I'd have to redesign things. Research bux will be used to buy the parts, so I can only buy commercially available parts, and not some vintage lens off Ebay.
For reference, there' also the Infinity USA Videomax with much longer working distance. (The K2 Distamax is even better, but getting to be too expensive)
Thanks

Macro_Cosmos
Posts: 1511
Joined: Mon Jan 15, 2018 9:23 pm
Contact:

Re: Help with setting up a "machine vision" microscope system

Post by Macro_Cosmos »

1) Since your sensor is relatively small, you might as well go with the often cheaper QV-Objective 5x with a higher numerical aperture of 0.28, matched to a 100mm tube lens. Adjust the magnification with a zoom lens designed for smaller sensors. Working distance of that objective, I believe, was ~28mm. You are building a measurescope of sorts, I suggest having coaxial illumination and off-axis illumination, they are both important. For an infinity conjugate system, the tube lens is focused to infinity, this allows planar optics to be inserted into the pathway, such as a beamsplitter cube used for coaxial illumination and polarisers. You should not place the tube lens outside of infinity focus to get higher magnification, all kinds of problems kick in.
2) Pixel count depends on your sensor size (type of sensor and wavelength too)
Assuming 550nm.
M 4/3: ~37MP, monochrome
aps-c: ~70MP, monochrome
Formula I use: https://www.microscopyu.com/tutorials/m ... resolution
There are better methods that incorporate more calculations, this one is very quick and produces good results.

Yes, a colour sensor is inferior to monochrome sensors, in almost every aspect aside from price, being fast and easy to use for routine work. It depends on your application. If you don't need colour, get a monochrome camera.
A colour sensor is not as sensitive (lower quantum efficiency), uses a Bayer Array which is usually in an RGGB pattern causing photosites only respond to light within the sub-region of the spectrum.

3) BD plan will allow darkfield illumination. Do you need darkfield? If not, build a coaxial light system and use a couple light emitting diodes mounted to flexible arms for off-axis epi-illumination.

You might be able to get away with just one of these: https://www.ebay.com/sch/i.html?_from=R ... m&_sacat=0
Do research of course, M 4/3 and aps-c sensors might be too large for this.

Chris S.
Site Admin
Posts: 4042
Joined: Sun Apr 05, 2009 9:55 pm
Location: Ohio, USA

Re: Help with setting up a "machine vision" microscope system

Post by Chris S. »

Bokemon, welcome aboard! :D
bokemon wrote:
Sun Jan 16, 2022 10:24 pm
There's an additional requirement, which is a fast readout of the sensor (especially in a limited Region Of Interest), so perhaps a regular photography camera won't work, and I have to resort to one of those camera modules with a USB 3 port. For budget reasons, it'll probably be a APS-C or 4/3" sensor, maybe even smaller.
It might be helpful if you would elaborate a bit on what “fast readout” and “limited region of interest” mean, precisely, in your system. These may be defining requirements. As an example, I routinely work with Mitutoyo objectives on a 24megapixel APS-C sensor (Nikon D7100 body), tethered via USB 2 (I think) to a rather old Windows laptop. Once the shutter closes, a jpeg image shows up on the laptop’s screen at what I perceive to be nearly instantaneous speed, and a full-res raw file appears an eye-blink later. However, I’m using eight-second shutter speeds with an additional two-second settling lag between shots, so my perception of “nearly instantaneous” may be unworkably slow to someone else. If doing machine vision, and fast readout is important, can we characterize that need more fully? I mention this because I suspect it may determine some important limits in your system design and workflow.

“Region of interest” is also important—does this mean the entire sensor frame, or just a portion of it? If just a portion, is this portion constant between shots? Or if it changes, how rapidly and by what means is it changed?

With a 5x Mitutoyo, I suspect that limited depth of field may be a challenging issue for your system. Granted, microchips tend to be very flat, and these microscope objectives have reasonably flat fields. So if you can set up your microchips to be very accurately perpendicular to the optical axis, you may be able to usefully photograph the entire chip without focus stacking. In real-life situations, though, I’d be inclined to focus stack. This would reduce or eliminate issues with focus error, imperfect subject alignment, flatness of lens field, etc. On the downside, doing even limited focus stacking requires multiple exposures of each chip, then software postprocessing. These steps can be made to run pretty rapidly, but depending on the throughput you need, and your requirement for fast readout, it may not be your best choice.

Another common approach is to accept reduced resolution, shoot at smaller apertures, and get more depth of field in a single shot. For this, you can use lenses with inherently smaller apertures, or stop down lenses with large apertures.

So, more information, please! :)

--Chris S.

bokemon
Posts: 20
Joined: Sun Jan 16, 2022 6:06 pm
Location: Silicon Valley, CA

Re: Help with setting up a "machine vision" microscope system

Post by bokemon »

Thanks for the replies so far.
1) I should look into zoom lens more. I assume all they do is rescale the "data" coming out of the objective, so it can't increase resolution or increase the FOV over the original.
2) wait, I thought the magnification was changed by moving the tube lens closer to the sensor. And the extra stuff gets added btw the objective and tube lens, which is in the "infinity space".
2.5) Well, in any case the point was that this objective can give a 30mm image circle, but it would be too big to fit on a smaller 4/3" sensor, so why not scale the image down somehow?
3) yes, I need color sensor
4) I probably don't need darkfield - better ask the other team members
5) In terms of speed, I'd like approx 30 fps over the whole sensor area, but this is including skipping pixels or binning, so shouldn't be much of an issue. But for small regions, e.g. 800 x 600, then I want high speeds like 60-100 fps.
6) I'm fine with a small DOF, and in fact this is a feature. Partially to check for sample flatness / tilt, and secondly, the plan is to have an encoder on the focus rail stage and use the focus to roughly determine the position / depth of the sample.

Just for reference, I already have a camera called the ZWO ASI294MC which is an astronomy camera with a 4/3" sized sony sensor. I use it for planetary imaging (where you need high framerates over a small area) and also Electronically Assisted Astronomy, where it just sits there stacking full-res images. It is totally capable of the speeds I mentioned, and so I assume many of these other "sensor with a USB 3 port" cameras are also.

Scarodactyl
Posts: 1617
Joined: Sat Apr 14, 2018 10:26 am

Re: Help with setting up a "machine vision" microscope system

Post by Scarodactyl »

If you move the tube lens closer to the sensor it will no longer be at infinity focus. If you want to resize the image you use a different focal length--a shorter fl tube lens will result in lower magnification, which would be good for a smaller sensor.

bokemon
Posts: 20
Joined: Sun Jan 16, 2022 6:06 pm
Location: Silicon Valley, CA

Re: Help with setting up a "machine vision" microscope system

Post by bokemon »

I guess I'm not understanding something here.
In Robert's web page, he has a setup where the ITL200 is moved closer to the sensor for 4.3x gain. Can I not use a beamsplitter in this setup?
https://www.closeuphotography.com/thorlabs-itl200/

Well, if this won't work, is there a recommended tube lens setup to make 3-4x mag out of a 5x mag objective?

Scarodactyl
Posts: 1617
Joined: Sat Apr 14, 2018 10:26 am

Re: Help with setting up a "machine vision" microscope system

Post by Scarodactyl »

Yeah, that's a special case which you can get away with on the 5x, but would probably make using a beamsplitter nonviable since there is no infinity space.
I could tell you how to do it affordably with used components but doing it well with new components from usual outlets could be a lot more expensive.

bokemon
Posts: 20
Joined: Sun Jan 16, 2022 6:06 pm
Location: Silicon Valley, CA

Re: Help with setting up a "machine vision" microscope system

Post by bokemon »

Still confused.
There is 75mm of blank space between the objective and tube lens in his setup. Isn't that the "infinity space"?

Macro_Cosmos
Posts: 1511
Joined: Mon Jan 15, 2018 9:23 pm
Contact:

Re: Help with setting up a "machine vision" microscope system

Post by Macro_Cosmos »

bokemon wrote:
Mon Jan 17, 2022 3:12 am
2) wait, I thought the magnification was changed by moving the tube lens closer to the sensor. And the extra stuff gets added btw the objective and tube lens, which is in the "infinity space".
2.5) Well, in any case the point was that this objective can give a 30mm image circle, but it would be too big to fit on a smaller 4/3" sensor, so why not scale the image down somehow?
3) yes, I need color sensor
4) I probably don't need darkfield - better ask the other team members
5) In terms of speed, I'd like approx 30 fps over the whole sensor area, but this is including skipping pixels or binning, so shouldn't be much of an issue. But for small regions, e.g. 800 x 600, then I want high speeds like 60-100 fps.
6) I'm fine with a small DOF, and in fact this is a feature. Partially to check for sample flatness / tilt, and secondly, the plan is to have an encoder on the focus rail stage and use the focus to roughly determine the position / depth of the sample.

Just for reference, I already have a camera called the ZWO ASI294MC which is an astronomy camera with a 4/3" sized sony sensor. I use it for planetary imaging (where you need high framerates over a small area) and also Electronically Assisted Astronomy, where it just sits there stacking full-res images. It is totally capable of the speeds I mentioned, and so I assume many of these other "sensor with a USB 3 port" cameras are also.
2) That is absolutely not the case. The tube lens is focused to infinity. If you "move" it, is it still focused to infinity? Your case is vastly different to what most of us do here. If the tube lens is not focused to infinity, adding in a beamsplitter and other optics will induce unimaginable amounts of spherical aberration. I have tried that myself.
Scaling can be done with a shorter focal length. With a 100mm tube lens, the 5x become a 2.5x, likewise a 400mm tube lens turns it into a 10x. Numerical aperture (NA) stays the same. You will not lose image quality assuming the tube lenses all perform well.
3) What kind of sensor? CMOS or CCD?
5) With a CCD, you can perform true pixel binning to increase readout speeds. With CMOS, pixel binning is either additive or the median is taken, this is done after a full area scan, this does not increase readout speed.

A region of interest (ROI) of 800x600 at 100 frames could be challenging depending on your USB 3.x camera model. You might need to decrease the bit depth as well, which does not seem important to your application as dynamic range does not seem like a concern.

30 FPS full sensor readout over USB3.x is challenging. The readout speed of a 2300x2300 sensor at full well capacity of 16-bit over USB 3.x is around 23 FPS. That is 5.3 megapixels. Some camera models offer faster scanning options which sacrifice readout noise. If you are using a consumer camera, this might be impossible as most are megapixel monsters. You can see that from the specifications of the ZWO camera -- only 19 FPS in 10-bit readout mode. Readout is effectively bottlenecked by the transfer speeds of USB3.x, you will be better off with a capture card (CoaXPress).

Another quick and easy alternative to an encoder is a gauge indicator.
bokemon wrote:
Mon Jan 17, 2022 10:44 am
Well, if this won't work, is there a recommended tube lens setup to make 3-4x mag out of a 5x mag objective?
You can use a zoom lens. The ideal zoom lens maintains infinity focus regardless of focal length. I am sure someone can recommend a good one that does that.

Macro_Cosmos
Posts: 1511
Joined: Mon Jan 15, 2018 9:23 pm
Contact:

Re: Help with setting up a "machine vision" microscope system

Post by Macro_Cosmos »

bokemon wrote:
Mon Jan 17, 2022 3:40 pm
Still confused.
There is 75mm of blank space between the objective and tube lens in his setup. Isn't that the "infinity space"?
No, infinity space in infinity conjugate systems is characterised by parallel light beams between the objective and the tube lens. When the tube lens is no longer focused to infinity, there is no more infinity space as the "beams" are now converging. Sticking a piece of glass in will bend the light rays further due to different refractive indices (1.4 to <2.0 in glass VS 1.0003... for air). This will induce spherical aberration.

bokemon
Posts: 20
Joined: Sun Jan 16, 2022 6:06 pm
Location: Silicon Valley, CA

Re: Help with setting up a "machine vision" microscope system

Post by bokemon »

ok, well I'm starting to dimly become aware of what's going on. Lemme know if I got this right:
If I put a sample at the objective's working distance, the objective will shoot parallel rays out the back, then go thru the tube lens, and if that is spaced at the correct focal length to the sensor, then an image can form on the sensor.
Case 2 is if the tube lens is too close to the sensor. In order to get that to focus, the tube lens' input needs to be a slightly converging beam. To accomplish this, you move the sample further away from the objective to "working distance + extra". This setup can still work, but you don't have that "infinity space" behind the objective any more.
Is this right? I dunno, that's what Robert seems to be doing on his website and getting different magnifications out of his 5x objectives.

Semi-related question: The Thor labs TTL200 has a spec for "field size at image plane" of 22mm. Yet people are able to image past the corners of an APS-C sensor. Why?
It looks like for good image quality plus infinity space at reduced magnification I could go with the Raynox 250 or the Thorlabs TTL 165?

lothman
Posts: 959
Joined: Sat Feb 14, 2009 7:00 am
Location: Stuttgart/Germany

Re: Help with setting up a "machine vision" microscope system

Post by lothman »

bokemon wrote:
Tue Jan 18, 2022 9:11 am
Semi-related question: The Thor labs TTL200 has a spec for "field size at image plane" of 22mm. Yet people are able to image past the corners of an APS-C sensor. Why?
It looks like for good image quality plus infinity space at reduced magnification I could go with the Raynox 250 or the Thorlabs TTL 165?
with a Raynox 150 tube lens (210mm focal length) the Mitutoyos keep their magnification and project on a full frame sensor (24x36 -> 43mm image circle). What Robert and other do on camera with a smaller sensor, is to use another tube lens with a shorter focal length like the Raynox 250 (109mm focal length). This shrinks down image circle on the sensor and reduces the magnification on the sensor but maintains or even increases the high resolution image information of the Mitutoyo on a smaller sensor.

rjlittlefield
Site Admin
Posts: 23562
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Re: Help with setting up a "machine vision" microscope system

Post by rjlittlefield »

bokemon wrote:
Tue Jan 18, 2022 9:11 am
Case 2 is if the tube lens is too close to the sensor. In order to get that to focus, the tube lens' input needs to be a slightly converging beam. To accomplish this, you move the sample further away from the objective to "working distance + extra". This setup can still work, but you don't have that "infinity space" behind the objective any more.
Is this right?
Yes, that is correct.
I dunno, that's what Robert seems to be doing on his website and getting different magnifications out of his 5x objectives.
Also correct. Reducing the extension while using the same tube lens turns the infinity section into a converging section. That change introduces some spherical aberration, but in Robert's specific situation the amount of added SA is small enough to not be a problem. So, he's sort of "breaking the rules but getting away with it". The concern for you is that changing the situation also changes the amount of added aberration -- in particular adding glass into a converging section is asking for trouble. The safe approach is to use a tube lens with shorter FL that preserves the infinity section.

--Rik

RobertOToole
Posts: 2627
Joined: Thu Jan 17, 2013 9:34 pm
Location: United States
Contact:

Re: Help with setting up a "machine vision" microscope system

Post by RobertOToole »

rjlittlefield wrote:
Tue Jan 18, 2022 3:20 pm
I dunno, that's what Robert seems to be doing on his website and getting different magnifications out of his 5x objectives.
Also correct. Reducing the extension while using the same tube lens turns the infinity section into a converging section. That change introduces some spherical aberration, but in Robert's specific situation the amount of added SA is small enough to not be a problem. So, he's sort of "breaking the rules but getting away with it". The concern for you is that changing the situation also changes the amount of added aberration -- in particular adding glass into a converging section is asking for trouble. The safe approach is to use a tube lens with shorter FL that preserves the infinity section.
Quick note about short focus.

I agree it is best to use a shorter FL lens focused at infinity rather than short focus. With short focus, in all the cases I remember at least, the image circle was reduced. Also I don't think I have had a really good result with a full frame camera using a tube lens set up with short focus.

I should mention that originally I found good IQ results with short focus testing tube lenses at different focus points, "past" infinity, infinity, 100 yards, 100 feet. Multiple focus point distance testing took way too much time and I haven't tested lenses like that since.

Best,

Robert

Macro_Cosmos
Posts: 1511
Joined: Mon Jan 15, 2018 9:23 pm
Contact:

Re: Help with setting up a "machine vision" microscope system

Post by Macro_Cosmos »

bokemon wrote:
Tue Jan 18, 2022 9:11 am
Semi-related question: The Thor labs TTL200 has a spec for "field size at image plane" of 22mm. Yet people are able to image past the corners of an APS-C sensor. Why?
It looks like for good image quality plus infinity space at reduced magnification I could go with the Raynox 250 or the Thorlabs TTL 165?
22mm is the "acceptable projection circle", it covers full-frame and goes a little beyond with further degradation in the corners.
It is matched to objectives which we consider "wide field of view" in the industry, going even wider, there are 25mm, 26.5mm, 28mm and 30mm (and a couple that is even wider for equally wide wallets).

With the Raynox 250 which I believe is around 109mm, you will get a bit more than 2.5x.

Stay away from the TTL165-A, this is designed specifically for Zeiss objectives and comes with its own correction. The TTL100-A could work. The TTL180-A is overpriced, it is simply a rehoused U-TLU which can be found on eBay -- sometimes new in the box for $150. Olympus infinity corrected objectives do not require any compensation on the eyepiece and/or tube lens' side. I sold several U-TLUs for $109 a while back. Get the U-SWTLU-C which has a field number of 26.5mm, larger than 22mm, and cheaper! For your application, an achromatic doublet focused to infinity might suffice. You can get a handful of achromatic doublets of different focal lengths for the price of one tube lens, you will need some kind of housing. As usual, I recommend SM2, expensive but you get what you paid for.

Post Reply Previous topicNext topic