Surface Defects Detection

[gautiercap]

Hello,

 

I’m currently working as an intern in AREVA (France) on a project that involves inspecting surfaces of nuclear plants primary components (mostly the reactor vessel and the lid) to look for defects and try to automatize the procedure. It’s currently done by watching hour long videos and the procedure, besides being both tiring and time consuming, leads to a high variability in judgments.

 

I thought about using the PTM/RTI method and my supervisor agreed to let me buy the tools required to build a dome.

 

Its diameter will be 450 mm and I’ll be using 96 white LEDs as light sources.

 

I’ve been strongly inspired by the work of G. Le Goïc and S. Samper (2011) who used the PTM technique for detection of appearance anomalies in high-value products requiring a perfect surface quality (watches, automotive, medical, automation).

 

I had some questions regarding this project and thought this forum could be of great help.

 

Firstly, I have some apprehension concerning the position of the 96 holes in the dome. I don’t really know how to go with the drilling procedure and how to be sure their orientation is rigorous. I included a scheme of said dome in the message.

 

Then, is it absolutely necessary to use reflecting spheres or can the light position coordinates be calculated using another method? I don’t really see how I could include spheres in the dome without obscuring the lowermost part of it. I’m also guessing it’s really important to make sure the inside of the dome is kept in the complete dark when all LEDs are turned off, right?

 

Lastly, the goal would be to automatize the process of finding defects. These defects are usually cracks, gutters, porosities, metal grindings, etc… I was wondering how the processed data could be treated using PTM/RTI to highlight such anomalies? Is it even possible with proper software or with different equipment?

 

Thank you so much in advance, I’ll make sure to update the post from time to time if I have further issues or if you guys are simply interested in the project.

 

Gautier

[Taylor]

Sounds like an interesting application of RTI.  I wouldn't presume to make recommendations about finding defects in nuclear containments or on metal, but your idea brings a couple of thoughts to mind:

• 96 LEDs seems more than necessary and will require a lot of data storage.  You could reasonably use 48 and have plenty of data to process an RTI.  I'd consider eliminating at least some of the high-angle light positions, which are likely to lead to specular reflections.
• Regarding the dome diameter, make sure you choose a camera and lens that will provide sufficient resolution for the kinds of defects you're trying to find.
• See other posts in this forum about calculating light positions with the dome method.  Also, see "Possible 3D Modelling Pathway for Automated change detection of chronologically separated RTIs" and "RTI Calibration with a Spatial Target."  I haven't used a dome myself, but have seen demonstrations.  If you want to use the results quantitatively and make comparisons over time, you should probably calibrate the dome using a reflective sphere to make sure your normals are accurate.  It shouldn't be too hard to attach a small sphere or two to the inside of the base of the dome (on a bracket extended a short distance from the perimeter).  Each sphere only needs to be 250 pixels in diameter, so it shouldn't interfere much with the images and would provide useful data.
• The positions of the LEDs isn't as important as knowing exactly where they are once you've placed them in the dome, and knowing which LED is lit for each image.  Some studies have shown advantages of having some irregularity in the light positions to calculate more accurate normals using the PTMFitter.  You could use any of a variety of techniques to lay out the positions on the dome, but calibrating the positions will be important after you've made the dome (reflective spheres help).
• Regarding which RTIViewer algorithms to use for finding defects, it's partly a trial-and-error method to see what works (specular enhancement and diffuse gain are frequently tried first, and I've found unsharp masking to be useful for viewing tool marks).  The false-color normal viewer is also useful.  Do you have any samples of the kinds of defects you're looking for that you can practice on (hopefully not on operating reactors)?  There are lots of options for post-processing the individual images or snapshots from the RTIViewer (various segmentation algorithms, PCA, etc.).  Remember that each individual image in an RTI sequence has to be processed identically (linear tone curve, white balance for color images, and exposure correction) for the normals to be accurate. 
• If you used a camera with a monochrome sensor, you might be able to isolate particular wavelengths that are optimized for finding certain defects, but a monochrome camera adds complexity and cost.  You can use interference filters on camera with an RGB sensor to isolate wavelengths, but you won't get as much resolution and the results are harder to use quantitatively.  Since you're using LEDs, you can select the LED wavelength that works best for the defects you're looking for.
• You don't need to block out stray ambient light completely if your LEDs are bright enough and your shutter is only open when the LEDs are lit.
• With a metallic object, you might have some trouble with specular reflections off the surface.  I understand the next version of RTIBuilder might have algorithms that are better at dealing with specular surfaces, but using lower-angle light positions will probably help.
• You might consider using UV LEDs as well as visible.  You might be able to use fluorescent dye to enhance certain defects.  Reflected UV and UV-induced visible fluorescence images would require longer exposures and might be a bit noisier than visible images.  Ambient light would be more of an issue with UV-induced visible fluorescence images. 

[gautiercap]

Taylor,

 

Thank you very much for your input on this. I must admit I’m very new to the RTI method and my limited time on the project makes it harder to digest and take all parameters into account.

 

That said, you highlighted some points that make a lot of sense and I had further questioning about the project. I’ll answer your questions as well.

 

I wasn’t the one who ordered most of the components so I’ll be using LEDs that are cold white (color temperature: 10000°K and luminous flux: 370 lumens). I’m going to follow your advice and use only 48 LEDs from angles that range from 15° to 55°.

 

I have not ordered the camera yet; I need to check with my supervisor about the price range we can afford. Would you recommend any model in particular? The dome will be about 450mm in diameter. I think we should be able to afford expensive equipment.

 

I didn’t realize the spheres could be that small. That might work then! Is that method extremely precise to find the light direction though? And will it give me sufficient information on the LEDs positions?

 

I have started experimenting with the three algorithms you mentioned (diffuse gain, specular enhancement and unsharp masking). They do seem to show great promise. I’ll definitely keep you updated on that.

 

I will have access to smaller scale models of nuclear plants components with known defects (usually cracks) created by electrical discharge machining. I firstly want to optimize the method of visualization (reduce the number of LEDs, find the right light orientations, etc...)  and then find a way to process the data in order to not only detect all defects but also possibly to proportion them. That way we could separate easily the defects that are critical from the ones that aren’t.

 

Regarding the latter issue, when using the HP Labs PTMViewer software I noticed a button named ‘Output Depth Values’ but it didn’t work. The error message read ‘Unable to open .cam file, aborting depth calculation’. That feature seemed extremely interesting especially for this project so I was also wondering if someone knew anything about this.

 

Thank you again for reading and helping me on this, I genuinely appreciate it.

 

Gautier

[Taylor]

Hi Gautier,

 

Re:  sphere diameter and accuracy, see Mark Mudge's reply to "Minimum reflective sphere diameter and HSH precision."

 

I've had limited experience using the HP Labs PTMViewer.  It has some interesting and useful features, but some of them were included for research purposes and I'm not sure what "Output depth values" does.  Maybe Tom Malzbender or other CHI folks can comment on this question. 

 

I'm afraid I can't offer much advice about cameras--I have a Panasonic GH2 micro four-thirds mirrorless camera modifed for UV-Vis-IR by removing the internal filter, but I don't know anyone else who is using this camera for RTI.  I haven't tried many cameras.  As Carla mentioned in response to another question about cameras, dpreview.com is a good place to get fairly detailed camera reviews.  Other things being equal (and they almost never are), you might consider a DSLR like the Nikon D810e, which has a 36 megapixel sensor and doesn't have an anti-aliasing filter.  It would potentially give you higher resolution and therefore more flexibility with your dome, but I haven't used one myself.  You want a camera that can shoot tethered (connected to a laptop or other computer).  Some cameras now have built-in radio triggers for the shutter, which is a plus.  You need to be able to control the shutter without touching the camera.  For studying reactor containments or other components, you'll have to consider the distance between the camera and the tethered computer.  USB cables don't work when they get too long (I think they should be less than about 3 meters).  You need something to amplify the signal if you string USB cables together. 

 

I recall someone saying they had trouble using RTIBuilder with images from a D800e (predecessor model to the D810e) because the file size produced by a 36 Mpixel sensor is so large, but I don't recall if this was confirmed; it might have been a bug in that unique setup.  Maybe Carla, Marlin, or Mark know if there are any file size limitations in RTIBuilder.  Any good camera with a good prime lens would likely produce good results, following CHI's documentation and workflow.  There are so many camera developments that I can't keep up.  It really comes down to preferences and budget.  The lens is probably more important than the camera, so you'll want a good prime lens, or two or three.  For capturing details, you probably need a prime lens with a focal length of at least 40-50 mm on a full-frame DSLR.   A 50mm or 60mm normal lens and a 90 mm macro on a full-frame DSLR would probably handle >80% of your needs, but others who know more about lenses might have more to add.  If you use a camera with a cropped sensor (i.e., smaller sensor size than a full-frame DSLR), the effective focal length of the lens is larger.  For example, a 20mm lens on my micro four-thirds GH2 is the equivalent of a 40 mm lens on a full-frame DSLR. 

 

With macro lenses you get less depth of field, so if your containment surface is curved, you might need to limit the focal length of the lens to something like 60 mm on a full-frame DSLR to keep the entire image in focus, or you'll just have to focus on smaller areas with a macro lens.  It's a trade-off you'll have to make.  You'd like to keep the aperture no smaller in diameter (no larger f/ numbers) than about f/11 to get sharper images.

 

You mentioned that one type of defect you're trying to look for is porosity.  If the pores you're looking for are very small, you might need a different sized dome and a different lens or setup to look at some finer details, depending on the resolution required.  I attended a recent workshop by Martin Jürgens on microscopic RTI at the 2+3D Photography conference at the Riksmuseum, where they used a small dome to capture RTIs using a microscope.  You can get USB microscopes that are pretty small and work quite well, I hear.

 

Taylor

[gautiercap]

Hello Taylor,

 

Thanks for these very complete precisions. Turns out we won't actually buy a camera, I will have to use some that the lab downstairs already has. I need to receive my Personal Protective Equipment before going there so I'll keep you updated on which camera we have and the kind of lense we can use.

 

The dome should be done by the middle/end of next week. Hopefully I will be able to present proper results by the middle of June. I'll make sure to share my progress with you guys.

 

To refocus my objectives on this project, I need to find detection criteria for surface defects. The dome is just the tool to help us visualize the surface. I need to find the proper parameters to vary to detect each types of defects. I'm looking for 7 types of defects:

 

• Discontinuities (cracks)
• Cavities (holes and porosity)
• Excrescences of matter (too much metal)
• Surface impurities (mostly oxides)
• Surface defects (scratches, scales, etc)
• Shape defects (mostly due to bad welding)
• Chemical defects

 

The minimal dimension we're looking for is 5µm in opening and 1 to 2mm in lenght.

 

Besides changing the light orientation and adding filters to the camera (both color and polarizing), is there anything else I should consider to highlight these defects?

 

Gautier

[samper]

I am Serge Samper,I was the research director of Gaëtan Le Goïc who has developped this technology (begining with a previous classic RTI tech) in my lab.

We have developped a scientific machine with specific calibration and hardware (I could not explain all our hardware here).

Specific algorithms have been also developped. We have much better results than those obtained by a classic PTM analysis.

Our method of decomposition of reflectance give a high level of accuracy. We are able to catch very small scratches (about 1/20 pixel).

We are still working on this topic with a team in Annecy. 

There are 2 phd on this work now (plus the one of Gaëtan). Thus the expertise is deep, and answers can be also detailled. For example, 96 led is a lot if you know exactly what to see, but if not, is it so expensive? No! Thus it is possible to decrease it but why? For what? We had all those questions and some answers.

There are lots of questions we can give answers. If the camera, the lenses... are well chosen a specific machine can be built and give good results. I could also explain how the data can be projected in an other space than visual one. It is of interest to analyse geometric slopes and curvatures for example... 

Solutions can be good or not by very small "details". 

Our actual research give some answers on discontinuities, cavities, excressences, impurities, scratches, scales and shape defects. For chemical defects, we should see (we are working on material detection).

If you want to detect, I really think we can give you good answers. If you want to measure deep cracks... there are other solutions. But perhaps would it be of interest to identify where and after to measure? In Gaetan's PhD we gave answers on this problem.

With our device, we measure 2 micron large scratches.

It would be simple to test your surfaces.

 

Serge

[Taylor]

Hi Serge,

 

It sounds like you have an interesting project and a lot of thought has gone into it.  Sorry if my attempts to respond to questions weren't helpful.  If you can share the algorithms you've developed or any of your results with this forum, I'm sure many members would be very interested to see how your system works. 

 

Could you provide a reference or link to G. Le Goïc and S. Samper (2011) mentioned by Gautier above?  I'd like to get a little more background on your work.  I found a paper that might be the one mentioned, but it's paywalled. 

 

Thanks,

Taylor

[gautiercap]

I contacted Serge since it seems like his expertise could be of a great help with my issues.

 

I'd also appreciate it a lot if you could share your work with us, and give some insights on what you've done so far on the project.

 

Thanks,

 

Gautier

[samper]

Hi Taylor,

Maurice Pillet and Hugues Favreliere are leading a team on this topic now. There are working with industrial partners who are willing to identify and characterize geometric defects on surfaces (scratchs, ...).

In http://www.symme-mesura.com you will find some (in french...) informations

If you go to "ressources", you will find publications (phd -in french...- but also journal papers in english, much better for you).

In researchgate.net there are some papers of our group such as:

 

https://www.researchgate.net/publication/272566459_Extended_visual_appearance_texture_features

https://www.researchgate.net/publication/260795860_Visual_quality_inspection_and_fine_anomalies

https://www.researchgate.net/publication/234772289_PTMs_developments_for_the_appearance_control_on_high-added_value_surfaces

 

But at this time, most of the developments have not been published.

You can find our emails on our papers if you want exchange. 

We are not involved in cultural heritage yet but it is a very interesting research domain where we could use our devices.

 

Serge