The Future of 3D Machine Vision: The Photometric Stereo Technique

Making use of 3D surface orientation, particularly the effect it has on reflected light, photometric stereo for industrial applications produces a contrast image which accentuates local 3D surface variations.

Courtesy of specialized new algorithms, increasing awareness of the necessity of good lighting to produce machine vision success, and low-cost multi-light solutions, this technique is gathering increasing attention.

The photometric stereo technique can show surface defects on textured surfaces such as synthetic leather.

The photometric stereo technique can show surface defects on textured surfaces such as synthetic leather.

Real-world objects have three dimensions: height, width, and depth. In order for automated systems like robots to operate successfully, they need to be able to “see” in these three dimensions. They are given this “sight’ by machine vision systems, which comprise of a camera, lighting, and a PC for image processing.

However, reducing the amount of data which must be processed in order to locate and analyze an object correctly constitutes one of the most significant challenges facing the machine vision industry.

In order to shrink the data, machine vision designers would use filters, lights, and black-and-white cameras. These address color machine vision applications. The grayscale images which are produced as a result can be processed more quickly as they contain less data.

In a similar vein, engineers would develop motion control systems and mechanical fixtures in order to solve a traditional 3D application using a 2D machine vision solution.

Designers are given greater processing power with today’s microprocessors, field programmable gate arrays (FPGAs), and graphic processor units (GPUs). However, processing power is still finite. The most cost-effective solution for 3D applications may be provided by a nascent machine vision technique known has ‘photometric stereo’.

3D Vision at a Glance

The need to reduce the amount of data needed in color and 3D applications has been eased by affordable processing power.

An example is provided by integrated laser triangulation systems for conveyor-based 3D systems, which have been facilitated by cheap data processing, lasers, and optics. These systems are able to generate tens of thousands of 2D profiles every second in the process of creating a 3D object map.

Another option is provided by new time-of-flight cameras. These provide low-resolution 3D maps for a variety of applications, without the safety risks of laser illumination.

For 3D projects with a larger area, multiple pictures can be taken of the same object from different locations by mounting single camera photogrammetric systems on the end of a robot.

Using these images, the 3D position of every pixel in the image can be calculated based on a predetermined geometric relationship between the camera and the object. In regards to large-area 3D inspections, two cameras are aligned side-by-side in order to mimic human eyes and capture 3D information.

Yet, in order to inspect objects without a large field of view at high-speed, qualitative data is potentially very useful, whereas quantitative 3D data isn’t always necessary for measurement purposes. This is where the photometric stereo technique intervenes.

Photometric Stereo Advantages

Measuring the height of any given pixel is not the primary concern of photometric stereo. Instead, this technique produces a contrast image accentuating local 3D surface variations by using 3D surface orientation and its effect on reflected light. The variations shown are potentially invisible when using traditional 2D imaging.

When using photometric stereo solutions, it is not necessary to know the exact 3D relationship between the object tested and the camera, nor is it necessary to use two cameras to capture 3D data. Rather, a single camera with multiple illumination sources is used.

By observing an object under different lighting conditions, its surface is estimated during the photometric stereo technique. The basis of this method is the observation that the amount of light a surface reflects depends upon the surface’s orientation in relation to the light source and the observer.

Courtesy of new specialized algorithms, an increasing awareness of the need for good lighting to ensure machine vision success, and low-cost multi-light solutions such as Smart Vision Lights’ LED Light Manager (LLM) (which allows four lights to be controlled via a simple browser-based interface at a lower cost than a frame grabber or smart camera break-out box), photometric stereo’s use in industrial applications is gathering increasing attention.

At present, photometric stereo applications’ unique benefits are enabling numerous common industrial inspection applications which were previously difficult, or impossible, to solve.

Application: Clips and Tires

Machine vision systems have always had issues reading raised letters on parts. This example shows a plastic connector with numerous functional surface features, as well as a directional symbol and the number two. There is no contrast, because there is not any difference between the raised letter and material of which the clip is comprised.

Manufacturers have used laser triangular systems on larger objects, like tires, in order to create a 3D surface map. These laser scanning systems are often a complex and costly solution for 3D measurements, even if they have become much more integrated and effective recently.

Figure 1 (Photo courtesy of Matrox Imaging)

Figure 2 (Photo courtesy of Matrox Imaging)

Figure 3 (Photo courtesy of Matrox Imaging)

Figure 4 (Photo courtesy of Matrox Imaging)

In these photos (figures 1-4), Smart Vision Lights’ linear miniature (LM) LED lights are positioned at 90-, 180-, 270-, and 360-degrees around the tire’s perimeter in order to illuminate the black plastic clip. They are controlled by an LLM. As each exposure is triggered by the Matrox camera, a light is triggered from a different direction by the LLM.

Figure 5 (Photo courtesy of Matrox Imaging)

Each image is fed by the camera into a PC running an image library photometric stereo registration algorithm. This combines all corresponding pixels, establishing local surface properties and producing one or more types of composite images from these. Examples can include a contrast image of the local 3D geometries or an albedo image (figure 5).

More is revealed by these composite images than by any of the constituent images alone. The edges forming the black-on-black lettering on the surface of the clips is clearly shown in the resulting composition. Additionally, the edges of the various injection molded parts which comprise the whole are shown.

Application: Synthetic Leather Perforations

In the next example, four more pictures of a synthetic leather material are shown (figures 6-9). Leatherette, similarly to the organic material which it mimics, has considerable surface texture. It is almost impossible for the human eye to visualize 100% surface texture over the full image, let alone a computer.

Figure 6 (Photo courtesy of Matrox Imaging)

Figure 7 (Photo courtesy of Matrox Imaging)

Figure 8 (Photo courtesy of Matrox Imaging)

Figure 9 (Photo courtesy of Matrox Imaging)

In each constituent image, strong shadows are created by the warp of the material while it lies on a supporting substrate, while, as a result of the strong light reflection, other parts of the image tend toward saturation.

Figure 10 (Photo courtesy of Matrox Imaging)

The photometric stereo registration algorithm (figure 10) produces a final composition which shows a texture which is evenly illuminated over the full field of the camera’s view, with sharp contrast across each crevice and the highlighting of holes.

It is also possible to use the photometric stereo technique on pores on metal machined surfaces, like engine heads. Other areas which will clearly be at an advantage as a result of these cost-effective photometric stereo solutions are cast parts, laser marking, and direct part marking systems such as dot peen.

Photometric Stereo Outlook

The 3D world in which we live will continue to be dependent on 3D vision solutions. A recent forecast from DBMR Research projected growth in the global 3D machine vision market at a CAGR of 9.5% between 2017 and 2024, increasing its annual value from $15.4 billion to nearly $32 billion.

However, high installation costs and a lack of technical knowledge constitute the most significant challenges to 3D machine vision market growth. As the packaged photometric stereo registration tool and one-click programming of the LLM LED light manager show, the machine vision industry is ready for the next significant step in 3D machine vision.

This information has been sourced, reviewed and adapted from materials provided by Smart Vision Lights.

For more information on this source, please visit Smart Vision Lights.

Tell Us What You Think

Do you have a review, update or anything you would like to add to this article?

Leave your feedback