From constantly running production lines in the automotive industry to quick and precise bin picking: lots of applications require high-quality 3D images and benefit from pre-processed data.
When 3D image processing is used in compute-intensive applications, interfaces and CPU power quickly become a bottleneck. In these cases it is very convenient if the 3D camera already does some of the computing itself.
Addressing this issue requires a reduction in data rates and the performance requirements of system components without adversely affecting data quality. Systems should also be space-saving and efficient. Embedded 3D cameras from the Ensenso XR series with integrated data processing present an ideal option in this scenario.
Many machine vision applications with 3D cameras function according to the principle of spatial vision (stereovision). In these applications, camera images are processed with high resolution and frame rates in order to ensure that results data is rapidly available to further processes. The calculation of three-dimensional data (point clouds) from the image provided by the stereo cameras requires a number of sophisticated process steps.
Ensenso XR: Powerful 3D cameras with integrated data processing. Image Credit: IDS Imaging Development Systems GmbH
These steps were previously performed by powerful industrial PCs (IPCs), but increasing speed and quality demands surrounding results data have been contemporary 3D stereo cameras (such as the Ensenso X series) employ high-resolution 2D cameras with Gigabit Ethernet interfaces.
Effectively transmitting 2D output data to the processing IPC involves optimizing network bandwidth use in order to avoid data loss or time delays. The processing power of the IPC hardware must also consistently increase so as to avoid restricting the overall system.
The performance of 3D camera systems can be further enhanced through the use of high-quality components. Exchangeable 2D cameras and the flexible design of the Ensenso X series allow these cameras to work with a range of data interfaces and sensor resolutions, continuing to grow in line with object size, speed and quality requirements.
Some applications are unable to justify the expense and space requirements of high-resolution, fast GigE cameras, high-performance network technology, specially shielded cables, and powerful PC hardware. Ensenso is pursuing a different approach with its new XR camera series, specifically aimed at these applications.
A key principle of the Internet of Things (IoT) is that each individual component in a distributed system should perform a specific task, producing results that can be utilized directly by other systems. For example, a 3D camera produces three-dimensional coordinates of pixels of a real object.
Comparison of 3D processes. Image Credit: IDS Imaging Development Systems GmbH
Onboard Ensenso 3D Processing
An SoC (System-on-Chip) is integrated into the Ensenso XR projector unit. This powerful feature allows the camera to execute 3D processes itself, including stereo analysis.
Once lens distortion has been corrected, 2D output images are transferred to an axis-parallel stereo system via a virtual camera rotation (rectification). This makes all subsequent analyses much simpler.
Highly optimized matching algorithms designed to accommodate static or moving scenes are used to search the recorded image pairs for corresponding pixels. The different perspectives of the cameras mean that these pixels are subject to different horizontal shifts in the image plane. This is referred to as disparity.
Geometric relationships in the parallel stereo system result in this disparity representing a measure of the spatial depth of a 3D point in millimeters. This takes place after the application of beam sets and known system parameters, for example, pixel sizes, focal lengths and the base length of the stereo system.
These pixel operations are time- and computation-intensive, requiring them to be performed in parallel by a supporting FPGA in the camera.
This enables the realization of a 3D data rate that is comparable to that of an Ensenso X system performing stereo analysis on a desktop PC equipped with an Intel Core i7 Quad CPU.
By combining integrated data processing with FlexView2 technology, accurate and fast acquisition of image details goes hand in hand.
Martin Hennemann, Product Manager and 3D Image Processing Specialist, IDS
Benefits of Embedded Functionality
Working alongside the FlexView2 technology, Ensenso XR36 series models can process up to 16 fast sequential images for the 3D dataset of a static scene. This takes place without the additional delay of transferring raw data to the host PC.
The displacement of the projector pattern by FlexView2 facilitates different 3D point results with each image pair, ultimately contributing to an extremely high-resolution 3D representation.
Because computation-intensive processes have been relocated into the camera, there is no more need to employ powerful industrial PCs. Network load is also reduced, thanks to the transfer of 3D result data rather than high-resolution 2D raw data.
Rapid, direct memory access between image acquisition and processing for high-resolution 3D data offers substantial value in terms of bandwidth reduction and result rate when compared to the use of an industrial PC for external processing.
In particular, multi-camera systems will benefit from the new Ensenso XR series’ resource-saving features. Transporting raw data from a number of high-resolution 2D cameras over a network is likely to result in bandwidth bottlenecks, causing frame rate drops that have will adversely affect overall performance.
Early evaluation and simultaneous data reduction of the XR series is an ideal solution for multi-camera systems, offering reliable result rates, reduced computing power of peripheral components and reduced space requirements. It is much easier to scale a 3D application to meet requirements when using Ensenso XR cameras.
The camera will only transmit the disparity map in order to further reduce the data rate. The 16-bit 1-channel image is notably smaller than a full point cloud (a 32-bit RDB image with color overlay). This simple conversion can be performed by the Ensenso SDK with very little computing load on the industrial PC.
Comparison of 3D multi-camera systems. Image Credit: IDS Imaging Development Systems GmbH
New Independence for 3D Vision Applications
The Ensenso XR stereo camera offers increased autonomy when compared to other 3D cameras. This is an important factor for 3D applications in terms of speed, performance requirements on network peripherals and IPC hardware.
The use of Ensenso XR cameras can simplify the whole construction of a 3D application while simultaneously reducing costs, particularly in multi-camera systems.
The new Ensenso XR projector unit also comes equipped with an integrated front light. This light supports working environment calibration, improving 2D camera image quality if no external lighting is available or the ambient light is insufficient.
As well as the existing GigE data connection, an additional WiFi interface is planned in the future. This will be very useful for temporary access to data and parameters in settings where wiring may be difficult or expensive.
Ongoing improvements to the Ensenso XR series have opened up new application areas for 3D camera technology. Integrated data processing is, therefore, the next logical step in effectively optimizing data exchange. You may use the Ensenso Selector on the IDS website to find the right 3D camera for your application.
This information has been sourced, reviewed and adapted from materials provided by IDS Imaging Development Systems GmbH.
For more information on this source, please visit IDS Imaging Development Systems GmbH.