Accuracy of treatment and diagnosis have been significantly improved due to the use of cameras in the medical device industry. Camera-enabled medical devices such as automated blood analyzers, digital microscopes, retina scanners, and skin scanners have also immensely decreased the medical professionals’ workload and improved the entire patient experience.
The progress made in embedded vision and camera technology has played a vital role in this shift with new kinds of cameras emerging that provide several modern medical applications. Among such camera innovations, RGB-IR cameras have been a game-changer due to their potential to capture infrared (IR) and visible light data with the help of the same camera in the absence of any color corruption.
This article will look at how RGB-IR cameras improve medical device performance by capturing images that fulfill the strict image quality requirements of modern-day medical procedures.
RGB-IR Cameras and their Growing Popularity in the Medical Industry
Before observing how RGB-IR cameras positively affect medical and life science applications, it is important to understand what they are and how they work.
Camera modules are typically available with sensors that have the potential to capture both IR and visible light in the same frame. This resulted in a decreased image quality due to the phenomenon known as color corruption. This happens when the IR data interpolates with the color (or RGB) data, and the color fidelity of the output image is impacted.
Two methods have been traditionally utilized to overcome this challenge. These are:
- Using two cameras to capture RGB and IR data.
- Utilizing an IR cut filter during the daytime to prevent the IR light from falling on the sensor.
In the former’s case, the system turns out to be heavier and clumsy to handle. However, the IR-cut filter eliminated this difficulty.
The IR cut filter has been embedded in the lens and functions using a mechanical switch. This leads to wear and tear, considerably decreasing the camera system's lifetime.
Technology
RGB-IR cameras are fitted with sensors that have dedicated IR and RGB pixels. In contrast to ordinary camera sensors that consist of the Bayer BGGR pattern, RGB-IR cameras utilize a different type of color filter array (CFA) that have an extra set of pixels, enabling only IR light to pass through them.
Given below is the CFA arrangement of an RGB-IR sensor:
Figure 1. Bayer pattern. Image Credit: e-con Systems, Inc.
Figure 2. RGB-IR filter pattern. Image Credit: e-con Systems, Inc.
Using this new technology, RGB and IR data obtained from the camera output could be separated without any mix-up. This is extremely useful in several medical applications such as fluorescence-guided surgery, ophthalmology, and remote patient monitoring.
Few of the key benefits RGB-IR cameras offer are:
- Availability of high imaging performance in day and night conditions for smooth 24x7 image and video capture
- Committed IR channel that helps quantify the amount of IR light in the RGB component for intelligent color correction
- A smaller form factor as users do not have to utilize two separate cameras to capture RGB and IR data
- Greater camera lifetime with non-mechanical switching between IR and visible imaging, resulting in a longer lifetime for the medical device combined with the camera.
Designing an Embedded Camera for RGB-IR Imaging
The camera system must be carefully developed as RGB IR imaging needs a special CFA. This would mean that several camera components such as the optics, sensor, and ISP need to come with the potential to receive and process images that are appropriate for enabling RGB-IR imaging.
This section discusses the main parameters to consider while assessing a camera for RGB-IR imaging in terms of its component features.
Sensor
The camera sensor should be produced with a pixel array that consists of RGB and IR pixels. Several sensor manufacturers such as onsemi and OmniVision provide RGB IR sensors with devoted IR and RGB pixels.
Given below is the sensor’s pixel pattern with RGB-IR capabilities:
Figure 3. Pixel array of an RGB-IR sensor. Image Credit: e-con Systems, Inc.
Optics
Most off-the-shelf lenses that are commercially available in the market come with an IR-cut filter to prevent IR light from penetrating during the daytime. This is because the bulk still utilizes the mechanical switch-based camera system to shift between day and night operation modes.
However, to make RGB-IR imaging possible, a lens with a dual bandpass filter has to be utilized. This will guarantee that light in both the visible and NIR spectrum drops on the sensor.
Given below is the transmission diagram of what a dual bandpass filter would look like:
Figure 4. Transmission diagram of a dual bandpass filter. Image Credit: e-con Systems, Inc.
ISP
As far as an RGB-IR camera is concerned, the image signal processor (ISP) is one of the most significant components. This is because isolating the RGB and IR frames from the data provided by the sensor needs broad ISP-side expertise, and
e-con Systems does this by utilizing its proprietary algorithm. The company's engineering team influenced its ISP expertise to develop an algorithm to deduct the IR contamination on the RGB channels to provide the right colors.
One of its other differentiators is the potential to adjust the ISP to generate processed RGB and IR frames according to the host platform’s needs.
Figure 5. RGB window imaging processing. Image Credit: e-con Systems, Inc.
Figure 6. IR window imaging processing. Image Credit: e-con Systems, Inc.
Key Medical Applications Where RGB-IR Cameras Are Making a Wave of Change
Users have seen several benefits of RGB-IR cameras. This section will examine how the cameras impact a few of the most famous camera-enabled medical applications.
Remote Patient Monitoring
Remote Patient Monitoring (RPM) systems benefit the medical industry by eliminating the need for attended patient monitoring. The highly advanced RPM systems currently available use artificial intelligence to automatically detect patient falls and movements. These life-changing devices use cameras to capture images and videos, making all of the above possible.
One of the main characteristics of a remote patient monitoring system is that it has to function smoothly 24 hours a day to guarantee the patient’s continuous monitoring.
RGB-IR cameras allow the RPM device to capture images during the night using IR imaging without needing two isolated cameras or a mechanical switch to operate an IR cut filter.
Figure 7. (Left) RGB image during daytime and (Right) IR image during night-time. Image Credit: e-con Systems, Inc.
Image Guided Surgery
Fluorescence-guided surgical procedures such as indocyanine green (ICG) surgery are employed in colorectal surgeries for verifying vascular perfusion of anastomotic ends, laparoscopic cholecystectomy, evaluating bowel viability in hernia cases, and mapping sentinel lymph node (source: sciencedirect.com).
ICG surgery functions based on the principle that the dye can turn out to be fluorescent when vulnerable to a wavelength of 750 to 800 nm. The camera must capture images in the near-infrared (NIR) spectrum.
Can an RGB camera be employed to observe the organs when it is still being used in an IR light source? The difficulty here is low spectral resolution. Cancer detection can be used as a sample application to understand this further.
In this context, attempting to capture the fluorescent image utilizing an ordinary RGB camera might not give sufficient details. At the same time, the cancer detection procedure can be efficiently assisted by utilizing the IR capabilities of an RGB-IR camera. In contrast, the RGB data analyzes abnormalities detected with the help of an IR output. This can be carried out with the help of a single-camera system.
Figure 8. (Left) RGB image during ICG Surgery and (Right) IR image during ICG Surgery. Image Credit: e-con Systems, Inc.
Ophthalmology
In ophthalmology devices such as retina scanners and fundus cameras, the precision of diagnosis has been heavily identified by the quality of images supplied by the camera modules within them.
IR imaging is required for detecting sub-retinal characteristics as the deposits show up bright and thickened in comparison to the retinal vessels, optic nerve heads, and choroidal vessels that are present in darker regions. Visible light imaging is needed at the same time to guarantee that the camera captures the essential eye details.
This needs a single camera that is available with both IR and visible imaging capabilities. Also, that is why RGB-IR cameras are the perfect fit for such use cases. With such new-age fitted cameras, doctors and clinicians can efficiently diagnose and treat ocular disorders with the advantage of getting both IR and visible images from the same camera.
Conclusion
RGB-IR cameras are available with many benefits and are ideally suited for several niches of medical applications that need concurrent IR and RGB imaging. Specialized NIR cameras are ideal for applications requiring extremely high NIR sensitivity and performance. Imaging experts at e-con Systems are available to offer advice before users continue with medical device camera integration.
This information has been sourced, reviewed and adapted from materials provided by e-con Systems, Inc.
For more information on this source, please visit e-con Systems, Inc.