May 11 2021
Since the advent of photography in the mid-19th century, imaging technology has certainly come a long way.
Image Credit: hxdbzxy/Shutterstock.com
Today, several sophisticated cameras meant for challenging applications depend on mechanisms that are significantly different from those seen in consumer-oriented devices. One of these advanced cameras uses the so-called “single-photon imaging,” which can yield highly superior results in fast dynamic scenes and dark conditions. But it is not known how this single-photon imaging is different from traditional imaging.
When capturing an image with a standard CMOS camera, similar to the ones used on smartphones, the camera sensor is exposed to a huge influx of photons at the time of a predefined exposure time. Within the sensor grid, every pixel produces an analog value that relies on the number of photons that strike that specific pixel during the exposure.
But this form of imaging has only a few strategies to tackle the moving objects; the movement of the object needs to be relatively slow when compared to the exposure time to prevent blurring. On the other hand, single-photon cameras are capable of capturing a quick burst of successive frames with extremely short individual exposure times. Such frames are binary―a grid of 0s and 1s that individually indicate whether a single photon reached each pixel or not at the time of exposure. To rebuild a real image from these binary frames, or bit planes, several of them had to be processed into one non-binary image. To achieve this, different levels of brightness can be assigned to all the pixels in the grid based on the number of bit planes that had a “1’ for every pixel.
Apart from its higher speed, the fully digital nature of single-photon imaging helps design ingenious image reconstruction algorithms that can compensate for difficult situations or technical restrictions.
Professor Takayuki Hamamoto from the Tokyo University of Science, Japan, has been guiding a team of researchers who are focused on further improving the capabilities of single-photon imaging.
In the new study performed by Professor Hamamoto and his research team, which was published in the IEEE Access journal, an extremely effective algorithm was designed to overcome the blurring effect triggered by movements in the imaged objects and also the frequent blurring of the whole image, for example, blurring caused by shaking of camera.
The researchers’ method addresses several restrictions of present-day deblurring methods for single-photon imaging, which generate low-quality images when numerous objects in the scene are shifting at varying speeds and dynamically overlapping one another.
Rather than altering the entire image as per the predicted movement of a solo object or based on spatial regions in which the object is believed to be moving, the recommended technique uses a more versatile approach.
A motion estimation algorithm initially tracks the motion of individual pixels based on statistical assessments of how bit values tend to change over time (across different bit planes). In this manner, as experimentally shown by the investigators, the movement of individual objects can be precisely predicted.
Our tests show that the proposed motion estimation technique produced results with errors of less than one pixel, even in dark conditions with few incident photons.
Takayuki Hamamoto, Professor, Tokyo University of Science
The researchers subsequently designed a deblurring algorithm that applies the outcomes of the motion estimation step. This second algorithm groups pixels with an analogous movement together, thus identifying individual objects in each bit plane shifting at varying speeds.
This makes it possible to deblur each area of the image separately in accordance with the movements of objects passing through it. With the help of simulations, the team demonstrated that their new method created high-quality and very sharp pictures, even in low-light dynamic scenarios packed with objects coursing at different velocities.
On the whole, the study results demonstrate clearly how single-photon imaging can be considerably enhanced if one gets down to designing effective image processing methods.
Methods for obtaining crisp images in photon-limited situations would be useful in several fields, including medicine, security, and science. Our approach will hopefully lead to new technology for high-quality imaging in dark environments, like outer space, and super-slow recording that will far exceed the capabilities of today's fastest cameras.
Takayuki Hamamoto, Professor, Tokyo University of Science
Professor Hamamoto further added that even consumer-level cameras may timely gain from the advancements in single-photon imaging.
Scientists are indeed getting closer to a new age in digital photography, and research work like this is significant for paving the way toward that future.
The study was funded by the Japan Society for the Promotion of Science (JSPS) KAKENHI under Grant JP19K12025 and Grant 20K19829.
Journal Reference:
Iwabuchi, K., et al. (2021) Image Quality Improvements Based on Motion-Based Deblurring for Single-Photon Imaging. IEEE Access. doi.org/10.1109/ACCESS.2021.3059293.
Source: https://www.tus.ac.jp/en/