Mirroring the Image Processing Power of the Human Eye

Penn State electrical engineering researchers developed a metasurface, an optical element that resembles a glass slide and uses tiny nanostructures positioned at different angles to control light, in an attempt to mimic the simple, instantaneous image processing power of the human eye.

Image Credit: air009/Shutterstock.com

The group published their invention in Nature Communications under the direction of corresponding author Xingjie Ni, an Associate Professor of electrical engineering and computer science (EECS) at Penn State.

The researchers found that in order for artificial intelligence (AI) systems to process images and identify objects, they can be slow and require a large amount of computing power and energy. On the other hand, images can be preprocessed and altered using the metasurface before they are taken by a camera, requiring less processing power and data bandwidth from a computer and artificial intelligence. 

The way the metasurface operates is by taking an image and converting it from the Cartesian coordinate system, which uses a bullseye-like pixel distribution, to the log-polar coordinate system, where image pixels are arranged in straight rows and columns along the x and y axes.

Like the arrangement of light receptors inside the human eye, the metasurface takes images and arranges them in a log-polar coordinate system - with denser pixels for the central, focused features and sparser pixels for the peripheral regions. This allows for the more important aspects of a photo to come through clearly while others remain less in focus, thereby saving data bandwidth.

Xingjie Ni, Corresponding Author and Associate Professor, Electrical Engineering and Computer Science (EECS), The Pennsylvania State University

The image is converted from the Cartesian system into log-polar coordinates by the metasurface when it is positioned in front of a camera, allowing light to pass through it before the image is digitalized and sent to a computer. The metasurface operates at the speed of light and requires no power because it uses nanostructures to bend light. 

As an image of an object can vary in size or orientation, it is desirable to preprocess images to make them resistant to scale and rotation changes. This preprocessing helps AI applications more easily recognize them as the same object.”

Xingjie Ni, Corresponding Author and Associate Professor, Electrical Engineering and Computer Science (EECS), The Pennsylvania State University

Researchers are also able to convert the log-polar image back into the original image with Cartesian coordinates by putting an alternative metasurface in front of a camera. 

The researchers claimed that there are numerous possible uses for the invention, such as target tracking and surveillance to map a car's movement through a city. 

A metasurface can be used in tandem with AI systems as a preprocessor, making it easier to recognize the same car from multiple street view cameras. Or if it is applied to a satellite, it could potentially track planes from takeoff to landing.” 

Xingjie Ni, Corresponding Author and Associate Professor, Electrical Engineering and Computer Science (EECS), The Pennsylvania State University

Zingwang Zhang, a former Postdoctoral Scholar in EECS; Ziaojie Zhang, a Graduate Student in EECS at the time of research; Yao Duan, a Penn State doctorate holder in EECS; and Lidan Zhang, also a Graduate Student in EECS, round out the co-authors in addition to Ni.

This work was supported by the National Science Foundation of the United States, NASA, the Office of Naval Research, the National Eye Institute of the National Institutes of Health, and the Gordon and Betty Moore Foundation. 

Journal Reference:

Zhang, X., et.al (2023) All-optical geometric image transformations enabled by ultrathin metasurfaces. Nature Communications. doi.org/10.1038/s41467-023-43981-x

Tell Us What You Think

Do you have a review, update or anything you would like to add to this news story?

Leave your feedback
Your comment type
Submit

While we only use edited and approved content for Azthena answers, it may on occasions provide incorrect responses. Please confirm any data provided with the related suppliers or authors. We do not provide medical advice, if you search for medical information you must always consult a medical professional before acting on any information provided.

Your questions, but not your email details will be shared with OpenAI and retained for 30 days in accordance with their privacy principles.

Please do not ask questions that use sensitive or confidential information.

Read the full Terms & Conditions.