Crashes involving pedestrians are a significant traffic safety hazard. Detection of pedestrians is a critical element of several intelligent systems that can be handled with multimodal fusion technology.
Study: Pedestrian detection based on light perception fusion of visible and thermal images. Image Credit: Varavin88/Shutterstock.com
According to a research paper published in Optics and Laser Technology, a team of researchers has suggested a unique visual and thermal imaging fusion technique based on light perception to tackle the inexplicable difficulty of neural network fusion techniques. The scientists designed a light perception module to detect the lighting conditions and guide the fusion process. The experimental findings demonstrated that their suggested technique can surpass the state-of-the-art methods with a reduced error rate and high generalizability.
Importance of Pedestrian Detection System
It is anticipated that 70% of the world's population will reside in cities by 2050 due to urbanization. Consequently, the traffic situation in metropolitan areas will worsen, and it is realistic to anticipate that pedestrians will be at greater risk in the future. Autonomous cars have gained widespread acceptance to enhance traffic safety and efficiency.
Effectively detecting pedestrians on the road in various traffic settings may assist human and artificial drivers in preventing collisions or mitigating their severity. Consequently, pedestrian detection should be implemented into autonomous cars as a key function for safe driving. It is an essential issue that autonomous vehicle-focused academics and engineers should thoroughly address.
Deep Learning Algorithms for Pedestrian Detection
The current development of deep learning methods, specifically convolutional neural networks (CNN) depending on improved computing capabilities, enhances pedestrian detection accuracy. The present deep learning-based object detectors may be essentially split into three groups, (i) region-based detectors, (ii) anchor-based detection systems, and (iii) anchor-free detection systems.
The region-based detector is often termed a two-stage detector that breaks the object identification problem into two stages: proposal formulation and proposal projection.
Unlike a region-based detector, the anchor-based detection system is a one-stage detector since it does not include a second stage for proposal generation.
The anchor-based detector categorizes and regresses the embedding of pre-defined scales and dimensions. It is an enhanced one-stage solution that avoids the necessity of anchor boxes and identifies objects straight from an image to overcome the anchor shape issue of the anchor-based systems.
Novel Fusion Framework for Pedestrian Detection
This study proposes Adaptive Soft-Gated Light Perception Fusion (ASG-LPF) based on an anchor-free detection system as a novel fusion framework for pedestrian detection.
The mechanism is depicted in Figure 1. The framework comprises four components, i.e., an image capture module, a pedestrian identification module, an illumination assessment module, and an image fusion module.
Visible and thermal images gathered by the image capture module are transmitted to the pedestrian detection module. The visual signals are also sent to the illumination perception module. The results of the pedestrian detection module and the illumination assessment module are sent to the image fusion module for further processing and to receive the final detection data.
Figure 1. Schematic representation of proposed framework ASG-LPF. Image Credit: Li, G., et al (2022)
Key Features of the Proposed Framework
The most essential feature of the proposed framework is its ability to maximize the efficiency of single-source pedestrian detectors and increase their robustness in complicated situations (e.g., night conditions). In addition, it can merge the detection data from several sensors without training a new fusion system. This can increase the fusion framework's generalization ability and decrease the training cost.
Summary of Fusion Research Findings
In this study, the suggested framework ASG-LPF is compared to four cutting-edge fusion techniques (i.e., CFD, HWF, WSFD, and ACF THOG).
The findings demonstrated that the ASG-LPF proposed by the researchers had the greatest performance among the investigated approaches in the mixed all-day and overnight scenarios.
ASG-LPF surpasses HWF, CFD, WSFD, and ACF THOG during the working day by 0.23%, 14.78%, 18.90%, and 22.7%, respectively.
ASG-LPF surpasses HWF, CFD, WSFD, and ACF THOG during the night by 4.91%, 13.21%, 31.43%, and 32.55%, respectively. In conclusion, the suggested framework provides better pedestrian detection capability and is a potential option for pedestrian detecting objects in intelligent systems.
Limitations and Future Work of Fusion Techniques
In automated vehicles, cameras and LIDAR are the primary sensors. However, in this research, LIDAR is not used for pedestrian detection. Future work on this research might include implementing sophisticated fusion techniques based on camera sensors and LIDAR detectors.
Transfer learning has been a potential option for pedestrian detection in various traffic settings, which may ease the complex and expensive human annotation issue. Future fusion technologies may potentially include notions of knowledge transfer for advancement.
Conclusion
This study suggested ASG-LPF, a new framework for visual and thermal image fusion dependent on light perception for pedestrian detection.
The fusion process of ASG-LPF is adaptable to diverse driving lighting effects, unlike most prior investigations.
The proposed ASG-LPF system has four distinct components to increase detection performance. The generalizability of the proposed fusion system has been demonstrated, suggesting that ASG- LPF’s performance can be enhanced by upgrading the pedestrian detection module’s algorithms.
Quantitative and qualitative comparisons of the proposed framework with the studied approaches demonstrate that it generally delivers the best detection performance.
Reference
Li, G., Lai, W., & Qu, X. (2022). Pedestrian detection based on light perception fusion of visible and thermal images. Optics & Laser Technology, 156, 108466. https://doi.org/10.1016/j.optlastec.2022.108466
Disclaimer: The views expressed here are those of the author expressed in their private capacity and do not necessarily represent the views of AZoM.com Limited T/A AZoNetwork the owner and operator of this website. This disclaimer forms part of the Terms and conditions of use of this website.