Autonomous Systems in the Internet of Vehicles
by Balamurugan Balusamy, Sandeep Kumar Mathivanan, Prabhu Jayagopal, S. K. B. Sangeetha, Ali Kashif Bashir
8Hybrid Multimodal Fusion Network (HMFNet) for Enhanced Perception in Autonomous Vehicles
Mahalakshmi1*, Ranjini K. S.2, Nidhi S. Vaishnaw3 and Jesla Joseph4
1Department of Mathematics, Nitte Meenakshi Institute of Technology, Bengaluru, Karnataka, India
2Department of Computer Applications, Dayanand Sagar College of Arts, Science and Commerce, Bengaluru, Karnataka, India
3Department of Mathematics, CMR Institute of Technology, Bengaluru, Karnataka, India
4School of Computer Science and Applications, REVA University, Bengaluru, Karnataka, India
Abstract
Multisensor image fusion is an essential component of autonomous vehicle perception systems, giving better object detection, localization, and scene understanding through the integration of multiple sensor information such as cameras, LiDAR, radar, and infrared. Classical fusion methods lack real-time performance, sensor alignment, and strength under adverse environments. To address these problems, we present a hybrid multimodal fusion network (HMFNet) that applies hierarchical fusion by using deep learning and fusing convolutional neural networks for feature extraction with transformer-based attention mechanisms for dynamic multimodal sensor input weighting. The framework integrates RGB, depth, and thermal information at the feature level as well as at the decision level to improve perception accuracy under poor visibility conditions such as fog, rain, and nighttime driving. Experimental evaluation, conducted on benchmarking ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access