Autonomous Systems in the Internet of Vehicles
by Balamurugan Balusamy, Sandeep Kumar Mathivanan, Prabhu Jayagopal, S. K. B. Sangeetha, Ali Kashif Bashir
15Multimodal Vision-LiDAR Transformer Fusion for End-to-End IoV-Based Autonomous Navigation
Mohan Mani1, Hariprasath K.2*, C. Vijayakumar3, Sathiyapriya Rajavelu2 and Sarawoot Boonkirdram4
1Department of Computer Science and Engineering, SRM University Delhi-NCR, Sonepat, Haryana, India
2School of Computing Science and Engineering, Galgotias University, Greater Noida, India
3Department of Computer Science and Engineering, Jain (Deemed-to-be University), Bangalore, Karnataka, India
4iProgram of Electrical and Electronics, Faculty of Industrial Technology, Sakon Nakhon Rajabhat University, Mueang, Sakon Nakhon, Thailand
Abstract
Single-modality (camera-only) vision-based end-to-end autonomous driving systems are susceptible to changes in the environment; they generally have poor generalization. To improve navigation reliability in IoV (Internet of Vehicles) paradigms, we introduce a transformer-based multimodal fusion network that fuses RGB images and LiDAR point clouds. A self-attention transformer is utilized to integrate spatial and depth information for end-to-end action prediction after using dual encoders for modal-specific processing. IoV edge servers provide data exchange between vehicles and in real-time updates to models. The system demonstrated its robustness and real-time applicability by attaining 93.65% navigation success rate, 91.12% collision-free driving rate, and 95.80% lane-keeping accuracy when tested on city and suburban driving routes in the CARLA simulator. ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access