Application of Multimodal Large Language Models in Autonomous Driving

Md Robiul Islam
12/21/2024
cs.CL

No confident code implementation found for this paper. 2 possible matches can be shown below.

Abstract

In this era of technological advancements, several cutting-edge techniques are being implemented to enhance Autonomous Driving (AD) systems, focusing on improving safety, efficiency, and adaptability in complex driving environments. However, AD still faces some problems including performance limitations. To address this problem, we conducted an in-depth study on implementing the Multi-modal Large Language Model. We constructed a Virtual Question Answering (VQA) dataset to fine-tune the model and address problems with the poor performance of MLLM on AD. We then break down the AD decision-making process by scene understanding, prediction, and decision-making. Chain of Thought has been used to make the decision more perfectly. Our experiments and detailed analysis of Autonomous Driving give an idea of how important MLLM is for AD.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 2 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

LLM4AD: Large Language Models for Autonomous Driving -- Concept, Review, Benchmark, Experiments, and Future Trends

89% match2024

LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving

89% match2023

DriVLM: Domain Adaptation of Vision-Language Models in Autonomous Driving

87% match2025

A Survey on Large Language Model-empowered Autonomous Driving

87% match2024

Multimodal Large Language Models for Enhanced Traffic Safety: A Comprehensive Review and Future Trends

86% match2025

A Survey on Vision-Language-Action Models for Autonomous Driving

86% match1 repo6072025

Cite this paper

@article{islam2024application,
  title  = {Application of Multimodal Large Language Models in Autonomous Driving},
  author = {Md Robiul Islam},
  year   = {2024},
  eprint = {2412.16410},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2412.16410v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.16410.svg)](https://codeofpaper.com/paper/2412.16410)

Discussion