M$^2$DAR: Multi-View Multi-Scale Driver Action Recognition with Vision Transformer

Yunsheng Ma, Liangqi Yuan, Amr Abdelraouf, Kyungtae Han, Rohit Gupta +2 more
5/13/2023
cs.CV

Official code: PurdueDigitalTwin/M2DAR (10 stars, MIT, last commit 2023-04-27).

Abstract

Ensuring traffic safety and preventing accidents is a critical goal in daily driving, where the advancement of computer vision technologies can be leveraged to achieve this goal. In this paper, we present a multi-view, multi-scale framework for naturalistic driving action recognition and localization in untrimmed videos, namely M$^2$DAR, with a particular focus on detecting distracted driving behaviors. Our system features a weight-sharing, multi-scale Transformer-based action recognition network that learns robust hierarchical representations. Furthermore, we propose a new election algorithm consisting of aggregation, filtering, merging, and selection processes to refine the preliminary results from the action recognition module across multiple views. Extensive experiments conducted on the 7th AI City Challenge Track 3 dataset demonstrate the effectiveness of our approach, where we achieved an overlap score of 0.5921 on the A2 test set. Our source code is available at \url{https://github.com/PurdueDigitalTwin/M2DAR}.

View on arXivView PDF

Code Implementations(1)

PurdueDigitalTwin/M2DAROfficial100%
MIT

This repository contains PyTorch code and models for M2DAR, a solution to the AI City Challenge Track 3 on driver action recognition in untrimmed videos.

101PythonApr 6, 20233 years agoMIT

Similar Papers

MVP: Robust Multi-View Practice for Driving Action Localization

88% match2022

Rethinking Top Probability from Multi-view for Distracted Driver Behaviour Localization

87% match2024

Density-Guided Label Smoothing for Temporal Localization of Driving Actions

87% match2024

Vision-Language Models can Identify Distracted Driver Behavior from Naturalistic Videos

87% match1 repo232023

A Contextual Analysis of Driver-Facing and Dual-View Video Inputs for Distraction Detection in Naturalistic Driving Environments

87% match2025

DSDFormer: An Innovative Transformer-Mamba Framework for Robust High-Precision Driver Distraction Identification

87% match2024

Cite this paper

@article{ma2023mdar,
  title  = {M$^2$DAR: Multi-View Multi-Scale Driver Action Recognition with Vision Transformer},
  author = {Yunsheng Ma and Liangqi Yuan and Amr Abdelraouf and Kyungtae Han and Rohit Gupta and Zihao Li and Ziran Wang},
  year   = {2023},
  eprint = {2305.08877},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.08877v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.08877.svg)](https://codeofpaper.com/paper/2305.08877)

Discussion