HawkDrive: A Transformer-driven Visual Perception System for Autonomous Driving in Night Scene

Ziang Guo, Stepan Perminov, Mikhail Konenkov, Dzmitry Tsetserukou
4/6/2024
cs.CVcs.RO

Official code: ZionGo6/HawkDrive (2 stars, last commit 2024-07-08).

Abstract

Many established vision perception systems for autonomous driving scenarios ignore the influence of light conditions, one of the key elements for driving safety. To address this problem, we present HawkDrive, a novel perception system with hardware and software solutions. Hardware that utilizes stereo vision perception, which has been demonstrated to be a more reliable way of estimating depth information than monocular vision, is partnered with the edge computing device Nvidia Jetson Xavier AGX. Our software for low light enhancement, depth estimation, and semantic segmentation tasks, is a transformer-based neural network. Our software stack, which enables fast inference and noise reduction, is packaged into system modules in Robot Operating System 2 (ROS2). Our experimental results have shown that the proposed end-to-end system is effective in improving the depth estimation and semantic segmentation performance. Our dataset and codes will be released at https://github.com/ZionGo6/HawkDrive.

View on arXivView PDF

Code Implementations(1)

ZionGo6/HawkDriveOfficial100%

This is the repo for dataset and codes of our paper 'HawkDrive: A Transformer-driven Visual Perception System for Autonomous Driving in Night Scene'

21C++, CudaApr 1, 20242 years ago

Similar Papers

LED: Light Enhanced Depth Estimation at Night

85% match1 repo152024

CLFT: Camera-LiDAR Fusion Transformer for Semantic Segmentation in Autonomous Driving

85% match2 repos382024

GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision

84% match2024

Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving

84% match1 repo852026

A biologically inspired separable learning vision model for real-time traffic object perception in Dark

84% match1 repo72025

Parametric Depth Based Feature Representation Learning for Object Detection and Segmentation in Bird’s-Eye View

82% match2026

Cite this paper

@article{guo2024hawkdrive,
  title  = {HawkDrive: A Transformer-driven Visual Perception System for Autonomous Driving in Night Scene},
  author = {Ziang Guo and Stepan Perminov and Mikhail Konenkov and Dzmitry Tsetserukou},
  year   = {2024},
  eprint = {2404.04653},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.04653v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.04653.svg)](https://codeofpaper.com/paper/2404.04653)

Discussion