TUMTraf V2X Cooperative Perception Dataset

Abstract

Cooperative perception offers several benefits for en-hancing the capabilities of autonomous vehicles and im-proving road safety. Using roadside sensors in addition to onboard sensors increases reliability and extends the sensor range. External sensors offer higher situational awareness for automated vehicles and prevent occlusions. We propose CoopDet3D, a cooperative multi-modal fusion model, and TUMTraf- V2X, a perception dataset, for the cooperative 3D object detection and tracking task. Our dataset contains 2,000 labeled point clouds and 5,000 labeled images from five roadside and four onboard sensors. It includes 30k 3D boxes with track IDs and precise GPS and IMU data. We labeled nine categories and covered occlusion scenarios with challenging driving maneuvers, like traffic violations, near-miss events, overtaking, and U-turns. Through multiple experiments, we show that our CoopDet3D camera-LiDARfusion model achieves an increase of +14.36 3D mAP compared to a vehicle camera-LiDARfusion model. Finally, we make our dataset, model, labeling tool, and devkit publicly available on our website.