The BRAVO Semantic Segmentation Challenge Results in UNCV2024

Tuan-Hung Vu, Eduardo Valle, Andrei Bursuc, Tommie Kerssies, Daan de Geus +14 more
9/23/2024
cs.CVcs.AIcs.LG

No official code found. Best implementation: valeoai/bravo_challenge (21 stars, last commit 2025-04-30).

Abstract

We propose the unified BRAVO challenge to benchmark the reliability of semantic segmentation models under realistic perturbations and unknown out-of-distribution (OOD) scenarios. We define two categories of reliability: (1) semantic reliability, which reflects the model's accuracy and calibration when exposed to various perturbations; and (2) OOD reliability, which measures the model's ability to detect object classes that are unknown during training. The challenge attracted nearly 100 submissions from international teams representing notable research institutions. The results reveal interesting insights into the importance of large-scale pre-training and minimal architectural design in developing robust and reliable semantic segmentation models.

View on arXivView PDF

Code Implementations(1)

valeoai/bravo_challengeCommunity100%

BRAVO Challenge Toolkit and Evaluation Code

212C++, CudaJun 5, 20241 years agoNOASSERTION

Similar Papers

First Place Solution to the ECCV 2024 BRAVO Challenge: Evaluating Robustness of Vision Foundation Models for Semantic Segmentation

84% match1 repo382024

Training-Free Out-Of-Distribution Segmentation With Foundation Models

82% match2025

Instance-wise Uncertainty for Class Imbalance in Semantic Segmentation

82% match2024

SemSegBench & DetecBench: Benchmarking Reliability and Generalization Beyond Classification

81% match1 repo32025

Reliability in Semantic Segmentation: Are we on the Right Track?

81% match1 repo242026

Evaluating the Adversarial Robustness of Semantic Segmentation: Trying Harder Pays Off

81% match1 repo32024

Cite this paper

@article{vu2024bravo,
  title  = {The BRAVO Semantic Segmentation Challenge Results in UNCV2024},
  author = {Tuan-Hung Vu and Eduardo Valle and Andrei Bursuc and Tommie Kerssies and Daan de Geus and Gijs Dubbelman and Long Qian and Bingke Zhu and Yingying Chen and Ming Tang and Jinqiao Wang and Tomáš Vojíř and Jan Šochman and Jiří Matas and Michael Smith and Frank Ferrie and Shamik Basu and Christos Sakaridis and Luc Van Gool},
  year   = {2024},
  eprint = {2409.15107},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.15107v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.15107.svg)](https://codeofpaper.com/paper/2409.15107)

Discussion