×

WildDash 2 Benchmark

New: WildDash 2 with 4256 public frames, new labels & panoptic GT!  
See also: RailSem19 dataset for rail scene understanding.

Contact

For any questions or suggestions, please create a new issue here us: github.

References

If you use WildDash in your work, please reference this paper:

[1] Unifying Panoptic Segmentation for Autonomous Driving
Zendel, Oliver and Schoerghuber, Matthias and Rainer, Bernhard and Murschitz, Markus and Beleznai, Csaba
Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

If you use RailSem19 in your work, please reference this paper:

[2] RailSem19: A Dataset for Semantic Rail Scene Understanding
Oliver Zendel, Markus Murschitz, Marcel Zeilinger, Daniel Steininger, Sara Abbasi, and Csaba Beleznai
Conference on Computer Vision and Pattern Recognition (CVPR) Workshop, 2019.

Please read this work for more background regarding our CV-HAZOP approach:

[3] How Good Is My Test Data? Introducing Safety Analysis for Computer Vision.
Zendel O., Murschitz M., Humenberger M., and Herzner W.
In International Journal of Computer Vision (IJCV), 2017.

[4] WildDash - Creating Hazard-Aware Benchmarks
Zendel, Oliver and Honauer, Katrin and Murschitz, Markus and Steininger, Daniel and Fernandez Dominguez, Gustavo
The European Conference on Computer Vision (ECCV), 2018.

Team

Oliver Zendel
oliver.zendel (at) ait.ac.at

Markus Murschitz
markus.murschitz (at) ait.ac.at

Katrin Honauer
katrin.honauer (at) iwr.uni-heidelberg.de

Gustavo Fernandez
gustavojavier.fernandez (at) ait.ac.at

Daniel Steininger
daniel.steininger (at) ait.ac.at

Bernhard Rainer
bernhard.rainer (at) ait.ac.at

Matthias Schörghuber
matthias.schoerghuber (at) ait.ac.at

Changelog

  • 2020-06-17 WildDash 2 beta launched
  • 2019-06-06 RailSem19 v1.0 released
  • 2018-11-22 Freeing up submission count from ROB submissions
  • 2018-09-17 Updated reference for WildDash ECCV 2018 publication
  • 2018-09-07 Validation GT improved for wd_val_01 (v1.0 instead of v0.8)
  • 2018-06-07 Algorithm results and evaluation visualizated at subpages accessible from both leaderboards
  • 2018-05-03 Added impact evaluation per hazard subset and cached table for faster viewing
  • 2018-04-03 The instance and semantic segmentation benchmarks are open for submissions.
  • 2018-03-30 Benchmark datasets for the instance & semantic segmentation are available for download.
  • 2018-02-01 Initial datasets for instance & semantic segmentation validation data are available for download.
  • 2018-01-30 Public access to WildDash website.

Privacy Policy

Data collected from visitors (no user account):
The timestamp, IP address, and entered website URL of visitors is recorded to allow web traffic analysis. This data is not shared externally. IP addresses are needed to differentiate between unique and reoccurring visits. Any report / document that relies on/uses this information only shows aggregated numbers and trends (users are anonymized).
Data collected from users/submissions:
General user information (Name, Institute) is needed to assure that the benchmark dataset and this service is used for research purposes only. Also, this information including IP addresses of users are necessary to prevent double submissions/cheating. User data is not visible publically. Aggregated numbers and trends may be part of public project deliverables (users are anonymized). Email addresses are necessary for the submission system to work. Additionally, they are only used for contacting the user with topics relevant to the WildDash benchmark/dataset and are not shared in any way. Information entered with each algorithm submissions are fully visible. This is necessary to provide context for the benchmarking entries.
Email contact with users:
The submission system automatically sends emails to inform the user about errors/status changes to their submission. In addition, news regarding the WildDash benchmark and dataset are sent to the user up to four times a year (only when there are updates to the data/benchmark relevant to participants of the benchmark so that a fair evaluation is possible).
Removal of user accounts/submissions:
Please sent an Email to: wilddash@ait.ac.at. Already visible public submission (including scores and visualizations of algorithm results) are part of the leaderboard and thus part of relative comparisons. These comparisons are essential to the actual content of the benchmark, its value for safety and scientific progress. Thus, we will not remove existing valid submissions. However, we will remove identifiable context information (author name, paper, project website, etc.) if there is a relevant reason for this. Also, we may change the visibility of deprecated/dated algorithms in such a way that they are not visible in the main leaderboard (only in the extended view).

Contact

For any questions or suggestions, please contact us: wilddash@ait.ac.at.

Acknowledgments

This project received financial support from the Horizon 2020 program of the European Union under the grant of the AutoDrive project "Advancing fail-aware, fail-safe, and fail-operational electronic components, systems, and architectures for fully automated driving to make future mobility safer, affordable, and end-user acceptable" (Grant No. 737469). Please visit www.autodrive-project.eu for more information.

We are grateful for the numerous authors who generously agreed to share frames of their dashcam videos for this research project. Individual attributions for the dashcam frames can be found in the download packages.

Furthermore, we gratefully acknowledge support for this research by the AIT Austrian Institute of Technology in Vienna and the Heidelberg Collaboratory for Image Processing (HCI).