TAO Open-World Results

Download all results for this benchmark

Click on a measure to sort the table accordingly. See below for a more detailed description.


Benchmark Statistics

TrackerOWTAallDetReallAssAall OWTAcomDetRecomAssAcomOWTAuncDetReuncAssAuncHz
OWTB
1. 		56.4 70.7 45.5 59.9 76.6 47.4 38.3 45.7 33.6 1.0
Y. Liu, I. Zulfikar, J. Luiten, A. Dave, D. Ramanan, B. Leibe, A. Osep, L. Leal-Taixe. Opening up Open-World Tracking. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
STCNTracker2
2. 		57.5 61.6 54.1 62.9 71.5 55.7 23.9 21.0 28.6 1.0
A. Athar, J. Luiten, P. Voigtlaender, T. Khurana, A. Dave, B. Leibe, D. Ramanan. BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Video. In WACV, 2023.
LeTracker
3. online method 		61.4 74.0 51.5 64.3 78.7 52.9 47.1 54.6 42.0 20.9
SequencesFramesTrajectoriesBoxes
1521947963167132

Evaluation Measures

Lower is better. Higher is better.
Measure Better Perfect Description
OWTAall higher 100%OWTA all [1]. OWTA (Open-World Tracking Accuracy) metric for Open-World Tracking [CITATION OWT PAPER] is a generalization of the recently proposed HOTA metric [CITATION HOTA PAPER] for closed-world tracking. OWTA takes both detection recall (DetRe) and association accuracy (AssA) into account and combines them into a single score for all classes.
DetReall higher 100%DetRe all [2]. TP /(TP + FN) averaged over localization thresholds for all classes.
AssAall higher 100%AssA all [2]. Association Jaccard index averaged over all matching detections and then averaged over localization thresholds for all classes.
OWTAcom higher 100%OWTA common [1]. OWTA (Open-World Tracking Accuracy) metric for Open-World Tracking [CITATION OWT PAPER] is a generalization of the recently proposed HOTA metric [CITATION HOTA PAPER] for closed-world tracking. OWTA takes both detection recall (DetRe) and association accuracy (AssA) into account and combines them into a single score for common classes.
DetRecom higher 100%DetRe common [2]. TP /(TP + FN) averaged over localization thresholds for common classes.
AssAcom higher 100%AssA common [2]. Association Jaccard index averaged over all matching detections and then averaged over localization thresholds for common classes.
OWTAunc higher 100%OWTA uncommon [1]. OWTA (Open-World Tracking Accuracy) metric for Open-World Tracking [CITATION OWT PAPER] is a generalization of the recently proposed HOTA metric [CITATION HOTA PAPER] for closed-world tracking. OWTA takes both detection recall (DetRe) and association accuracy (AssA) into account and combines them into a single score for uncommon classes.
DetReunc higher 100%DetRe uncommon [2]. TP /(TP + FN) averaged over localization thresholds for uncommon classes.
AssAunc higher 100%AssA uncommon [2]. Association Jaccard index averaged over all matching detections and then averaged over localization thresholds for uncommon classes.
Hz higher Inf.Processing speed (in frames per second excluding the detector) on the benchmark. The frequency is provided by the authors and not officially evaluated by the MOTChallenge.

Legend

Symbol Description
online method This is an online (causal) method, i.e. the solution is immediately available with each incoming frame and cannot be changed at any later time.
using public detections This method used the provided detection set as input.
using private detections This method used a private detection set as input.
new This entry has been submitted or updated less than a week ago.

References:


[1] Liu, Y., Zulfikar, I.E., Luiten, J., Dave, A., Ramanan, D., Leibe, B., Osep, A. & Leal-Taixe, L. Opening up Open-World Tracking. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[2] Jonathon Luiten, A.O. & Leibe, B. HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking. International Journal of Computer Vision, 2020.



Contact