MOTSynth-MOT-CVPR22 Results
Click on a measure to sort the table accordingly. See below for a more detailed description.
| Tracker | MOTA | IDF1 | HOTA | MT | ML | FP | FN | Rcll | Prcn | AssA | DetA | AssRe | AssPr | DetRe | DetPr | LocA | FAF | ID Sw. | Frag | Hz | |
| SIA_Track 1.   | 77.6 | 76.8 | 63.2 | 453 (57.7) | 88 (11.2) | 15,406 | 25,284 | 86.6 | 91.4 | 62.3 | 64.4 | 69.5 | 75.3 | 72.2 | 76.2 | 83.6 | 2.6 | 1,363 (0.0) | 1,085 (0.0) | 24.2 | |
| SI Analytics | |||||||||||||||||||||
| SDTracker 2. | 77.5 | 77.2 | 61.4 | 453 (57.7) | 91 (11.6) | 16,057 | 25,613 | 86.4 | 91.0 | 61.0 | 62.2 | 67.8 | 74.4 | 69.9 | 73.7 | 81.6 | 2.7 | 565 (0.0) | 848 (0.0) | 1.2 | |
| PieTrack 3. | 74.1 | 73.6 | 58.7 | 393 (50.1) | 113 (14.4) | 16,692 | 31,206 | 83.4 | 90.4 | 57.9 | 59.9 | 64.3 | 72.6 | 67.6 | 73.3 | 81.8 | 2.8 | 759 (0.0) | 921 (0.0) | 3.6 | |
| PAII CVG | |||||||||||||||||||||
| AIICMOT_v2 4. | 73.6 | 68.1 | 56.1 | 337 (42.9) | 151 (19.2) | 8,830 | 39,875 | 78.8 | 94.4 | 53.1 | 59.6 | 60.4 | 71.0 | 64.6 | 77.4 | 82.6 | 1.5 | 882 (0.0) | 1,024 (0.0) | 19.7 | |
| AIIC Vision, Midea Group | |||||||||||||||||||||
| fe_track 5.  | 73.4 | 72.7 | 58.4 | 498 (63.4) | 65 (8.3) | 25,971 | 22,363 | 88.1 | 86.5 | 57.3 | 60.1 | 65.6 | 70.2 | 70.8 | 69.4 | 81.1 | 4.4 | 1,631 (0.0) | 1,287 (0.0) | 18.4 | |
| Synth_SORTv1 6. | 67.0 | 76.4 | 58.0 | 359 (45.7) | 132 (16.8) | 13,564 | 48,133 | 74.4 | 91.2 | 61.5 | 54.9 | 67.7 | 77.3 | 60.6 | 74.2 | 81.4 | 2.3 | 352 (0.0) | 817 (0.0) | 4.9 | |
| Tracktor++v2 7. | 56.5 | 53.0 | 42.8 | 205 (26.1) | 205 (26.1) | 10,891 | 70,032 | 62.8 | 91.6 | 39.7 | 46.5 | 43.8 | 73.1 | 50.1 | 73.1 | 79.4 | 1.8 | 946 (0.0) | 1,594 (0.0) | 1.5 | |
| P. Bergmann, T. Meinhardt, L. Leal-Taixé. Tracking without bells and whistles. In ICCV, 2019. | |||||||||||||||||||||
| FNT_v1 8. | 55.7 | 60.2 | 46.3 | 208 (26.5) | 247 (31.5) | 16,788 | 65,834 | 65.0 | 87.9 | 46.3 | 46.6 | 53.3 | 64.4 | 51.3 | 69.5 | 77.9 | 2.8 | 767 (0.0) | 1,615 (0.0) | 39.5 | |
| Anonymous submission | |||||||||||||||||||||
| Sequences | Frames | Trajectories | Boxes |
| 7 | 5919 | 785 | 188076 |
Evaluation Measures
Lower is better. Higher is better.| Measure | Better | Perfect | Description |
| MOTA | higher | 100% | Multi-Object Tracking Accuracy (+/- denotes standard deviation across all sequences) [1]. This measure combines three error sources: false positives, missed targets and identity switches. |
| IDF1 | higher | 100% | ID F1 Score [2]. The ratio of correctly identified detections over the average number of ground-truth and computed detections. |
| HOTA | higher | 100% | Higher Order Tracking Accuracy [3]. Geometric mean of detection accuracy and association accuracy. Averaged across localization thresholds. |
| MT | higher | 100% | Mostly tracked targets. The ratio of ground-truth trajectories that are covered by a track hypothesis for at least 80% of their respective life span. |
| ML | lower | 0% | Mostly lost targets. The ratio of ground-truth trajectories that are covered by a track hypothesis for at most 20% of their respective life span. |
| FP | lower | 0 | The total number of false positives. |
| FN | lower | 0 | The total number of false negatives (missed targets). |
| Rcll | higher | 100% | Ratio of correct detections to total number of GT boxes. |
| Prcn | higher | 100% | Ratio of TP / (TP+FP). |
| AssA | higher | 100% | Association Accuracy [3]. Association Jaccard index averaged over all matching detections and then averaged over localization thresholds. |
| DetA | higher | 100% | Detection Accuracy [3]. Detection Jaccard index averaged over localization thresholds. |
| AssRe | higher | 100% | Association Recall [3]. TPA / (TPA + FNA) averaged over all matching detections and then averaged over localization thresholds. |
| AssPr | higher | 100% | Association Precision [3]. TPA / (TPA + FPA) averaged over all matching detections and then averaged over localization thresholds. |
| DetRe | higher | 100% | Detection Recall [3]. TP /(TP + FN) averaged over localization thresholds. |
| DetPr | higher | 100% | Detection Precision [3]. TP /(TP + FP) averaged over localization thresholds. |
| LocA | higher | 100% | Localization Accuracy [3]. Average localization similarity averaged over all matching detections and averaged over localization thresholds. |
| FAF | lower | 0 | The average number of false alarms per frame. |
| ID Sw. | lower | 0 | Number of Identity Switches (ID switch ratio = #ID switches / recall) [4]. Please note that we follow the stricter definition of identity switches as described in the reference |
| Frag | lower | 0 | The total number of times a trajectory is fragmented (i.e. interrupted during tracking). |
| Hz | higher | Inf. | Processing speed (in frames per second excluding the detector) on the benchmark. The frequency is provided by the authors and not officially evaluated by the MOTChallenge. |
Legend
| Symbol | Description |
|
This is an online (causal) method, i.e. the solution is immediately available with each incoming frame and cannot be changed at any later time. |
|
This method used the provided detection set as input. |
|
This method used a private detection set as input. |
 |
This entry has been submitted or updated less than a week ago. |
References:
| [1] | Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. Image and Video Processing, 2008(1):1-10, 2008. |
| [2] | Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. In ECCV workshop on Benchmarking Multi-Target Tracking, 2016. |
| [3] | HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking. International Journal of Computer Vision, 2020. |
| [4] | Learning to associate: HybridBoosted multi-target tracker for crowded scene. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2009. |
