CVPR 2020 MOTS Challenge Results
Click on a measure to sort the table accordingly. See below for a more detailed description.
| Tracker | sMOTSA | IDF1 | MOTSA | MOTSP | MODSA | MT | ML | TP | FP | FN | Rcll | Prcn | ID Sw. | Frag | Hz |
| ReMOTS 1.  | 69.9 | 75.0 | 83.9 | 84.0 | 85.1 | 248 (75.6) | 12 (3.7) | 28,270 | 819 | 3,999 | 87.6 | 97.2 | 388 (442.9) | 621 (708.8) | 0.3 |
| F. Yang, X. Chang, C. Dang, Z. Zheng, S. Sakti, S. Nakamura, Y. Wu. ReMOTS: Self-Supervised Refining Multi-Object Tracking and Segmentation. In , 2020. | |||||||||||||||
| GMPHD_SAF 2.   | 68.4 | 64.9 | 82.6 | 83.9 | 84.4 | 248 (75.6) | 10 (3.0) | 28,382 | 1,161 | 3,887 | 88.0 | 96.1 | 569 (646.9) | 770 (875.5) | 3.8 |
| Y. Song, M. Jeon. Online Multi-Object Tracking and Segmentation with GMPHD Filter and Simple Affinity Fusion. In arXiv preprint arXiv:2009.00100, 2020. | |||||||||||||||
| DD_Vision 3. | 66.6 | 71.8 | 79.7 | 84.4 | 80.7 | 243 (74.1) | 15 (4.6) | 27,114 | 1,067 | 5,155 | 84.0 | 96.2 | 341 (405.8) | 559 (665.3) | 1.6 |
| L. Yang Liu. Tracking by Segmentation: Person-ReID and Optical Flow Based Offline Tracker for the MOTSChallenge 2020. In , 2020. | |||||||||||||||
| TrackRCNN 4. | 40.6 | 42.4 | 55.2 | 76.1 | 56.9 | 127 (38.7) | 71 (21.6) | 19,628 | 1,261 | 12,641 | 60.8 | 94.0 | 567 (932.2) | 868 (1,427.0) | 2.0 |
| P. Voigtlaender, M. Krause, A. O\usep, J. Luiten, B. Sekar, A. Geiger, B. Leibe. MOTS: Multi-Object Tracking and Segmentation. In CVPR, 2019. | |||||||||||||||
| Sequences | Frames | Trajectories | Boxes |
| 4 | 3044 | 328 | 32269 |
Evaluation Measures
Lower is better. Higher is better.| Measure | Better | Perfect | Description |
| sMOTSA | higher | 100% | Mask-based Soft Multi-Object Tracking Accuracy (+/- denotes standard deviation across all sequences) [1]. Soft version of MOTSA, accumulates the mask overlaps of true positives instead of only counting how many masks reach an IoU of more than 0.5. |
| IDF1 | higher | 100% | ID F1 Score [2]. The ratio of correctly identified detections over the average number of ground-truth and computed detections. |
| MOTSA | higher | 100% | Mask-based Multi-Object Tracking Accuracy (+/- denotes standard deviation across all sequences) [1]. Variant of MOTA, evaluated based on mask overlap (mask IoU) |
| MOTSP | higher | 100% | Mask-overlap based variant of Multi-Object Tracking Precision. [1]. Variant of MOTP, evaluated based on mask IoU instead of bounding box IoU. |
| MODSA | higher | 100% | Mask-overlap based Multi-Object Detection Accuracy [1]. Variant of MODA, evaluated based on the mask overlap (mask IoU). |
| MT | higher | 100% | Mostly tracked targets. The ratio of ground-truth trajectories that are covered by a track hypothesis for at least 80% of their respective life span. |
| ML | lower | 0% | Mostly lost targets. The ratio of ground-truth trajectories that are covered by a track hypothesis for at most 20% of their respective life span. |
| TP | higher | #GT | The total number of true positives. |
| FP | lower | 0 | The total number of false positives. |
| FN | lower | 0 | The total number of false negatives (missed targets). |
| Rcll | higher | 100% | Ratio of correct detections to total number of GT boxes. |
| Prcn | higher | 100% | Ratio of TP / (TP+FP). |
| ID Sw. | lower | 0 | Number of Identity Switches (ID switch ratio = #ID switches / recall) [3]. Please note that we follow the stricter definition of identity switches as described in the reference |
| Frag | lower | 0 | The total number of times a trajectory is fragmented (i.e. interrupted during tracking). |
| Hz | higher | Inf. | Processing speed (in frames per second excluding the detector) on the benchmark. The frequency is provided by the authors and not officially evaluated by the MOTChallenge. |
Legend
| Symbol | Description |
|
This is an online (causal) method, i.e. the solution is immediately available with each incoming frame and cannot be changed at any later time. |
|
This method used the provided detection set as input. |
|
This method used a private detection set as input. |
 |
This entry has been submitted or updated less than a week ago. |
References:
| [1] | MOTS: Multi-Object Tracking and Segmentation. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019. |
| [2] | Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. In ECCV workshop on Benchmarking Multi-Target Tracking, 2016. |
| [3] | Learning to associate: HybridBoosted multi-target tracker for crowded scene. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2009. |
