MOTS Results
Click on a measure to sort the table accordingly. See below for a more detailed description.
| Tracker | sMOTSA | IDF1 | MOTSA | MOTSP | MODSA | MT | ML | TP | FP | FN | Rcll | Prcn | ID Sw. | Frag | Hz |
| ReMOTSv2 1.  | 70.4 | 75.8 | 84.4 | 84.0 | 85.1 | 248 (75.6) | 12 (3.7) | 28,270 | 819 | 3,999 | 87.6 | 97.2 | 229 (261.4) | 579 (660.9) | 0.3 |
| F. Yan, Z. Wang, Y. Wu, S. Sakti, S. Nakamura. Tackling multiple object tracking with complicated motions — Re-designing the integration of motion and appearance. In Image and Vision Computing, 2022. | |||||||||||||||
| ReMOTS_ 2.  | 70.4 | 75.0 | 84.4 | 84.0 | 85.1 | 248 (75.6) | 12 (3.7) | 28,270 | 819 | 3,999 | 87.6 | 97.2 | 231 (263.7) | 579 (660.9) | 0.3 |
| F. Yang, X. Chang, C. Dang, Z. Zheng, S. Sakti, S. Nakamura, Y. Wu. ReMOTS: Refining Multi-Object Tracking and Segmentation. In , 2020. | |||||||||||||||
| EMNT 3.  | 70.0 | 77.0 | 83.7 | 84.1 | 84.5 | 234 (71.3) | 13 (4.0) | 27,943 | 666 | 4,326 | 86.6 | 97.7 | 261 (301.4) | 618 (713.7) | 11.7 |
| S. Wang, H. Sheng, D. Yang, Y. Zhang, Y. Wu and S. Wang, "Extendable Multiple Nodes Recurrent Tracking Framework with RTU++," in IEEE Transactions on Image Processing, 2022, doi: 10.1109/TIP.2022.3192706. | |||||||||||||||
| MAF_HDA 4. | 69.9 | 67.0 | 83.8 | 84.1 | 85.1 | 245 (74.7) | 12 (3.7) | 28,311 | 863 | 3,958 | 87.7 | 97.0 | 401 (457.1) | 598 (681.6) | 4.6 |
| Y. Song, Y. Yoon, K. Yoon, M. Jeon. Multi-Object Tracking and Segmentation with Embedding Mask-based Affinity Fusion in Hierarchical Data Association. In IEEE Access, 2022. | |||||||||||||||
| VFS_MOTS 5. | 69.7 | 68.2 | 83.5 | 84.2 | 84.6 | 253 (77.1) | 9 (2.7) | 28,087 | 800 | 4,182 | 87.0 | 97.2 | 342 (392.9) | 625 (718.1) | 7.6 |
| COSTA_st 6. | 69.5 | 70.3 | 83.3 | 84.2 | 84.6 | 253 (77.1) | 9 (2.7) | 28,093 | 806 | 4,176 | 87.1 | 97.2 | 421 (483.6) | 717 (823.6) | 2.1 |
| GMPHD_MAF 7. | 69.4 | 66.4 | 83.3 | 84.2 | 84.8 | 249 (75.9) | 11 (3.4) | 28,284 | 935 | 3,985 | 87.7 | 96.8 | 484 (552.2) | 711 (811.2) | 2.6 |
| Y. Song, Y. Yoon, K. Yoon, M. Jeon, S. Lee, W. Pedrycz. Online Multi-Object Tracking and Segmentation with GMPHD Filter and Mask-based Affinity Fusion. In [Online]. Available: arXiv:2009.00100, 2021. | |||||||||||||||
| UniTrack 8. | 68.9 | 67.2 | 82.6 | 84.2 | 84.5 | 253 (77.1) | 9 (2.7) | 28,084 | 801 | 4,185 | 87.0 | 97.2 | 622 (714.7) | 825 (947.9) | 7.6 |
| OPITrack_ 9. | 63.5 | 45.4 | 75.5 | 84.6 | 76.6 | 218 (66.5) | 28 (8.5) | 25,152 | 441 | 7,117 | 77.9 | 98.3 | 342 (438.8) | 583 (748.0) | 3,382.2 |
| Y. Gao, H. Xu, Y. Zheng, J. Li and X. Gao, "An Object Point Set Inductive Tracker for Multi-Object Tracking and Segmentation," in IEEE Transactions on Image Processing, 2022, doi: 10.1109/TIP.2022.3203607. | |||||||||||||||
| ZXPointTrack 10. | 62.3 | 42.9 | 76.8 | 82.3 | 78.4 | 186 (56.7) | 41 (12.5) | 26,276 | 963 | 5,993 | 81.4 | 96.5 | 541 (664.4) | 868 (1,066.0) | 4.6 |
| TBD | |||||||||||||||
| Tracker | sMOTSA | IDF1 | MOTSA | MOTSP | MODSA | MT | ML | TP | FP | FN | Rcll | Prcn | ID Sw. | Frag | Hz |
| CCP 11. | 59.3 | 58.1 | 75.5 | 80.3 | 77.0 | 204 (62.2) | 32 (9.8) | 26,610 | 1,759 | 5,659 | 82.5 | 93.8 | 484 (586.9) | 645 (782.2) | 10.5 |
| TBD | |||||||||||||||
| MPNTrackSeg 12. | 58.6 | 68.8 | 73.7 | 80.6 | 74.3 | 207 (63.1) | 26 (7.9) | 25,036 | 1,059 | 7,233 | 77.6 | 95.9 | 202 (260.4) | 635 (818.5) | 2.3 |
| G. Bras'o, O. Cetintas, L. Leal-Taix'e. Multi-Object Tracking and Segmentation Via Neural Message Passing. In International Journal of Computer Vision, 2022. | |||||||||||||||
| SORTS_ReID 13. | 55.8 | 65.8 | 69.1 | 81.9 | 70.0 | 107 (32.6) | 52 (15.9) | 23,671 | 1,076 | 8,598 | 73.4 | 95.7 | 304 (414.4) | 576 (785.2) | 36.4 |
| Not yet published | |||||||||||||||
| SORTS 14. | 55.0 | 57.3 | 68.3 | 81.9 | 70.0 | 107 (32.6) | 52 (15.9) | 23,671 | 1,076 | 8,598 | 73.4 | 95.7 | 552 (752.5) | 577 (786.6) | 54.5 |
| M. Ahrnbom, M. Nilsson, H. Ard"o. Real-time and Online Segmentation Multi-target Tracking with Track Revival Re-identification.. In VISIGRAPP (5: VISAPP), 2021. | |||||||||||||||
| Struct_MOTS 15. | 55.0 | 64.0 | 64.2 | 85.9 | 65.2 | 165 (50.3) | 52 (15.9) | 21,215 | 166 | 11,054 | 65.7 | 99.2 | 330 (501.9) | 609 (926.3) | 8.7 |
| UBVision 16. | 52.8 | 58.3 | 67.4 | 79.9 | 69.2 | 162 (49.4) | 54 (16.5) | 23,465 | 1,142 | 8,804 | 72.7 | 95.4 | 568 (781.1) | 702 (965.4) | 10.1 |
| TrackRCNN 17. | 40.6 | 42.4 | 55.2 | 76.1 | 56.9 | 127 (38.7) | 71 (21.6) | 19,628 | 1,261 | 12,641 | 60.8 | 94.0 | 567 (932.2) | 868 (1,427.0) | 2.0 |
| P. Voigtlaender, M. Krause, A. O\usep, J. Luiten, B. Sekar, A. Geiger, B. Leibe. MOTS: Multi-Object Tracking and Segmentation. In CVPR, 2019. | |||||||||||||||
| Sequences | Frames | Trajectories | Boxes |
| 4 | 3044 | 328 | 32269 |
Evaluation Measures
Lower is better. Higher is better.| Measure | Better | Perfect | Description |
| sMOTSA | higher | 100% | Mask-based Soft Multi-Object Tracking Accuracy (+/- denotes standard deviation across all sequences) [1]. Soft version of MOTSA, accumulates the mask overlaps of true positives instead of only counting how many masks reach an IoU of more than 0.5. |
| IDF1 | higher | 100% | ID F1 Score [2]. The ratio of correctly identified detections over the average number of ground-truth and computed detections. |
| MOTSA | higher | 100% | Mask-based Multi-Object Tracking Accuracy (+/- denotes standard deviation across all sequences) [1]. Variant of MOTA, evaluated based on mask overlap (mask IoU) |
| MOTSP | higher | 100% | Mask-overlap based variant of Multi-Object Tracking Precision. [1]. Variant of MOTP, evaluated based on mask IoU instead of bounding box IoU. |
| MODSA | higher | 100% | Mask-overlap based Multi-Object Detection Accuracy [1]. Variant of MODA, evaluated based on the mask overlap (mask IoU). |
| MT | higher | 100% | Mostly tracked targets. The ratio of ground-truth trajectories that are covered by a track hypothesis for at least 80% of their respective life span. |
| ML | lower | 0% | Mostly lost targets. The ratio of ground-truth trajectories that are covered by a track hypothesis for at most 20% of their respective life span. |
| TP | higher | #GT | The total number of true positives. |
| FP | lower | 0 | The total number of false positives. |
| FN | lower | 0 | The total number of false negatives (missed targets). |
| Rcll | higher | 100% | Ratio of correct detections to total number of GT boxes. |
| Prcn | higher | 100% | Ratio of TP / (TP+FP). |
| ID Sw. | lower | 0 | Number of Identity Switches (ID switch ratio = #ID switches / recall) [3]. Please note that we follow the stricter definition of identity switches as described in the reference |
| Frag | lower | 0 | The total number of times a trajectory is fragmented (i.e. interrupted during tracking). |
| Hz | higher | Inf. | Processing speed (in frames per second excluding the detector) on the benchmark. The frequency is provided by the authors and not officially evaluated by the MOTChallenge. |
Legend
| Symbol | Description |
|
This is an online (causal) method, i.e. the solution is immediately available with each incoming frame and cannot be changed at any later time. |
|
This method used the provided detection set as input. |
|
This method used a private detection set as input. |
 |
This entry has been submitted or updated less than a week ago. |
References:
| [1] | MOTS: Multi-Object Tracking and Segmentation. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019. |
| [2] | Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. In ECCV workshop on Benchmarking Multi-Target Tracking, 2016. |
| [3] | Learning to associate: HybridBoosted multi-target tracker for crowded scene. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2009. |
