3D MOT 2015 Results
Click on a measure to sort the table accordingly. See below for a more detailed description.
| Tracker | Avg Rank | MOTA | IDF1 | MT | ML | FP | FN | ID Sw. | Frag | Hz | Detector |
| LP3D 1.  | 4.9 | 35.9 ±11.1 | 0.0 | 20.9% | 16.4% | 3,588 | 6,593 | 580 (9.6) | 659 (10.9) | 83.5 | Public |
| MOT baseline: Linear programming on 3D image coordinates. | |||||||||||
| LPSFM 2. | 5.2 | 35.9 ±6.3 | 0.0 | 13.8% | 21.6% | 2,031 | 8,206 | 520 (10.2) | 601 (11.8) | 8.4 | Public |
| L. Leal-Taixé, G. Pons-Moll, B. Rosenhahn. Everybody needs somebody: modeling social and grouping behavior on a linear programming multiple people tracker. In IEEE International Conference on Computer Vision Workshops (ICCVW). 1st Workshop on Modeling, Simulation and Visual Analysis of Large Crowds, 2011. | |||||||||||
| KalmanSFM 3.  | 6.3 | 25.0 ±8.5 | 0.0 | 6.7% | 14.6% | 3,161 | 7,599 | 1,838 (33.6) | 1,686 (30.8) | 30.6 | Public |
| S. Pellegrini, A. Ess, K. Schindler, L. Gool. You'll never walk alone: Modeling social behavior for multi-target tracking.. In ICCV, 2009. | |||||||||||
| DBN 4. | 3.4 | 51.1 ±7.6 | 0.0 | 28.7% | 17.9% | 2,077 | 5,746 | 380 (5.8) | 418 (6.4) | 0.1 | Public |
| T. Klinger, F. Rottensteiner, C. Heipke. Probabilistic Multi-Person Tracking using Dynamic Bayes Networks. In ISPRS Workshop on Image Sequence Analysis (ISA), 2015. | |||||||||||
| GPDBN 5. | 3.4 | 49.8 ±6.6 | 0.0 | 25.7% | 17.2% | 1,813 | 6,300 | 311 (5.0) | 386 (6.2) | 0.1 | Public |
| T. Klinger, F. Rottensteiner, C. Heipke. Probabilistic multi-person localisation and tracking in image sequences. In ISPRS Journal of Photogrammetry and Remote Sensing, 2017. | |||||||||||
| SVT 6. | 6.8 | 34.2 ±15.2 | 0.0 | 11.2% | 25.4% | 3,057 | 7,454 | 532 (9.6) | 611 (11.0) | 1.9 | Public |
| Longyin Wen, Zhen Lei, Ming-Ching Chang, Honggang Qi, Siwei Lyu. Multi-Camera Multi-Target Tracking with Space-Time-View Hyper-graph. IJCV, 2016. | |||||||||||
| AMIR3D 7. | 7.1 | 25.0 ±10.8 | 0.0 | 3.0% | 27.6% | 2,038 | 9,084 | 1,462 (31.9) | 1,647 (35.9) | 1.2 | Public |
| A. Sadeghian, A. Alahi, S. Savarese. Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies. In ICCV, 2017. | |||||||||||
| MCFPHD 8. | 4.8 | 39.9 ±12.3 | 0.0 | 25.7% | 16.8% | 3,029 | 6,700 | 363 (6.0) | 529 (8.8) | 17.7 | Public |
| N. Wojke, D. Paulus. Global data association for the Probability Hypothesis Density filter using network flows. In 2016 IEEE International Conference on Robotics and Automation, ICRA, 2016. | |||||||||||
| MCG 9. | 6.2 | 35.9 ±7.5 | 31.9 | 8.2% | 25.7% | 1,600 | 8,464 | 692 (14.0) | 1,017 (20.5) | 0.1 | Public |
| Anonymous submission | |||||||||||
| GustavHX 10. | 3.8 | 42.5 ±0.2 | 45.0 | 25.7% | 15.7% | 2,735 | 6,623 | 302 (5.0) | 431 (7.1) | 0.0 | Public |
| Anonymous submission | |||||||||||
| Tracker | Avg Rank | MOTA | IDF1 | MT | ML | FP | FN | ID Sw. | Frag | Hz | Detector |
| MOANA 11. | 3.2 | 52.7 ±14.4 | 62.4 | 28.4% | 22.0% | 2,226 | 5,551 | 167 (2.5) | 586 (8.8) | 19.4 | Public |
| Z. Tang, J. Hwang. MOANA: An Online Learned Adaptive Appearance Model for Robust Multiple Object Tracking in 3D. In IEEE Access (submitted), 2018. | |||||||||||
Benchmark Statistics
| Sequences | Frames | Trajectories | Boxes |
| 2 | 886 | 268 | 16789 |
Difficulty Analysis
Sequence difficulty (from easiest to hardest, measured by average MOTA)
PETS09-S2L2
(40.2% MOTA)
AVG-TownCentre
(23.4% MOTA)
Evaluation Measures
Lower is better. Higher is better.| Measure | Better | Perfect | Description |
| Avg Rank | lower | 1 | This is the rank of each tracker averaged over all present evaluation measures. |
| MOTA | higher | 100 % | Multiple Object Tracking Accuracy [1]. This measure combines three error sources: false positives, missed targets and identity switches. |
| MOTP | higher | 100 % | Multiple Object Tracking Precision [1]. The misalignment between the annotated and the predicted bounding boxes. |
| IDF1 | higher | 100 % | ID F1 Score [2]. The ratio of correctly identified detections over the average number of ground-truth and computed detections. |
| FAF | lower | 0 | The average number of false alarms per frame. |
| MT | higher | 100 % | Mostly tracked targets. The ratio of ground-truth trajectories that are covered by a track hypothesis for at least 80% of their respective life span. |
| ML | lower | 0 % | Mostly lost targets. The ratio of ground-truth trajectories that are covered by a track hypothesis for at most 20% of their respective life span. |
| FP | lower | 0 | The total number of false positives. |
| FN | lower | 0 | The total number of false negatives (missed targets). |
| ID Sw. | lower | 0 | The total number of identity switches. Please note that we follow the stricter definition of identity switches as described in [3]. |
| Frag | lower | 0 | The total number of times a trajectory is fragmented (i.e. interrupted during tracking). |
| Hz | higher | Inf. | Processing speed (in frames per second excluding the detector) on the benchmark. |
Legend
| Symbol | Description |
|
This is an online (causal) method, i.e. the solution is immediately available with each incoming frame and cannot be changed at any later time. |
|
This method used the provided detection set as input. |
 |
This entry has been submitted or updated less than a week ago. |
References:
| [1] | Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. Image and Video Processing, 2008(1):1-10, 2008. |
| [2] | Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. In ECCV workshop on Benchmarking Multi-Target Tracking, 2016. |
| [3] | Learning to associate: HybridBoosted multi-target tracker for crowded scene. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2009. |
