DukeMTMCT Results

Click on a measure to sort the table accordingly. See below for a more detailed description.


Easy Test Set

Single Camera (all)

Tracker IDF1 IDP IDR MOTA MOTP FAF MT ML FP FN ID Sw. Frag
MTMC_CDSC
1. using public detections
77.087.668.670.975.80.0574011038,655268,3986934,717
Y. Tesfaye, E. Zemene, A. Prati, M. Pelillo, M. Shah. Multi-Target Tracking in Multiple Non-Overlapping Cameras using Constrained Dominant Sets. In CoRR, 2017.
PT_BIPCC
2. using public detections
71.284.861.459.378.70.0966623468,634361,589290783
Anonymous submission
dirBIPCC
3. online method using public detections
70.083.260.459.078.70.1066523471,381361,673298799
Anonymous submission
BIPCC
4. online method using public detections
70.183.660.459.478.70.0966523468,147361,672300801
E. Ristani, F. Solera, R. Zou, R. Cucchiara, C. Tomasi. Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. In ECCV workshop on Benchmarking Multi-Target Tracking, 2016.
lx_b
5. online method using public detections
70.388.158.561.378.70.0464024726,845382,524246788
Anonymous submission

Multi-Camera

Tracker IDF1 IDP IDR
PT_BIPCC
1. using public detections
34.941.630.1
Anonymous submission
dirBIPCC
2. online method using public detections
52.162.045.0
Anonymous submission
BIPCC
3. online method using public detections
56.267.048.4
E. Ristani, F. Solera, R. Zou, R. Cucchiara, C. Tomasi. Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. In ECCV workshop on Benchmarking Multi-Target Tracking, 2016.
lx_b
4. online method using public detections
58.072.648.2
Anonymous submission
MTMC_CDSC
5. using public detections
60.068.353.5
Y. Tesfaye, E. Zemene, A. Prati, M. Pelillo, M. Shah. Multi-Target Tracking in Multiple Non-Overlapping Cameras using Constrained Dominant Sets. In CoRR, 2017.

Hard Test Set

Single Camera (all)

Tracker IDF1 IDP IDR MOTA MOTP FAF MT ML FP FN ID Sw. Frag
MTMC_CDSC
1. using public detections
65.581.454.759.675.40.093489926,643260,0731,6375,024
Y. Tesfaye, E. Zemene, A. Prati, M. Pelillo, M. Shah. Multi-Target Tracking in Multiple Non-Overlapping Cameras using Constrained Dominant Sets. In CoRR, 2017.
lx_b
2. online method using public detections
64.280.453.453.677.10.1633610745,370285,1926211,049
Anonymous submission
dirBIPCC
3. online method using public detections
64.380.353.653.977.10.1633810245,472283,0146531,078
Anonymous submission
PT_BIPCC
4. using public detections
65.081.854.054.477.10.1433510440,978283,7046611,054
Anonymous submission
BIPCC
5. online method using public detections
64.581.253.554.677.10.1433810339,599283,3766521,073
E. Ristani, F. Solera, R. Zou, R. Cucchiara, C. Tomasi. Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. In ECCV workshop on Benchmarking Multi-Target Tracking, 2016.

Multi-Camera

Tracker IDF1 IDP IDR
PT_BIPCC
1. using public detections
32.941.327.3
Anonymous submission
dirBIPCC
2. online method using public detections
45.056.337.5
Anonymous submission
BIPCC
3. online method using public detections
47.359.639.2
E. Ristani, F. Solera, R. Zou, R. Cucchiara, C. Tomasi. Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. In ECCV workshop on Benchmarking Multi-Target Tracking, 2016.
lx_b
4. online method using public detections
48.360.640.2
Anonymous submission
MTMC_CDSC
5. using public detections
50.963.242.6
Y. Tesfaye, E. Zemene, A. Prati, M. Pelillo, M. Shah. Multi-Target Tracking in Multiple Non-Overlapping Cameras using Constrained Dominant Sets. In CoRR, 2017.


Evaluation Measures

Lower is better. Higher is better.
Measure Better Perfect Description
IDF1 higher 100 % ID F1 Score [1]. The ratio of correctly identified detections over the average number of ground-truth and computed detections.
IDP higher 100 % ID Precision [1]. Identification precision is the fraction of computed detections that are correctly identified.
IDR higher 100 % ID Recall [1]. Identification recall is the fraction of ground truth detections that are correctly identified.
MOTA higher 100 % Multiple Object Tracking Accuracy [2]. This measure combines three error sources: false positives, missed targets and identity switches.
MOTP higher 100 % Multiple Object Tracking Precision [2]. The misalignment between the annotated and the predicted bounding boxes.
FAF lower 0 The average number of false alarms per frame.
MT higher 100 % Mostly tracked targets. The ratio of ground-truth trajectories that are covered by a track hypothesis for at least 80% of their respective life span.
ML lower 0 % Mostly lost targets. The ratio of ground-truth trajectories that are covered by a track hypothesis for at most 20% of their respective life span.
FP lower 0 The total number of false positives.
FN lower 0 The total number of false negatives (missed targets).
ID Sw. lower 0 The total number of identity switches. Please note that we follow the stricter definition of identity switches as described in [3].
Frag lower 0 The total number of times a trajectory is fragmented (i.e. interrupted during tracking).

Legend

Symbol Description
online method This is an online (causal) method, i.e. the solution is immediately available with each incoming frame and cannot be changed at any later time.
using public detections This method used the provided detection set as input.
new This entry has been submitted or updated less than a week ago.

References:


[1] Ristani, E., Solera, F., Zou, R., Cucchiara, R. & Tomasi, C. Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. In ECCV workshop on Benchmarking Multi-Target Tracking, 2016.
[2] Bernardin, K. & Stiefelhagen, R. Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. Image and Video Processing, 2008(1):1-10, 2008.
[3] Li, Y., Huang, C. & Nevatia, R. Learning to associate: HybridBoosted multi-target tracker for crowded scene. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2009.