RNN Tracker

TUD-Crossing PETS09-S2L2 ETH-Jelmoli ETH-Linthescher ETH-Crossing AVG-TownCentre ADL-Rundle-1 ADL-Rundle-3 KITTI-16 KITTI-19

Short name:

RNN_LSTM

Benchmark:

Description:

We present a novel approach to online multi-target tracking based on
recurrent neural networks (RNNs). Tracking multiple objects in
real-world scenes involves many challenges, including a) an a-priori
unknown and time-varying number of targets, b) a continuous state
estimation of all present targets, and c) a discrete combinatorial
problem of data association. Most previous methods involve complex
models that require tedious tuning of parameters. Here, we propose for
the first time, a full end-to-end learning approach for online
multi-target tracking based on deep learning. Existing deep learning
methods are not designed for the above challenges and cannot be
trivially applied to the task. Our solution addresses all of the above
points in a principled way. Experiments on both synthetic and real data
show competitive results obtained at 300 Hz on a standard CPU, and pave
the way towards future research in this direction.

Hardware:

3 GHz, 1 CPU

Detector:

Public

Processing:

Online

Last submitted:

March 15, 2016 (1 year ago)

Published:

March 15, 2016 at 08:24:33 CET

Submissions:

2

Open source:

Yes

Project page / code:

Reference:

A. Milan, S. Rezatofighi, A. Dick, I. Reid, K. Schindler. Online Multi-Target Tracking using Recurrent Neural Networks. In AAAI, 2017.

Benchmark performance:

MOTAMOTPFAFMTMLFPFNID Sw.FragSpecificationsDetector
19.071.02.05.5 % 45.6 % 11,57836,7061,4902,0813 GHz, 1 CPUPublic
IDF1ID PrecisionID Recall
17.123.013.6

Detailed performance:

Sequence MOTA IDF1 MOTP FAF GT MT ML FP FN ID Sw Frag
TUD-Crossing57.234.671.70.41330.8 % 15.4 % 813484349
PETS09-S2L238.318.671.62.3429.5 % 14.3 % 1,0164,611320417
ETH-Jelmoli34.816.673.30.74517.8 % 28.9 % 3141,2805986
ETH-Linthescher12.44.774.70.11971.5 % 79.7 % 1647,60949102
ETH-Crossing21.116.175.50.1260.0 % 57.7 % 27757716
AVG-TownCentre13.46.968.82.72263.5 % 41.2 % 1,2064,682299414
ADL-Rundle-1-2.220.669.98.43218.8 % 25.0 % 4,2135,058241336
ADL-Rundle-323.723.772.03.5446.8 % 20.5 % 2,1935,407158189
KITTI-1626.327.268.51.4170.0 % 11.8 % 29089667116
KITTI-1917.711.568.31.3626.5 % 25.8 % 1,3882,818191242
Venice-112.722.771.71.5170.0 % 47.1 % 6863,24056114

Raw data:


RNN_LSTM