MPNTrack: Learning a Neural Solver for Multiple Object Tracking

TUD-Crossing


Benchmark:

Short name:

MPNTrack

Detector:

Public

Description:

Graphs offer a natural way to formulate Multiple Object Tracking (MOT) within the tracking-by-detection paradigm. However, they also introduce a major challenge for learning methods, as defining a model that can operate on such a structured domain is not trivial. As a consequence, most learning-based work has been devoted to learning better features for MOT, and then using these with well-established optimization frameworks. In this work, we exploit the classical network flow formulation of MOT to define a fully differentiable framework based on Message Passing Networks (MPNs). By operating directly on the graph domain, our method can reason globally over an entire set of detections and predict final solutions. Hence, we show that learning in MOT does not need to be restricted to feature extraction, but it can also be applied to the data association step. We show a significant improvement in both MOTA and IDF1 on three publicly available benchmarks.

Reference:

G. Brasó, L. Leal-Taixé. Learning a Neural Solver for Multiple Object Tracking. In CVPR, 2020.

Last submitted:

April 16, 2020 (2 months ago)

Published:

April 17, 2020 at 10:41:55 CET

Submissions:

1

Open source:

Yes

Hardware:

NVIDIA Quadro P5000

Runtime:

6.5 Hz

Benchmark performance:

Sequence MOTA IDF1 MOTP MT ML FP FN Recall Precision FAF ID Sw. Frag
2D MOT 201551.558.676.0225 (31.2)187 (25.9)7,62021,78064.683.91.3375 (5.8)872 (13.5)

Detailed performance:

Sequence MOTA IDF1 MOTP MT ML FP FN Recall Precision FAF ID Sw. Frag
ADL-Rundle-133.353.974.01533,0253,14366.267.16.142111
ADL-Rundle-355.961.881.61981,0013,45466.087.01.63235
AVG-TownCentre60.462.571.586374392,33567.391.61.058222
ETH-Crossing52.162.582.67106141658.590.60.336
ETH-Jelmoli60.472.680.3181337262675.383.70.8730
ETH-Linthescher49.158.580.144966923,82657.288.10.62473
KITTI-1655.569.570.92110763662.690.90.51433
KITTI-1949.162.771.314145992,08860.984.50.634102
PETS09-S2L255.243.774.8625833,59362.791.21.3147238
TUD-Crossing80.762.773.7704515785.895.50.21113
Venice-151.767.674.3736961,50667.081.51.539

Raw data: