MPNTrack: Learning a Neural Solver for Multiple Object Tracking

MOT17-14-DPM


Rendering of new sequences is currently deactivated due to heavy load.

Rendering of new sequences is currently deactivated due to heavy load.

Rendering of new sequences is currently deactivated due to heavy load.

Benchmark:

Short name:

MPNTrack

Detector:

Public

Description:

Graphs offer a natural way to formulate Multiple Object Tracking (MOT) within the tracking-by-detection paradigm. However, they also introduce a major challenge for learning methods, as defining a model that can operate on such a structured domain is not trivial. As a consequence, most learning-based work has been devoted to learning better features for MOT, and then using these with well-established optimization frameworks. In this work, we exploit the classical network flow formulation of MOT to define a fully differentiable framework based on Message Passing Networks (MPNs). By operating directly on the graph domain, our method can reason globally over an entire set of detections and predict final solutions. Hence, we show that learning in MOT does not need to be restricted to feature extraction, but it can also be applied to the data association step. We show a significant improvement in both MOTA and IDF1 on three publicly available benchmarks.

Reference:

G. Braso, L. Leal-Taixe. Learning a Neural Solver for Multiple Object Tracking. In CVPR, 2020.

Last submitted:

April 16, 2020 (4 years ago)

Published:

April 17, 2020 at 10:43:15 CET

Submissions:

1

Open source:

No

Hardware:

NVIDIA Quadro P5000

Runtime:

6.5 Hz

Benchmark performance:

Sequence MOTA IDF1 HOTA MT ML FP FN Rcll Prcn AssA DetA AssRe AssPr DetRe DetPr LocA FAF ID Sw. Frag
MOT1758.861.749.0679 (28.8)788 (33.5)17,413213,59462.195.351.147.357.174.350.276.981.51.01,185 (19.1)2,265 (36.4)

Detailed performance:

Sequence MOTA IDF1 HOTA MT ML FP FN Rcll Prcn AssA DetA AssRe AssPr DetRe DetPr LocA FAF ID Sw. Frag
MOT17-01-DPM50.263.749.4811553,15751.198.464.138.170.776.539.776.580.10.1214
MOT17-01-FRCNN48.860.547.510103132,98153.891.758.338.865.473.541.871.479.60.7617
MOT17-01-SDP49.562.147.410103282,92554.791.557.239.465.372.642.671.279.50.7721
MOT17-03-DPM69.567.553.464181,74230,11971.297.752.554.656.777.557.578.881.81.2117186
MOT17-03-FRCNN70.568.753.661171,59529,19972.197.952.355.355.778.758.178.981.81.1113196
MOT17-03-SDP76.470.656.384152,18022,41878.697.453.759.459.075.262.978.081.21.5133237
MOT17-06-DPM55.658.948.174848764,32163.389.548.747.767.559.452.373.981.70.738101
MOT17-06-FRCNN58.060.248.987611,2903,60369.486.447.750.368.557.657.070.981.31.152132
MOT17-06-SDP58.462.349.592631,3533,50370.386.048.450.870.157.357.870.681.31.148130
MOT17-07-DPM45.651.039.98175148,61749.094.242.437.745.475.639.776.280.91.057118
MOT17-07-FRCNN44.750.839.69186718,60749.092.542.537.245.775.739.574.680.71.362120
MOT17-07-SDP47.752.240.911147728,00052.692.042.239.945.774.542.674.480.51.561134
MOT17-08-DPM32.141.837.8173847313,82234.693.951.228.057.971.529.179.283.80.84464
MOT17-08-FRCNN31.141.436.9163760513,91534.192.349.927.558.170.128.777.683.51.04359
MOT17-08-SDP32.441.037.0183680013,43336.490.647.229.257.266.130.776.483.11.35065
MOT17-12-DPM46.157.947.120403674,28650.592.355.939.962.274.942.577.683.50.41636
MOT17-12-FRCNN45.155.344.818413054,44348.793.352.038.860.969.041.078.483.50.31431
MOT17-12-SDP47.157.546.719433354,23151.293.054.040.463.171.343.078.083.50.41533
MOT17-14-DPM34.846.233.3137560911,36638.592.138.728.943.766.130.472.779.60.882141
MOT17-14-FRCNN35.648.235.220741,11410,68742.287.540.730.645.766.333.068.578.61.5108204
MOT17-14-SDP39.449.836.420661,1169,96146.188.440.033.345.665.136.069.178.71.5117226

Raw data: