Learning to Track: Online Multi-Object Tracking by Decision Making

TUD-Crossing PETS09-S2L2 ETH-Jelmoli ETH-Linthescher ETH-Crossing AVG-TownCentre ADL-Rundle-1 ADL-Rundle-3 KITTI-16 KITTI-19

Short name:

MDP

Benchmark:

Description:

Online Multi-Object Tracking (MOT) has wide applications in time-critical video analysis scenarios, such as robot navigation and autonomous driving. In tracking-by-detection, a major challenge of online MOT is how to robustly associate noisy object detections on a new video frame with previously tracked objects. In this work, we formulate the online MOT problem as decision making in Markov Decision Processes (MDPs), where the lifetime of an object is modeled with a MDP. Learning a similarity function for data association is equivalent to learning a policy for the MDP, and the policy learning is approached in a reinforcement learning fashion which benefits from both advantages of offline-learning and online-learning for data association. Moreover, our framework can naturally handle the birth/death and appearance/disappearance of targets by treating them as state transitions in the MDP while leveraging existing online single object tracking methods. We conduct experiments on the MOT Benchmark to verify the effectiveness of our method.

Hardware:

3.5 Ghz, 8 cores

Detector:

Public

Processing:

Online

Last submitted:

April 20, 2015 (2 years ago)

Published:

March 31, 2015 at 12:53:29 CET

Submissions:

5

Open source:

Yes

Project page / code:

Reference:

Y. Xiang, A. Alahi, S. Savarese. Learning to Track: Online Multi-Object Tracking by Decision Making. In International Conference on Computer Vision (ICCV), 2015.

Benchmark performance:

MOTAMOTPFAFMTMLFPFNID Sw.FragSpecificationsDetector
30.371.31.713.0 % 38.4 % 9,71732,4226801,5003.5 Ghz, 8 coresPublic
IDF1ID PrecisionID Recall
44.757.836.4

Detailed performance:

Sequence MOTA IDF1 MOTP FAF GT MT ML FP FN ID Sw Frag
TUD-Crossing69.467.473.90.11353.8 % 7.7 % 24305825
PETS09-S2L247.838.569.81.54214.3 % 7.1 % 6614,163206362
ETH-Jelmoli32.959.073.61.54517.8 % 28.9 % 6391,0412271
ETH-Linthescher27.236.474.70.21976.1 % 64.0 % 1916,26248107
ETH-Crossing28.847.174.70.32611.5 % 46.2 % 59655015
AVG-TownCentre25.447.569.73.422617.7 % 33.6 % 1,5173,691122264
ADL-Rundle-116.247.771.56.33225.0 % 28.1 % 3,1574,59749140
ADL-Rundle-334.846.573.12.04411.4 % 29.5 % 1,2245,32678114
KITTI-1640.455.073.01.0170.0 % 17.6 % 2047753466
KITTI-1926.647.265.91.1626.5 % 22.6 % 1,1982,65866242
Venice-115.932.172.41.9175.9 % 41.2 % 8432,9494794

Raw data:


TC
MDP