Temporal Dynamic Appearance Modeling for Online Multi-person Tracking

TUD-Crossing PETS09-S2L2 ETH-Jelmoli ETH-Linthescher ETH-Crossing AVG-TownCentre ADL-Rundle-1 ADL-Rundle-3 KITTI-16 KITTI-19

Short name:

TDAM

Benchmark:

Description:

Robust online multi-person tracking requires the correct associations of online detection responses with existing trajectories. We address this problem by developing a novel appearance modeling approach to provide accurate appearance affinities to guide data association. In contrast to most existing algorithms that only consider the spatial structure of human appearances, we exploit the temporal dynamic characteristics within temporal appearance sequences to discriminate different persons. The temporal dynamic makes a sufficient complement to the spatial structure of varying appearances in the feature space, which significantly improves the affinity measurement between trajectories and detections. We propose a feature selection algorithm to describe the appearance variations with mid-level semantic features, and demonstrate its usefulness in terms of temporal dynamic appearance modeling. Moreover, the appearance model is learned incrementally by alternatively evaluating newly-observed appearances and adjusting the model parameters to be suitable for online tracking. Reliable tracking of multiple persons in complex scenes is achieved by incorporating the learned model into an online tracking-by-detection framework. Our experiments on the challenging benchmark MOTChallenge 2015 demonstrate that our method outperforms the state-of-the-art multi-person tracking algorithms.

Hardware:

3.4 GHZ, 1 Core

Detector:

Public

Processing:

Online

Last submitted:

August 26, 2015 (2 years ago)

Published:

April 21, 2015 at 11:05:50 CET

Submissions:

4

Open source:

No

Project page / code:

n/a

Reference:

M. Yang, Y. Jia. Temporal dynamic appearance modeling for online multi-person tracking. In Computer Vision and Image Understanding, 2016.

Benchmark performance:

MOTAMOTPFAFMTMLFPFNID Sw.FragSpecificationsDetector
33.072.81.713.3 % 39.1 % 10,06430,6174641,5063.4 GHZ, 1 CorePublic
IDF1ID PrecisionID Recall
46.157.638.3

Detailed performance:

Sequence MOTA IDF1 MOTP FAF GT MT ML FP FN ID Sw Frag
TUD-Crossing55.463.373.80.31323.1 % 7.7 % 52431830
PETS09-S2L243.140.569.41.5429.5 % 11.9 % 6534,673158412
ETH-Jelmoli41.260.872.40.64515.6 % 33.3 % 2741,2071159
ETH-Linthescher37.043.176.10.11978.1 % 63.5 % 1165,4733399
ETH-Crossing25.140.274.10.1267.7 % 61.5 % 18731211
AVG-TownCentre25.350.770.33.322615.0 % 39.4 % 1,4773,79468223
ADL-Rundle-129.247.472.05.63228.1 % 9.4 % 2,7813,74662230
ADL-Rundle-329.341.478.63.14411.4 % 18.2 % 1,9555,18448104
KITTI-1642.764.972.91.31729.4 % 17.6 % 2776851236
KITTI-1920.348.366.31.56211.3 % 22.6 % 1,5572,67133212
Venice-135.240.875.02.01723.5 % 17.6 % 9042,0222990

Raw data:


TDAM
DBN