TRCKFRMER_PR: "TrackFormer: Multi-Object Tracking with Transformers" with private detections

MOT17-01-DPM


Benchmark:

MOT17 | MOT20 |

Short name:

TRCKFRMER_PR

Detector:

Private

Description:

The challenging task of multi-object tracking (MOT) requires simultaneous reasoning about track initialization, identity, and spatiotemporal trajectories. We formulate this task as a frame-to-frame set prediction problem and introduce TrackFormer, an end-to-end MOT approach based on an encoder-decoder Transformer architecture. Our model achieves data association between frames via attention by evolving a set of track predictions through a video sequence. The Transformer decoder initializes new tracks from static object queries and autoregressively follows existing tracks in space and time with the new concept of identity preserving track queries. Both decoder query types benefit from self- and encoder-decoder attention on global frame-level features, thereby omitting any additional graph optimization and matching or modeling of motion and appearance. TrackFormer represents a new tracking-by-attention paradigm and yields state-of-the-art performance on the task of multi-object tracking (MOT17) and segmentation (MOTS20).

Reference:

T. Meinhardt, A. Kirillov, L. Leal-Taixe, C. Feichtenhofer. TrackFormer: Multi-Object Tracking with Transformers. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

Last submitted:

April 29, 2022 (2 months ago)

Published:

April 29, 2022 at 11:04:08 CET

Submissions:

1

Open source:

Yes

Hardware:

7 x 32 GB GPUs

Runtime:

5.7 Hz

Benchmark performance:

Sequence MOTA IDF1 HOTA MT ML FP FN Rcll Prcn AssA DetA AssRe AssPr DetRe DetPr LocA FAF ID Sw. Frag
MOT1774.168.057.31,113 (47.3)246 (10.4)34,602108,77780.792.954.160.958.078.866.876.982.81.92,829 (0.0)4,221 (0.0)

Detailed performance:

Sequence MOTA IDF1 HOTA MT ML FP FN Rcll Prcn AssA DetA AssRe AssPr DetRe DetPr LocA FAF ID Sw. Frag
MOT17-01-DPM57.949.744.21144772,19166.089.940.948.242.981.153.572.982.11.14570
MOT17-01-FRCNN57.949.744.21144772,19166.089.940.948.242.981.153.572.982.11.14570
MOT17-01-SDP57.949.744.21144772,19166.089.940.948.242.981.153.572.982.11.14570
MOT17-03-DPM88.679.666.312432,4699,36591.197.561.971.365.681.375.781.083.31.6122316
MOT17-03-FRCNN88.679.666.312432,4699,36591.197.561.971.365.681.375.781.083.31.6122316
MOT17-03-SDP88.679.666.312432,4699,36591.197.561.971.365.681.375.781.083.31.6122316
MOT17-06-DPM59.860.851.0104271,7912,77576.583.448.154.455.076.064.470.383.81.5173194
MOT17-06-FRCNN59.860.851.0104271,7912,77576.583.448.154.455.076.064.470.383.81.5173194
MOT17-06-SDP59.860.851.0104271,7912,77576.583.448.154.455.076.064.470.383.81.5173194
MOT17-07-DPM65.649.542.82451,0304,67172.392.234.254.038.071.459.175.381.92.1118172
MOT17-07-FRCNN65.649.542.82451,0304,67172.392.234.254.038.071.459.175.381.92.1118172
MOT17-07-SDP65.649.542.82451,0304,67172.392.234.254.038.071.459.175.381.92.1118172
MOT17-08-DPM54.542.539.22491,4617,86162.890.133.047.536.171.151.874.481.52.3279320
MOT17-08-FRCNN54.542.539.22491,4617,86162.890.133.047.536.171.151.874.481.52.3279320
MOT17-08-SDP54.542.539.22491,4617,86162.890.133.047.536.171.151.874.481.52.3279320
MOT17-12-DPM51.863.053.543141,8802,25873.977.357.150.462.679.963.065.884.62.14287
MOT17-12-FRCNN51.863.053.543141,8802,25873.977.357.150.462.679.963.065.884.62.14287
MOT17-12-SDP51.863.053.543141,8802,25873.977.357.150.462.679.963.065.884.62.14287
MOT17-14-DPM47.454.941.341202,4267,13861.482.441.142.244.373.848.865.579.73.2164248
MOT17-14-FRCNN47.454.941.341202,4267,13861.482.441.142.244.373.848.865.579.73.2164248
MOT17-14-SDP47.454.941.341202,4267,13861.482.441.142.244.373.848.865.579.73.2164248

Raw data:


TRCKFRMER_PR