Towards object detection in the real world

Abstract

Seeing, recognizing and understanding objects in real world is the critical path for computer vision systems to be applied in human life and practical applications. Object detection is one of the most fundamental techniques. Towards object detection in real world, there are various challenges on its generalization, efficiency and robustness. Addressing these challenges requires systematic innovation in network architecture, training datasets, and learning algorithms.

In this dissertation, we study three important problems related to universal object detection, including (i) end-to-end object detection network architecture, (ii) object tracking in the video and (iii) object recognition in the open world.

For end-to-end object detection, we present a systematic analysis on its learning mechanism and introduce two advanced network architecture, Sparse R-CNN and OneNet, for end-to-end object detection. These methods exhibit competitive performance of accuracy, inference time and training convergence on the challenging object detection datasets, especially on crowded scenes.

For object tracking in the video, we introduce a unified framework for object detection and tracking, TransTrack, by solving these two tasks from a unified query-key perspective. Furthermore, we propose DanceTrack, a new dataset where objects are in uniform appearance and diverse motion, to inspire tracking algorithms that rely on both visual discrimination and motion analysis.

For object recognition in the open world, we demonstrate the hierarchical structure of visual concepts from the perspectives of open category and open granularity. We introduce VLPart, a detector with the ability to predict both open-vocabulary objects and their part segmentation.