Video object co-segmentation and video vectorization

Abstract

Video, as a rapidly growing multimedia data, has been increasingly affecting our life in expression communication and interactions with the outer world. Compared to traditional image or text, it has better ability to convey information due to its time-sequential nature which, however, makes it a challenging task to extract information from it or process it. To keep up with its rapid extension, the research community has endeavored to develop accompanying tools that assist users analyzing and processing videos. This thesis demonstrates two systems for video analysis and process, video object co-segmentation and video vectorization.

In the system of video object co-segmentation, we address an issue of co-segmenting the common foreground object from a group of video sequences based on that multiple videos may share a common foreground object, such as a family member in home videos or a leading role in various clips of a movie or TV series. We propose a novel co-segmentation algorithm for video by taking full advantage of its appearance and motion features. Our algorithm is well-designed so that it can be differentiated from the algorithms for other forms of data, e.g. image or geometric shapes. We compare our method with the existing related works and our approach outperforms state-of-the-art methods.

In the system of video vectorization, we study a vectorization method for videos. Vector-based graphical contents are being increasingly used in smartphones and computers, and becoming the main form of media on the Internet, demonstrated by the popularity of vectorized image editing tools such as Adobe Illustrator or CorelDraw. We realize that it would not work if simply applying existing image vectorization techniques to individual frames, because of the lack of consideration of temporal coherence between video frames that would cause unacceptable flickering. Consequently, we propose a method that treats the video as a spatial-temporal volume and uses 3D tetrahedral meshes for the vector-based representation. We present novel techniques for simplification and subdivision of a tetrahedral mesh to achieve high simplification ratio while preserving features and ensuring color fidelity. The proposed mesh simplification algorithm can be further applied to fast mesh generation for large-scale volumetric data with multiple labeled regions such as medical data. We also demonstrate the superiority of our approach by comparison with related works.