A study on image and surface restoration

Abstract

Restoration is an important technology in both image processing and surface recon- struction. How to maintain meaningful information while eliminating undesired noises or details remains challenging. In this thesis, we present approaches to tackle the problem in terms of removing noises from point sets, retrieving structure from images, and suppressing moiré artefacts from moiré photos. We first demonstrate how to apply an L0 optimization directly to point clouds, which produces sparser solutions and sharper surfaces than either the L1 or L2 norm. In computer graphics, the input to a 3D reconstruction algorithm is com- monly a point set acquired from the object in question. However, despite new methods and acquisition hardware, errors such as noise and outliers inevitably appear in these point sets. Yet denoising point sets is inherently a challenging problem since, by definition, there is no connectivity information to guide the denoising process. Observing that many common objects are piecewise smooth, we explicitly take advantage of L0 sparsity to optimize for such a a surface. Our method can faithfully recover sharp features while at the same time smoothing the remaining regions even in the presence of large amounts of noise. Then, based on the characteristics of L0 norm, we further extend it to retrieve image structure, which is inherently sparse and piece-wise smooth comparing to details/texture. Retrieving salient structure from textured images is an important but difficult problem in computer vision because texture, which can be irregular, anisotropic, non-uniform and complex, shares many of the same properties as structure. We present a method to retrieve such structures using an L0 minimization of a modified form of the relative total variation metric. Thanks to the characteristics shared by texture and small structures, our method is effective at retrieving structure based on scale as well. Our method outperforms state-of-art methods in texture removal as well as scale space filtering. We also demonstrate our method’s ability in other applications such as edge detection, clip art compression artefact removal, and inverse half-toning. Finally, we present a novel multiresolution fully convolutional network for automatically removing moiré artefacts from photos, which remains a common and complex problem when digital screens are taken by cameras. This moiré phenomenon is a result of the interference between the pixel grid of the camera sensor and device screen, severely damaging the visual quality of photos. Since a moiré pattern spans over a wide range of frequencies, out deep network performs a nonlinear multiresolution analysis of the input image before computing how to cancel moiré artefacts within every frequency band. We also create a large-scale benchmark dataset with 150,000 image pairs for the investigation and evaluation of moiré pattern removal algorithms. Our deep network achieves state-of-the-art performance on this dataset in comparison to existing deep learning architectures for image restoration problems.