SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views

Abstract

<p>We introduce <em>SparseNeuS</em>, a novel neural rendering based method for the task of surface reconstruction from multi-view images. This task becomes more difficult when only sparse images are provided as input, a scenario where existing neural reconstruction approaches usually produce incomplete or distorted results. Moreover, their inability of generalizing to unseen new scenes impedes their application in practice. Contrarily, <em>SparseNeuS</em> can generalize to new scenes and work well with sparse images (as few as 2 or 3). <em>SparseNeuS</em> adopts signed distance function (SDF) as the surface representation, and learns generalizable priors from image features by introducing <em>geometry encoding</em> volumes for generic surface prediction. Moreover, several strategies are introduced to effectively leverage sparse views for high-quality reconstruction, including 1) a multi-level geometry reasoning framework to recover the surfaces in a coarse-to-fine manner; 2) a multi-scale color blending scheme for more reliable color prediction; 3) a consistency-aware fine-tuning scheme to control the inconsistent regions caused by occlusion and noise. Extensive experiments demonstrate that our approach not only outperforms the state-of-the-art methods, but also exhibits good efficiency, generalizability, and flexibility (Visit our project page: <a href="https://www.xxlong.site/SparseNeuS">https://www.xxlong.site/SparseNeuS</a>).</p>