Automatic 3D Modeling and Editing of Immersive Indoor Environments from a Single Omnidirectional Image

Abstract

Over the past few years, there has been significant research interest in the automatic 3D reconstruction and modeling of indoor scenes, resulting in a well-defined emerging field. Within this context, 360-degree panoramic acquisition has emerged as an effective solution for indoor environments. It offers rapid and comprehensive coverage, even from a single viewpoint, and is compatible with a wide range of professional and consumer acquisition devices, making indoor data capture efficient and cost-effective. Panoramic images have also become integral to creating immersive content directly from real-world scenes and supporting various Virtual Reality applications. To fully support immersion, a system must also respond to viewpoint translation. While many solutions have been proposed for multiview capture setups, performing view synthesis from single-shot panoramas is of primary importance, due to the convenience and diffusion of sparse capturing through monocular 360-degree cameras. However, view synthesis relies on estimating the geometric model of the imaged environment, explicitly or implicitly, to perform occlusion-aware reprojection and synthesize disoccluded content. This aspect is even more crucial if the purpose is also to derive other non-obvious information from the original view, such as a model of the permanent structure without clutter. To achieve immersive visualization and effective editing in indoor 3D reconstruction, this thesis addresses several fundamental research questions related to depth and layout estimation and novel view synthesis, extending the state of the art in these tasks and in their combination aimed at indoor immersive exploration and editing, starting from a single 360-degree image.