Continuous Image Outpainting with Neural ODE

Abstract

Generalised image outpainting is an important and active research topic in computer vision, which aims to extend appealing content all-side around a given image. Existing state-of-the-art outpainting methods often rely on discrete extrapolation to extend the feature map in the bottleneck. They thus suffer from content unsmoothness, especially in circumstances where the outlines of objects in the extrapolated regions are incoherent with the input sub-images. To mitigate this issue, we design a novel bottleneck with Neural ODEs to make continuous extrapolation in latent space, which could be a plug-in for many deep learning frameworks. Our ODE-based network continuously transforms the state and makes accurate predictions by learning the incremental relationship among latent points, leading to both smooth and structured feature representation. Experimental results on three real-world datasets both applied on transformer-based and CNN-based frameworks show that our methods could generate more realistic and coherent images against the state-of-the-art image outpainting approaches. Our code is available at https://github.com/PengleiGao/Continuous-Image-Outpainting-with-Neural-ODE .