Hybrid dual stream blender for wide baseline view synthesis - Archive ouverte HAL Access content directly
Journal Articles Signal Processing: Image Communication Year : 2021

Hybrid dual stream blender for wide baseline view synthesis

(1, 2) , (3) , (4, 5, 6) , (1, 4, 5, 6) , (2, 7)
1
2
3
4
5
6
7

Abstract

Free navigation of a scene requires warping some reference views to some desired target viewpoint and blending them to synthesize a virtual view. Convolutional Neural Networks (ConvNets) based methods can learn both the warping and blending tasks jointly. Such methods are often designed for moderate inter-camera baseline distance and larger kernels are required for warping if the baseline distance increases. Algorithmic methods can in principle deal with large baselines, however the synthesized view suffers from artifacts near disoccluded pixels. We present a hybrid approach where first, reference views are algorithmically warped to the target position and then are blended via a ConvNet. Preliminary view warping allows reducing the size of the convolutional kernels and thus the learnable parameters count. We propose a residual encoder-decoder for image blending with a Siamese encoder to further keep the parameters count low. We also contribute a hole inpainting algorithm to fill the disocclusions in the warped views. Our view synthesis experiments on real multiview sequences show better objective image quality than state-of-the-art methods due to fewer artifacts in the synthesized images.
Fichier principal
Vignette du fichier
Hobloss et al. - 2021 - Hybrid dual stream blender for wide baseline view .pdf (16.9 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03330264 , version 1 (06-09-2021)

Identifiers

Cite

Nour Hobloss, Lu Zhang, Stephane Lathuiliere, Marco Cagnazzo, Attilio Fiandrotti. Hybrid dual stream blender for wide baseline view synthesis. Signal Processing: Image Communication, 2021, 97, pp.116366. ⟨10.1016/j.image.2021.116366⟩. ⟨hal-03330264⟩
93 View
7 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More