MuRF: Multi-Baseline Radiance Fields

Haofei Xu1,2     Anpei Chen1,2     Yuedong Chen3     Christos Sakaridis1     Yulun Zhang1    
Marc Pollefeys1     Andreas Geiger2,†     Fisher Yu1,†    

CVPR 2024

1ETH Zurich     2University of Tübingen, Tübingen AI Center     3Monash University     †Joint last author    
Paper Code

MuRF supports multiple different baseline settings.

Feed-forward novel view synthesis on unseen scenes from 2 input views.

TL;DR

  • A general feed-forward approach to solving sparse view synthesis under multiple different baseline settings (small and large baselines, and different number of input views).
  • A target view frustum volume representation and a convolutional radiance field decoder enable high-quality novel view synthesis.
  • State-of-the-art performance across multiple different baseline settings and diverse datasets (DTU, RealEstate10K, LLFF, and Mip-NeRF 360 dataset).

Overview

Multi-view image encoder + target view frustum volume + convolutional radiance decoder ⇒ high-quality rendering

Comparisons across Multiple Baselines

Previous methods are specifically designed for either small (e.g., ENeRF) or large (e.g., AttnRend) baselines. However, no existing method performs well on both. See the visual comparisons on DTU here and RealEstate10K here.

System-Level Comparisons

MuRF achieves state-of-the-art performance under various evaluation settings.

DTU

Feed-forward novel view synthesis on unseen scenes from 2 input views.


Feed-forward novel view synthesis on unseen scenes from 3 input views.


RealEstate10K

Feed-forward novel view synthesis on unseen scenes from 2 input views.

Acknowledgements

We thank Shaofei Wang, Zehao Yu and Stefano Esposito for the insightful comments on the early draft of this work. We thank Tobias Fischer and Kashyap Chitta for the constructive discussions. We thank Takeru Miyato for the help with the RealEstate10K dataset. Andreas Geiger was supported by the ERC Starting Grant LEGO-3D (850533) and the DFG EXC number 2064/1 - project number 390727645.

BibTeX

@inproceedings{xu2024murf,
      title={MuRF: Multi-Baseline Radiance Fields},
      author={Xu, Haofei and Chen, Anpei and Chen, Yuedong and Sakaridis, Christos and Zhang, Yulun and Pollefeys, Marc and Geiger, Andreas and Yu, Fisher},
      booktitle={CVPR},
      year={2024}
    }

awesome webpage template