1University of Pennsylvania 2Apple 3The University of Hong Kong
TL;DR: We propose a 3D distillation pipeline to train a feedforward 3D generative model that operates in less than one second.
3D generation has seen remarkable progress in recent years. Existing techniques, such as score distillation methods, produce notable results but require extensive per-scene optimization, impacting time efficiency. Alternatively, reconstruction-based approaches prioritize efficiency but compromise the quality due to their limited handling of uncertainty. We introduce GECO, a novel method for high-quality 3D generative modeling that operates within a second. Our approach addresses the prevalent issues of uncertainty and inefficiency in current methods through a two-stage approach. In the initial stage, we train a single-step multi-view generative model with score distillation. Then, a second-stage distillation is applied to address the challenge of view inconsistency from the multi-view prediction. This two-stage process ensures a balanced approach to 3D generation, optimizing both quality and efficiency. Our comprehensive experiments demonstrate that GECO achieves high-quality image-to-3D generation with an unprecedented level of efficiency.
GECO can achieve great results on image-to-3D generation for both GSO dataset (top row) and in the wild data (bottom row).
GECO can handle the back-view of the object thanks to uncertainty modeling.
@article{wang2024geco,
title={GECO: Generative Image-to-3D within a Second},
author={Wang, Chen and Gu, Jiatao and Long, Xiaoxiao and Liu, Yuan and Liu, Lingjie},
journal={arXiv},
year={2024}
}