GECO: Generative Image-to-3D within a SECOnd

Arxiv 2024

Chen Wang1, Jiatao Gu2, Xiaoxiao Long3, Yuan Liu3, Lingjie Liu1

1University of Pennsylvania    2Apple     3The University of Hong Kong     


TL;DR: We propose a 3D distillation pipeline to train a feedforward 3D generative model that operates in less than one second.

3D generation has seen remarkable progress in recent years. Existing techniques, such as score distillation methods, produce notable results but require extensive per-scene optimization, impacting time efficiency. Alternatively, reconstruction-based approaches prioritize efficiency but compromise the quality due to their limited handling of uncertainty. We introduce GECO, a novel method for high-quality 3D generative modeling that operates within a second. Our approach addresses the prevalent issues of uncertainty and inefficiency in current methods through a two-stage approach. In the initial stage, we train a single-step multi-view generative model with score distillation. Then, a second-stage distillation is applied to address the challenge of view inconsistency from the multi-view prediction. This two-stage process ensures a balanced approach to 3D generation, optimizing both quality and efficiency. Our comprehensive experiments demonstrate that GECO achieves high-quality image-to-3D generation with an unprecedented level of efficiency.

Image to 3D

GECO can achieve great results on image-to-3D generation for both GSO dataset (top row) and in the wild data (bottom row).

Comparison over Reconstruction-based Methods

GECO can handle the back-view of the object thanks to uncertainty modeling.


  title={GECO: Generative Image-to-3D within a Second},
  author={Wang, Chen and Gu, Jiatao and Long, Xiaoxiao and Liu, Yuan and Liu, Lingjie},