Affiliation:
1. Simon Fraser University
2. Canada CIFAR AI Chair, Amii
Abstract
AbstractRecent years have seen an explosion of work and interest in text‐to‐3D shape generation. Much of the progress is driven by advances in 3D representations, large‐scale pretraining and representation learning for text and image data enabling generative AI models, and differentiable rendering. Computational systems that can perform text‐to‐3D shape generation have captivated the popular imagination as they enable non‐expert users to easily create 3D content directly from text. However, there are still many limitations and challenges remaining in this problem space. In this state‐of‐the‐art report, we provide a survey of the underlying technology and methods enabling text‐to‐3D shape generation to summarize the background literature. We then derive a systematic categorization of recent work on text‐to‐3D shape generation based on the type of supervision data required. Finally, we discuss limitations of the existing categories of methods, and delineate promising directions for future work.
Reference156 articles.
1. Achlioptas Panos Diamanti Olga Mitliagkas Ioannis andGuibas Leonidas. “Learning representations and generative models for 3D point clouds”.International conference on machine learning.2018 40–49. arXiv: 1707.02392 [cs.CV] 21.
2. Achlioptas Panos Fan Judy Hawkins Robert et al. “ShapeGlot: Learning language for shape differentiation”.Proceedings of the IEEE/CVF International Conference on Computer Vision.2019 8938–8947. arXiv: 1905.02925 [cs.CL] 5 6 8.
3. Achlioptas Panos Huang Ian Sung Minhyuk et al. “ShapeTalk: A Language Dataset and Framework for 3D Shape Edits and Deformations”.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023 12685–12694. doi:10.1109/CVPR52729.2023.012205.
4. Armandpour Mohammadreza Zheng Huangjie Sadeghian Ali et al. “Re‐imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D alleviate Janus problem and Beyond”.arXiv preprint arXiv:2304.04968(2023). arXiv: 2304.04968 [cs.CV] 14.