Affiliation:
1. City University of Hong Kong, China
2. City University of Hong Kong, China and Monash University, Australia
Abstract
Scalable Vector Graphics (SVG) is a popular vector image format that offers good support for interactivity and animation. Despite its appealing characteristics, creating custom SVG content can be challenging for users due to the steep learning curve required to understand SVG grammars or get familiar with professional editing software. Recent advancements in text-to-image generation have inspired researchers to explore vector graphics synthesis using either image-based methods (i.e., text → raster image → vector graphics) combining text-to-image generation models with image vectorization, or language-based methods (i.e., text → vector graphics script) through pretrained large language models. Nevertheless, these methods suffer from limitations in terms of generation quality, diversity, and flexibility. In this paper, we introduce IconShop, a text-guided vector icon synthesis method using autoregressive transformers. The key to success of our approach is to sequentialize and tokenize SVG paths (and textual descriptions as guidance) into a uniquely decodable token sequence. With that, we are able to exploit the sequence learning power of autoregressive transformers, while enabling both unconditional and text-conditioned icon synthesis. Through standard training to predict the next token on a large-scale vector icon dataset accompanied by textural descriptions, the proposed IconShop consistently exhibits better icon synthesis capability than existing image-based and language-based methods both quantitatively (using the FID and CLIP scores) and qualitatively (through formal subjective user studies). Meanwhile, we observe a dramatic improvement in generation diversity, which is validated by the objective Uniqueness and Novelty measures. More importantly, we demonstrate the flexibility of IconShop with multiple novel icon synthesis tasks, including icon editing, icon interpolation, icon semantic combination, and icon design auto-suggestion.
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Graphics and Computer-Aided Design
Reference60 articles.
1. Armen Aghajanyan Bernie Huang Candace Ross Vladimir Karpukhin Hu Xu Naman Goyal Dmytro Okhonko Mandar Joshi Gargi Ghosh Mike Lewis and Luke Zettlemoyer. 2022. CM3: A Causal Masked Multimodal Model of the Internet. arXiv preprint arXiv:2201.07520. Armen Aghajanyan Bernie Huang Candace Ross Vladimir Karpukhin Hu Xu Naman Goyal Dmytro Okhonko Mandar Joshi Gargi Ghosh Mike Lewis and Luke Zettlemoyer. 2022. CM3: A Causal Masked Multimodal Model of the Internet. arXiv preprint arXiv:2201.07520.
2. Haruka Aoki and Kiyoharu Aizawa . 2022 . SVG Vector Font Generation for Chinese Characters with Transformer. In IEEE International Conference on Image Processing. 646--650 . Haruka Aoki and Kiyoharu Aizawa. 2022. SVG Vector Font Generation for Chinese Characters with Transformer. In IEEE International Conference on Image Processing. 646--650.
3. Mohammad Bavarian Heewoo Jun Nikolas Tezak John Schulman Christine McLeavey Jerry Tworek and Mark Chen. 2022. Efficient Training of Language Models to Fill in the Middle. arXiv preprint arXiv:2207.14255. Mohammad Bavarian Heewoo Jun Nikolas Tezak John Schulman Christine McLeavey Jerry Tworek and Mark Chen. 2022. Efficient Training of Language Models to Fill in the Middle. arXiv preprint arXiv:2207.14255.
4. Automatic and interactive evolution of vector graphics images with genetic algorithms
5. Tim Brooks , Aleksander Holynski , and Alexei A . Efros . 2022 . InstructPix2Pix: Learning to Follow Image Editing Instructions . arXiv preprint arXiv:2211.09800. Tim Brooks, Aleksander Holynski, and Alexei A. Efros. 2022. InstructPix2Pix: Learning to Follow Image Editing Instructions. arXiv preprint arXiv:2211.09800.
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献
1. Better Little People Pictures: Generative Creation of Demographically Diverse Anthropographics;Proceedings of the CHI Conference on Human Factors in Computing Systems;2024-05-11
2. Sketch Video Synthesis;Computer Graphics Forum;2024-04-30