Affiliation:
1. NVIDIA, Canada
2. Asana, Canada
3. NVIDIA, Canada, University of Toronto, Canada, and Vector Institute, Canada
Abstract
We propose Neural Brushstroke Engine, the first method to apply deep generative models to learn a distribution of interactive drawing tools. Our conditional GAN model learns the latent space of drawing styles from a small set (about 200) of unlabeled images in different media. Once trained, a single model can texturize stroke patches drawn by the artist, emulating a diverse collection of brush styles in the latent space. In order to enable interactive painting on a canvas of arbitrary size, we design a painting engine able to support real-time seamless patch-based generation, while allowing artists direct control of stroke shape, color and thickness. We show that the latent space learned by our model generalizes to unseen drawing and more experimental styles (e.g. beads) by embedding real styles into the latent space. We explore other applications of the continuous latent space, such as optimizing brushes to enable painting in the style of an existing artwork, automatic line drawing stylization, brush interpolation, and even natural language search over a continuous space of drawing tools. Our prototype received positive feedback from a small group of digital artists.
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Graphics and Computer-Aided Design
Reference98 articles.
1. Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?
2. Image2StyleGAN++: How to Edit the Embedded Images?
3. Rameen Abdal , Peihao Zhu , John Femiani , Niloy J Mitra , and Peter Wonka . 2021a. CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions. arXiv preprint arXiv:2112.05219 ( 2021 ). Rameen Abdal, Peihao Zhu, John Femiani, Niloy J Mitra, and Peter Wonka. 2021a. CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions. arXiv preprint arXiv:2112.05219 (2021).
4. Labels4Free: Unsupervised Segmentation using StyleGAN
5. StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献