hjc-owo commited on
Commit
b57e8bf
1 Parent(s): 966ae59

fix(python): README.md

Browse files
Files changed (2) hide show
  1. README copy.md +0 -304
  2. README.md +1 -0
README copy.md DELETED
@@ -1,304 +0,0 @@
1
- ---
2
- title: SVGRender
3
- emoji: 💻
4
- colorFrom: gray
5
- colorTo: yellow
6
- sdk: gradio
7
- sdk_version: 4.20.1
8
- python_version: 3.10.12
9
- app_file: app.py
10
- pinned: false
11
- license: apache-2.0
12
- ---
13
-
14
- <h1 id="ptsvg" align="center">Pytorch-SVGRender</h1>
15
-
16
- <p align="center">
17
- <a href="https://www.python.org/"><img src="https://img.shields.io/badge/python-3.10-or?logo=python" alt="pyhton"></a>
18
- <a href="http://mozilla.org/MPL/2.0/"><img src="https://img.shields.io/badge/license-MPL2.0-orange" alt="license"></a>
19
- <a href="https://ximinng.github.io/PyTorch-SVGRender-project/"><img src="https://img.shields.io/badge/website-Gitpage-yellow" alt="website"></a>
20
- <a href="https://pytorch-svgrender.readthedocs.io/en/latest/index.html"><img src="https://img.shields.io/badge/docs-readthedocs-purple" alt="docs"></a>
21
- </p>
22
-
23
- <div align="center">
24
- <img src="./assets/logo.png" style="width: 350px; height: 300px;" alt="Pytorch-SVGRender">
25
- <p><strong>Pytorch-SVGRender: </strong>The go-to library for differentiable rendering methods for SVG generation.</p>
26
- </div>
27
- <p align="center">
28
- <a href="#recent-updates">Updates</a> •
29
- <a href="#table-of-contents">Table of Contents</a> •
30
- <a href="#installation">Installation</a> •
31
- <a href="#quickstart">Quickstart</a> •
32
- <a href="#faq">FAQ</a> •
33
- <a href="#todo">TODO</a> •
34
- <a href="#acknowledgement">Acknowledgment</a> •
35
- <a href="#citation">Citation</a> •
36
- <a href="#licence">Licence</a>
37
- </p>
38
-
39
- <h2 align="center">Recent Updates</h2>
40
-
41
- - [12/2023] 🔥 We open-sourced Pytorch-SVGRender V1.0.
42
-
43
- <h2 align="center">Table of Contents</h2>
44
- <p align="right"><a href="#ptsvg"><sup>▴ Back to top</sup></a></p>
45
-
46
- ### 1. Image Vectorization
47
-
48
- - DiffVG: Differentiable Vector Graphics Rasterization for Editing and Learning (`SIGGRAPH 2020`)
49
-
50
- [[Project]](https://people.csail.mit.edu/tzumao/diffvg/) [[Paper]](https://cseweb.ucsd.edu/~tzli/diffvg/diffvg.pdf) [[Code]](https://github.com/BachiLi/diffvg)
51
-
52
- DiffVG is a differentiable rasterizer for 2D vector graphics. **This repository is heavily based on DiffVG.**
53
-
54
- - LIVE: Towards Layer-wise Image Vectorization (`CVPR 2022`)
55
-
56
- [[Project]](https://ma-xu.github.io/LIVE/) [[Paper]](https://ma-xu.github.io/LIVE/index_files/CVPR22_LIVE_main.pdf) [[Code]](https://github.com/Picsart-AI-Research/LIVE-Layerwise-Image-Vectorization)
57
-
58
- - CLIPasso: Semantically-Aware Object Sketching (`SIGGRAPH 2022`)
59
-
60
- [[Project]](https://clipasso.github.io/clipasso/) [[Paper]](https://arxiv.org/abs/2202.05822) [[Code]](https://github.com/yael-vinker/CLIPasso)
61
-
62
- - CLIPascene: Scene Sketching with Different Types and Levels of Abstraction (`ICCV 2023`)
63
-
64
- [[Project]](https://clipascene.github.io/CLIPascene/) [[Paper]](https://arxiv.org/abs/2211.17256) [[Code]](https://github.com/yael-vinker/SceneSketch)
65
-
66
- ### 2. Text-to-SVG Synthesis
67
-
68
- - CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders (`NIPS 2022`)
69
-
70
- [[Paper]](https://arxiv.org/abs/2106.14843) [[Code]](https://github.com/kvfrans/clipdraw)
71
-
72
- - StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Synthesis
73
-
74
- [[Live]](https://slideslive.com/38970834/styleclipdraw-coupling-content-and-style-in-texttodrawing-synthesis?ref=account-folder-92044-folders) [[Paper]](https://arxiv.org/abs/2202.12362) [[Code]](https://github.com/pschaldenbrand/StyleCLIPDraw)
75
-
76
- - CLIPFont: Texture Guided Vector WordArt Generation (`BMVC 2022`)
77
-
78
- [[Paper]](https://bmvc2022.mpi-inf.mpg.de/0543.pdf) [[Code]](https://github.com/songyiren98/CLIPFont)
79
-
80
- - VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models (`CVPR 2023`)
81
-
82
- [[Project]](https://vectorfusion.github.io/) [[Paper]](https://openaccess.thecvf.com/content/CVPR2023/papers/Jain_VectorFusion_Text-to-SVG_by_Abstracting_Pixel-Based_Diffusion_Models_CVPR_2023_paper.pdf)
83
-
84
- - DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models (`NIPS 2023`)
85
-
86
- [[Project]](https://ximinng.github.io/DiffSketcher-project/) [[Live]](https://neurips.cc/virtual/2023/poster/72425) [[Paper]](https://arxiv.org/abs/2306.14685) [[Code]](https://github.com/ximinng/DiffSketcher)
87
-
88
- - Word-As-Image for Semantic Typography (`SIGGRAPH 2023`)
89
-
90
- [[Project]](https://wordasimage.github.io/Word-As-Image-Page/) [[Paper]](https://arxiv.org/abs/2303.01818) [[Code]](https://github.com/Shiriluz/Word-As-Image)
91
-
92
- - SVGDreamer: Text Guided SVG Generation with Diffusion Model (`CVPR 2024`)
93
-
94
- [[Project]](https://ximinng.github.io/SVGDreamer-project/) [[Paper]](https://arxiv.org/abs/2312.16476) [[code]](https://github.com/ximinng/SVGDreamer)
95
-
96
- <h2 align="center">Installation</h2>
97
-
98
- You can follow the steps below to quickly get up and running with PyTorch-SVGRender.
99
- These steps will let you run quick inference locally.
100
-
101
- In the top level directory run,
102
-
103
- ```bash
104
- sh script/install.sh
105
- ```
106
-
107
- Note: Make sure that the script file has execution **permissions** (you can give them using `chmod +x script.sh`), and
108
- then run the script.
109
-
110
- For more information, please refer to
111
- the [Install.md](https://github.com/ximinng/PyTorch-SVGRender/blob/main/Install.md).
112
-
113
- <h2 align="center">Quickstart</h2>
114
- <p align="right"><a href="#ptsvg"><sup>▴ Back to top</sup></a></p>
115
-
116
- **For more information, [read the docs](https://pytorch-svgrender.readthedocs.io/en/latest/index.html).**
117
-
118
- ### 1. Basic Usage
119
-
120
- **DiffVG** vectorizes any raster images:
121
-
122
- ```shell
123
- python svg_render.py x=diffvg target='./data/fallingwater.png'
124
- # change 'num_paths' and 'num_iter' for better results
125
- python svg_render.py x=diffvg target='./data/fallingwater.png' x.num_paths=512 x.num_iter=2000
126
- ```
127
-
128
- **LIVE** vectorizes the raster emojis images (in original PNG format):
129
-
130
- ```shell
131
- python svg_render.py x=live target='./data/simile.png'
132
- # change 'num_paths' and 'schedule_each' for better results
133
- python svg_render.py x=live target='./data/simile.png' x.num_paths=5 x.schedule_each=1
134
- ```
135
-
136
- **CLIPasso** synthesizes vectorized sketches from images:
137
-
138
- **note:** first download the U2Net model `sh script/download_u2net.sh`.
139
-
140
- ```shell
141
- python svg_render.py x=clipasso target='./data/horse.png'
142
- ```
143
-
144
- **CLIPascene** synthesizes vectorized sketches from images:
145
-
146
- **note:** first download the U2Net model `sh script/download_u2net.sh`, and make sure the `./data/background` folder and
147
- the `./data/scene` folder exist with target images.
148
-
149
- ```shell
150
- python svg_render.py x=clipascene target='ballerina.png'
151
- ```
152
-
153
- **CLIPDraw** synthesizes SVGs based on text prompts:
154
-
155
- ```shell
156
- python svg_render.py x=clipdraw "prompt='a photo of a cat'"
157
- ```
158
-
159
- **StyleCLIPDraw** synthesizes SVG based on a text prompt and a reference image:
160
-
161
- ```shell
162
- python svg_render.py x=styleclipdraw "prompt='a photo of a cat'" target='./data/starry.png'
163
- ```
164
-
165
- **CLIPFont** styles vector fonts according to text prompts:
166
-
167
- ```shell
168
- python svg_render.py x=clipfont "prompt='Starry Night by Vincent van gogh'" target='./data/alphabet1.svg'
169
- ```
170
-
171
- ---
172
-
173
- > Because the following methods rely on stable diffusion, add `diffuser.download=True` to the command the **first time** you
174
- run the script.
175
-
176
- **SVGDreamer** generates various styles of SVG based on text prompts. It supports the use of six vector primitives,
177
- including Iconography, Sketch, Pixel Art, Low-Poly, Painting, and Ink and Wash.
178
-
179
- ```shell
180
- # primitive: iconography
181
- ## 1. German shepherd
182
- python svg_render.py x=svgdreamer "prompt='A colorful German shepherd in vector art. tending on artstation.'" save_step=30 x.guidance.n_particle=6 x.guidance.vsd_n_particle=4 x.guidance.phi_n_particle=2 result_path='./svgdreamer/GermanShepherd'
183
- ## 2. sydney opera house
184
- python svg_render.py x=svgdreamer "prompt='Sydney opera house. oil painting. by Van Gogh'" save_step=30 x.guidance.n_particle=6 x.guidance.vsd_n_particle=4 x.guidance.phi_n_particle=2 x.num_paths=512 result_path='./svgdreamer/Sydney'
185
- # primitive: low-ploy
186
- python svg_render.py x=svgdreamer "prompt='A picture of a bald eagle. low-ploy. polygon'" x.style='low-poly' save_step=30 x.guidance.n_particle=6 x.guidance.vsd_n_particle=4 x.guidance.phi_n_particle=2 x.guidance.num_iter=1000 result_path='./svgdreamer/eagle'
187
- # primitive: pixel-art
188
- python svg_render.py x=svgdreamer "prompt='Darth vader with lightsaber. ultrarealistic. pixelart. trending on artstation.'" x.style='pixelart' save_step=30 x.guidance.n_particle=6 x.guidance.vsd_n_particle=4 x.guidance.phi_n_particle=2 x.guidance.num_iter=1000 result_path='./svgdreamer/DarthVader'
189
- # primitive: painting
190
- python svg_render.py x=svgdreamer "prompt='self portrait of Van Gogh. oil painting. cmyk portrait. multi colored. defiant and beautiful. cmyk. expressive eyes.'" x.style='painting' save_step=50 x.guidance.n_particle=6 x.guidance.vsd_n_particle=4 x.guidance.phi_n_particle=2 x.guidance.t_schedule='randint' x.num_paths=1500 result_path='./svgdreamer/VanGogh_portrait'
191
- # primitive: sketch
192
- python svg_render.py x=svgdreamer "prompt='A free-hand drawing of A speeding Lamborghini. black and white drawing.'" x.style='sketch' save_step=30 x.guidance.n_particle=6 x.guidance.vsd_n_particle=4 x.guidance.phi_n_particle=2 x.guidance.t_schedule='randint' x.num_paths=128 result_path='./svgdreamer/Lamborghini'
193
- # primitive: ink and wash
194
- python svg_render.py x=svgdreamer "prompt='Big Wild Goose Pagoda. ink style. Minimalist abstract art grayscale watercolor.'" x.style='ink' save_step=30 x.guidance.n_particle=6 x.guidance.vsd_n_particle=4 x.guidance.phi_n_particle=2 x.guidance.t_schedule='randint' x.num_paths=128 x.width=6 result_path='./svgdreamer/BigWildGoosePagoda'
195
- ```
196
-
197
- **VectorFusion** synthesizes SVGs in various styles based on text prompts:
198
-
199
- ```shell
200
- # Iconography style
201
- python svg_render.py x=vectorfusion x.style='iconography' "prompt='a panda rowing a boat in a pond. minimal flat 2d vector icon. lineal color. trending on artstation.'"
202
- # PixelArt style
203
- python svg_render.py x=vectorfusion x.style='pixelart' "prompt='a panda rowing a boat in a pond. pixel art. trending on artstation.'"
204
- # Sketch style
205
- python svg_render.py x=vectorfusion x.style='sketch' "prompt='a panda rowing a boat in a pond. minimal 2d line drawing. trending on artstation.'"
206
- ```
207
-
208
- Following SVGDreamer, we've added three additional styles (`Paining`, `Ink and Wash` and `low-ploy`) to VectorFusion.
209
-
210
- **DiffSketcher** synthesizes vector sketches based on text prompts:
211
-
212
- ```shell
213
- # DiffSketcher
214
- python svg_render.py x=diffsketcher "prompt='a photo of Sydney opera house'" x.token_ind=5 seed=8019
215
- # DiffSketcher, variable stroke width
216
- python svg_render.py x=diffsketcher "prompt='a photo of Sydney opera house'" x.token_ind=5 x.optim_width=True seed=8019
217
- # DiffSketcher RGBA version
218
- python svg_render.py x=diffsketcher "prompt='a photo of Sydney opera house'" x.token_ind=5 x.optim_width=True x.optim_rgba=True x.optim_opacity=False seed=8019
219
- # DiffSketcher + style transfer
220
- python svg_render.py x=stylediffsketcher "prompt='The French Revolution. highly detailed. 8k. ornate. intricate. cinematic. dehazed. atmospheric. oil painting. by Van Gogh'" x.token_ind=4 x.num_paths=2000 target='./data/starry.png' seed=876809
221
- ```
222
-
223
- **Word-As-Image** follow a text prompt to style a letter in a word:
224
-
225
- ```shell
226
- # Inject the meaning of the word bunny into the 'Y' in the word 'BUNNY'
227
- python svg_render.py x=wordasimage x.word='BUNNY' prompt='BUNNY' x.optim_letter='Y'
228
- ```
229
-
230
- ### 2. SDS Loss based Approach
231
-
232
- This is achieved by utilizing a pretrained text-to-image diffusion model as a strong image prior to supervise the
233
- training of the PyDiffVG, enabling rendering SVG alignment with the text. This remarkable capability is fundamentally
234
- grounded in the use of Score Distillation Sampling (SDS). SDS acts as the core mechanism that lifts raster images from
235
- diffusion models to the SVG domain, enabling the training of SVG parameters without images.
236
- This includes the methods VectorFusion, DiffSketcher and SVGDreamer.
237
-
238
- We only compare the performance of SDS, which means that no other loss is used:
239
-
240
- ```shell
241
- # SDS loss
242
- python svg_render.py x=vectorfusion "prompt='a panda rowing a boat in a pond. minimal flat 2d vector icon. lineal color. trending on artstation.'"
243
- # Input Augmentation SDS loss (LSDS loss)
244
- python svg_render.py x=vectorfusion x.style='sketch' "prompt='an elephant. minimal 2d line drawing. trending on artstation.'"
245
- # Input Augmentation SDS loss (ASDS loss)
246
- python svg_render.py x=diffsketcher "prompt='an elephant. minimal 2d line drawing. trending on artstation.'" x.token_ind=2 x.sds.grad_scale=1 x.sds.num_aug=4 x.clip.vis_loss=0 x.perceptual.coeff=0 x.opacity_delta=0.3
247
- # Vectorized Particle-based Score Distillation (VPSD loss)
248
- python svg_render.py x=svgdreamer "prompt='a panda rowing a boat in a pond. minimal flat 2d vector icon. lineal color. trending on artstation.'" save_step=60 x.guidance.n_particle=6 x.guidance.vsd_n_particle=4 x.guidance.phi_n_particle=2
249
- ```
250
-
251
- <h2 align="center">FAQ</h2>
252
- <p align="right"><a href="#ptsvg"><sup>▴ Back to top</sup></a></p>
253
-
254
- - Q: Where can I get more scripts and visualizations?
255
- - A: check the [pytorch-svgrender.readthedocs.io](https://pytorch-svgrender.readthedocs.io/en/latest/index.html).
256
-
257
- - Q: An error says HuggingFace cannot find the model in the disk cache.
258
- - A: Add *`diffuser.download=True`* to the command for downloading model checkpoints the **first time** you run the script.
259
-
260
- <h2 align="center">TODO</h2>
261
- <p align="right"><a href="#ptsvg"><sup>▴ Back to top</sup></a></p>
262
-
263
- - [x] integrated SVGDreamer.
264
-
265
- <h2 align="center">Acknowledgement</h2>
266
- <p align="right"><a href="#ptsvg"><sup>▴ Back to top</sup></a></p>
267
-
268
- The project is built based on the following repository:
269
-
270
- [BachiLi/diffvg](https://github.com/BachiLi/diffvg),
271
- [huggingface/diffusers](https://github.com/huggingface/diffusers),
272
- [threestudio-project/threestudio](https://github.com/threestudio-project/threestudio),
273
- [yael-vinker/CLIPasso](https://github.com/yael-vinker/CLIPasso),
274
- [ximinng/DiffSketcher](https://github.com/ximinng/DiffSketcher),
275
- [THUDM/ImageReward](https://github.com/THUDM/ImageReward),
276
- [advimman/lama](https://github.com/advimman/lama)
277
-
278
- We gratefully thank the authors for their wonderful works.
279
-
280
- <h2 align="center">Citation</h2>
281
- <p align="right"><a href="#ptsvg"><sup>▴ Back to top</sup></a></p>
282
-
283
- If you use this code for your research, please cite the following work:
284
-
285
- ```
286
- @article{xing2023svgdreamer,
287
- title={SVGDreamer: Text Guided SVG Generation with Diffusion Model},
288
- author={Xing, Ximing and Zhou, Haitao and Wang, Chuang and Zhang, Jing and Xu, Dong and Yu, Qian},
289
- journal={arXiv preprint arXiv:2312.16476},
290
- year={2023}
291
- }
292
- @inproceedings{xing2023diffsketcher,
293
- title={DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models},
294
- author={XiMing Xing and Chuang Wang and Haitao Zhou and Jing Zhang and Qian Yu and Dong Xu},
295
- booktitle={Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS)},
296
- year={2023},
297
- url={https://openreview.net/forum?id=CY1xatvEQj}
298
- }
299
- ```
300
-
301
- <h2 align="center">Licence</h2>
302
- <p align="right"><a href="#ptsvg"><sup>▴ Back to top</sup></a></p>
303
-
304
- This work is licensed under a **Mozilla Public License Version 2.0**.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
README.md CHANGED
@@ -5,6 +5,7 @@ colorFrom: indigo
5
  colorTo: blue
6
  sdk: gradio
7
  sdk_version: 4.21.0
 
8
  app_file: app.py
9
  pinned: false
10
  license: apache-2.0
 
5
  colorTo: blue
6
  sdk: gradio
7
  sdk_version: 4.21.0
8
+ python_version: 3.10.12
9
  app_file: app.py
10
  pinned: false
11
  license: apache-2.0