peteli commited on
Commit
434004a
1 Parent(s): a15d99e

Upload main.ipynb

Browse files
Files changed (1) hide show
  1. main.ipynb +380 -0
main.ipynb ADDED
@@ -0,0 +1,380 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "metadata": {},
6
+ "source": [
7
+ "<div align=\"center\">\n",
8
+ " <img src=\"https://user-images.githubusercontent.com/11793384/215372703-4385f66a-abe4-44c7-9626-96b7b65270c8.png\" width=\"40%\" height=\"40%\" />\n",
9
+ "</div>\n",
10
+ "\n",
11
+ "<center><font face=\"微软雅黑\" size=5> <b>【飞桨黑客松-AIGC - DreamBooth LoRA】This is my HomeTown</b></font></center> \n",
12
+ "\n",
13
+ "<br>\n",
14
+ "\n",
15
+ "<center>🔥🔥🔥 前往官网了解题目详情、报名、冲大奖、玩创意!👉👉👉 <a href=\"https://github.com/PaddlePaddle/Paddle/issues/50631#task105\">https://github.com/PaddlePaddle/Paddle/issues/50631#task105</a> </center>"
16
+ ]
17
+ },
18
+ {
19
+ "cell_type": "markdown",
20
+ "metadata": {},
21
+ "source": [
22
+ "# 1. 背景介绍\n",
23
+ "\n",
24
+ "<h2> <center>紫金花海</center>\n",
25
+ " <br>\n",
26
+ " <center>The Purple Flower Sea</center> \n",
27
+ " <br>\n",
28
+ " <left> <font size=\"2\">紫金苑景区,金昌市民习惯称为“紫金花海”,位于金昌市区的北部,新华路街道天津路与泰安路交叉口北150米,总面积有700多亩。是按照景城一体化理念打造的香草花卉生态休闲旅游景区。</font></left>\n",
29
+ "</h2>\n",
30
+ "\n",
31
+ "![](https://ai-studio-static-online.cdn.bcebos.com/ebc26abd8a7949139e19167ccc9aae6d39709727b1b9448f82fafde6adfcf4f5)\n",
32
+ "\n",
33
+ "<center> \n",
34
+ " <font size=\"2\">\n",
35
+ " \n",
36
+ " 微风袭来\n",
37
+ " 荡漾起的紫色“海浪”映入眼帘\n",
38
+ " 沁人心脾的清香\n",
39
+ " 淡淡地飘过每个角落\n",
40
+ " 小憩在花海旁\n",
41
+ " 倾听着花的物语\n",
42
+ " 骑着单车\n",
43
+ " 穿梭在紫色海洋\n",
44
+ " 掠过小湖\n",
45
+ " 静静享受紫色连波的浪漫\n",
46
+ " 走过小桥\n",
47
+ " 感受着紫色田园的美好\n",
48
+ "\n",
49
+ "![#pic_center](https://ai-studio-static-online.cdn.bcebos.com/e20c7bbeb85349058fab2ad7c4a8718df4461d0a9a8b4bc8bb0ed6b3bd561db1)\n",
50
+ " \n",
51
+ " <center>\n",
52
+ " <font size=\"2\">\n",
53
+ " \n",
54
+ " 被花海围绕的湖边\n",
55
+ " 倒映出一张张欢乐的笑脸\n",
56
+ " 穿过小小的别墅庄园\n",
57
+ " 来到充满爱意的婚礼广场\n",
58
+ " 浪漫而惬意的气氛\n",
59
+ " 像是走进了童话世界\n",
60
+ " 蜜蜂在“海浪”中辛勤游荡\n",
61
+ " 蝴蝶在“海浪”中嬉戏打闹\n",
62
+ " 小鸟在“海浪”中合奏吟唱\n",
63
+ " 人儿在“海浪”中忘记烦恼\n",
64
+ " 一个个穿梭在“海浪”的身影\n",
65
+ " 仿佛都在诉说着这别致的一切\n",
66
+ "\n",
67
+ " ![](https://ai-studio-static-online.cdn.bcebos.com/5b2415cf91a24d1583c0108cd528c89b7bdb8ca4218c4c0f9999f6cc2542f0ab)\n",
68
+ " \n",
69
+ " <center> \n",
70
+ " <font size=\"2\">\n",
71
+ " \n",
72
+ " 迎着初秋的微风\n",
73
+ " 远离城市的喧嚣\n",
74
+ " 游荡在这梦幻浪漫的紫色海洋\n",
75
+ " 把一切的不开心都遗忘\n",
76
+ " 欢迎大家来紫色小城游玩噢~\n",
77
+ "![](https://ai-studio-static-online.cdn.bcebos.com/ef94e24c6cd9482ab169e03b96c5883618f9d5fde7f746dc884cf6bc86504244)\n",
78
+ " "
79
+ ]
80
+ },
81
+ {
82
+ "cell_type": "markdown",
83
+ "metadata": {},
84
+ "source": [
85
+ "# 2. 准备工作 \n",
86
+ "## 2.1 环境安装 \n",
87
+ " \n",
88
+ "```sh\n",
89
+ "pip install \"paddlenlp>=2.5.2\" \"ppdiffusers>=0.11.1\" safetensors --user\n",
90
+ "```\n",
91
+ "</font>\n"
92
+ ]
93
+ },
94
+ {
95
+ "cell_type": "code",
96
+ "execution_count": null,
97
+ "metadata": {
98
+ "scrolled": true,
99
+ "tags": []
100
+ },
101
+ "outputs": [],
102
+ "source": [
103
+ "# 请运行这里安装所需要的依赖环境!!\n",
104
+ "!pip install \"paddlenlp==2.5.2\" safetensors \"ppdiffusers>=0.11.1\" --user"
105
+ ]
106
+ },
107
+ {
108
+ "cell_type": "markdown",
109
+ "metadata": {},
110
+ "source": [
111
+ "## 2.2 [Hugging Face Space](https://huggingface.co/) 注册和登录 \n",
112
+ "\n",
113
+ "题目要求将模型上传到 Hugging Face,需要先注册、登录。\n",
114
+ "\n",
115
+ "* <font face=\"微软雅黑\" size=3>**注册和登录:https://huggingface.co/join** </font>\n",
116
+ " <div align=\"center\">\n",
117
+ " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/fa96268f8d0047fa8ad25b678fb209631ebe08c463884378bdd268fdc025a808\" width=\"40%\" height=\"40%\" />\n",
118
+ " </div> \n",
119
+ "* <font face=\"微软雅黑\" size=3>**获取登录 Token**</font>\n",
120
+ " <div align=\"center\">\n",
121
+ " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/5cd101321369427ca94d685ad082779084243fe6839f49bc8b173478a8eaf03c\" width=\"40%\" height=\"40%\" />\n",
122
+ " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/7df3372db6f94ba2bdb636ee4a23c74bf2e05ef5a6e849c4b2e5fb3d4e3679e1\" width=\"40%\" height=\"40%\" />\n",
123
+ " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/ebc309852fba4ae6b799d519d196b5eee76bf21682db4cb8852e01d6ae6f0602\" width=\"40%\" height=\"40%\" /> \n",
124
+ " </div> \n",
125
+ "\n",
126
+ "* <font face=\"微软雅黑\" size=3>**Aistudio 登录 Huggingface Hub**</font>\n",
127
+ " <div align=\"center\">\n",
128
+ " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/d9e2c88b9a8d43f08f7379f4c8bef35c81992475e98b4c958f337f02b8174b56\" width=\"40%\" height=\"40%\" />\n",
129
+ " </div> \n",
130
+ " \n",
131
+ "Tips:为了方便我们之后上传权重,我们需要登录 Huggingface Hub,想要了解更多的信息我们可以查阅 [官方文档](https://huggingface.co/docs/huggingface_hub/quick-start)。\n"
132
+ ]
133
+ },
134
+ {
135
+ "cell_type": "code",
136
+ "execution_count": 2,
137
+ "metadata": {
138
+ "execution": {
139
+ "iopub.execute_input": "2023-03-11T05:39:13.575730Z",
140
+ "iopub.status.busy": "2023-03-11T05:39:13.575259Z",
141
+ "iopub.status.idle": "2023-03-11T05:39:14.194481Z",
142
+ "shell.execute_reply": "2023-03-11T05:39:14.193575Z",
143
+ "shell.execute_reply.started": "2023-03-11T05:39:13.575699Z"
144
+ },
145
+ "scrolled": true,
146
+ "tags": []
147
+ },
148
+ "outputs": [
149
+ {
150
+ "data": {
151
+ "application/vnd.jupyter.widget-view+json": {
152
+ "model_id": "b5d6b75aa009457784dcba9d1597d9c3",
153
+ "version_major": 2,
154
+ "version_minor": 0
155
+ },
156
+ "text/plain": [
157
+ "VBox(children=(HTML(value='<center> <img\\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…"
158
+ ]
159
+ },
160
+ "metadata": {},
161
+ "output_type": "display_data"
162
+ }
163
+ ],
164
+ "source": [
165
+ "# 登录 Huggingface Hub\n",
166
+ "!git config --global credential.helper store\n",
167
+ "from huggingface_hub import login\n",
168
+ "login()"
169
+ ]
170
+ },
171
+ {
172
+ "cell_type": "markdown",
173
+ "metadata": {},
174
+ "source": [
175
+ "- tips:如何检测是否登录成功?\n",
176
+ "\n",
177
+ "打开日志控制控制台,查看日志。\n",
178
+ "\n",
179
+ "![](https://ai-studio-static-online.cdn.bcebos.com/6bdaeac42a7a462fb0d08446f0f0144206d903f36e1047de8bf1b4bda5e8f94e)\n",
180
+ "\n",
181
+ "登录成功时,日志如下:\n",
182
+ "\n",
183
+ "![](https://ai-studio-static-online.cdn.bcebos.com/c193dab68ec24af8bcca15fd28535af9a52227fbb9474dc184a4eb8c7c5cfa35)\n"
184
+ ]
185
+ },
186
+ {
187
+ "cell_type": "markdown",
188
+ "metadata": {},
189
+ "source": [
190
+ "# 3. 如何训练模型,并上传到HF\n",
191
+ "\n",
192
+ "## 3.1 上传图片\n",
193
+ "* <font face=\"微软雅黑\" size=3>首先,我们需要将所需训练的图片上传到aistudio上的文件夹, 我们可以通过👉**拖拽上传** 的方式,将我们所需的图片上传至指定的文件夹。</font>\n",
194
+ "* <font face=\"微软雅黑\" size=3>在这里,我们已经在👉**hometown**文件夹准备好了如下所示的5张图片。</font>\n",
195
+ "![](https://ai-studio-static-online.cdn.bcebos.com/cb72e87e227b4da18b6a0eba01da507ab055bcaa30904c1382e0878c781c043b)\n",
196
+ "![](https://ai-studio-static-online.cdn.bcebos.com/6a0f4b223b2747c7af9e4d98129f65b2328c90036c3c479f92febab318db6028)\n",
197
+ "![](https://ai-studio-static-online.cdn.bcebos.com/f8a2589d91834d24b4e05cbc0c01fd41f67ea102f6f4462eb34080eabf8a455f)\n",
198
+ "![](https://ai-studio-static-online.cdn.bcebos.com/60ee699e7ee344dab55c5bed6f2901c9c0bc176db47941cebf82860a4eeab01b)\n",
199
+ "![](https://ai-studio-static-online.cdn.bcebos.com/b3e37a3182434966ae83d8e82fdd55df96e5b1c945bf41edad808a4d36cb327b)\n"
200
+ ]
201
+ },
202
+ {
203
+ "cell_type": "markdown",
204
+ "metadata": {},
205
+ "source": [
206
+ "## 3.2 训练参数调整\n",
207
+ "\n",
208
+ "<font face=\"微软雅黑\" size=3> 在训练过程中,我们可以尝试修改训练的默认参数,下面将从三个方面介绍部分参数。\n",
209
+ "\n",
210
+ "👉主要修改的参数:\n",
211
+ "> * **pretrained_model_name_or_path** :想要训练的模型名称或者本地路径的模型,例如:`\"runwayml/stable-diffusion-v1-5\"`,更多模型可参考 [PaddleNLP 文档](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/ppdiffusers#ppdiffusers%E6%A8%A1%E5%9E%8B%E6%94%AF%E6%8C%81%E7%9A%84%E6%9D%83%E9%87%8D)。\n",
212
+ "> * **instance_data_dir**:训练图片所在的文件夹目录,我们可以将图片上传至aistudio项目。\n",
213
+ "> * **instance_prompt**:训练所使用的 `Prompt` 文本。\n",
214
+ "> * **resolution**:训练时图像的分辨率,建议为 `512`。\n",
215
+ "> * **output_dir**:训练过程中,模型保存的目录。\n",
216
+ "> * **checkpointing_steps**:每隔多少步保存模型,默认为`100`步。\n",
217
+ "> * **learning_rate**:训练使用的学习率,当我使用 `LoRA` 训练模型的时候,我们需要使用更大的学习率,因此我们这里使用 `1e-4` 而不是 `2e-6`。\n",
218
+ "> * **max_train_steps**:最大训练的步数,默认为`500`步。\n",
219
+ " \n",
220
+ "👉可选修改的参数:\n",
221
+ "> * **train_batch_size**:训练时候使用的 `batch_size`���当我们的GPU显存比较大的时候可以加大这个值,默认值为`4`。\n",
222
+ "> * **gradient_accumulation_steps**:梯度累积的步数,当我们GPU显存比较小的时候还想模拟大的训练批次,我们可以适当增加梯度累积的步数,默认值为`1`。\n",
223
+ "> * **seed**:随机种子,设置后可以复现训练结果。\n",
224
+ "> * **lora_rank**:`LoRA` 层的 `rank` 值,默认值为4,最终我们会得到 **3.5MB** 的模型,我们可以适当修改这个值,如:`32、64、128、256` 等。\n",
225
+ "> * **lr_scheduler**:学习率衰减策略,可以是`\"linear\", \"constant\", \"cosine\"`等。\n",
226
+ "> * **lr_warmup_steps**:学习率衰减前,`warmup` 到最大学习率所需要的步数。\n",
227
+ " \n",
228
+ "👉训练过程中评估使用的参数:\n",
229
+ "> * **num_validation_images**:训练的过程中,我们希望返回多少张图片,默认值为`4`张图片。\n",
230
+ "> * **validation_prompt**:训练的过程中我们会评估训练的怎么样,因此我们需要设置评估使用的 `prompt` 文本。\n",
231
+ "> * **validation_steps**:每隔多少个 `steps` 评估模型,我们可以查看训练的进度条,知道当前到了第几个 `steps`。\n",
232
+ " \n",
233
+ "🔥**Tips:**\n",
234
+ " 训练过程中会每隔 `validation_steps` 将生成的图片保存到 `{你指定的输出路径}/validation_images/{步数}.jpg`\n",
235
+ "\n",
236
+ "<div align=\"center\">\n",
237
+ " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/86ab5ed4a31f4cf1a4b9ba1c52ccc94e2731c711dff6420eb3745f718431cb29\" width=\"40%\" height=\"40%\" />\n",
238
+ "</div> \n",
239
+ " \n",
240
+ "👉权重上传的参数:\n",
241
+ "> * **push_to_hub**: 是否将模型上传到 `huggingface hub`,默认值为 `False`。\n",
242
+ "> * **hub_token**: 上传到 `huggingface hub` 所需要使用的 `token`,如果我们已经登录了,那么我们就无需填写。\n",
243
+ "> * **hub_model_id**: 上传到 `huggingface hub` 的模型库名称, 如果为 `None` 的话表示我们将使用 `output_dir` 的名称作为模型库名称。\n",
244
+ "\n",
245
+ "在下面的例子中,由于我们前面已经登录了,因此我们可以开启 **push_to_hub** 按钮,将最终训练好的模型同步上传到 `huggingface.co`\n",
246
+ " \n",
247
+ "当我们开启`push_to_hub`后,等待程序运行完毕后会自动将权重上传到这个路径 https://huggingface.co/{你的用户名}/{你指定的输出路径} ,例如: https://huggingface.co/peteli/hometown\n",
248
+ "</font>\n",
249
+ "\n"
250
+ ]
251
+ },
252
+ {
253
+ "cell_type": "code",
254
+ "execution_count": null,
255
+ "metadata": {
256
+ "scrolled": true,
257
+ "tags": []
258
+ },
259
+ "outputs": [],
260
+ "source": [
261
+ "!python train_dreambooth_lora.py \\\n",
262
+ " --pretrained_model_name_or_path=\"runwayml/stable-diffusion-v1-5\" \\\n",
263
+ " --instance_data_dir=\"./hometown\" \\\n",
264
+ " --output_dir=\"lora_outputs\" \\\n",
265
+ " --instance_prompt=\"a sea of purple and gold flowers\" \\\n",
266
+ " --resolution=512 \\\n",
267
+ " --train_batch_size=1 \\\n",
268
+ " --gradient_accumulation_steps=1 \\\n",
269
+ " --checkpointing_steps=100 \\\n",
270
+ " --learning_rate=1e-4 \\\n",
271
+ " --lr_scheduler=\"constant\" \\\n",
272
+ " --lr_warmup_steps=0 \\\n",
273
+ " --max_train_steps=1000 \\\n",
274
+ " --seed=0 \\\n",
275
+ " --lora_rank=4 \\\n",
276
+ " --push_to_hub=False \\\n",
277
+ " --validation_prompt=\"a sea of lavender and gold flowers in the world of fairy tales is my hometown\" \\\n",
278
+ " --validation_steps=100 \\\n",
279
+ " --num_validation_images=4"
280
+ ]
281
+ },
282
+ {
283
+ "cell_type": "markdown",
284
+ "metadata": {},
285
+ "source": [
286
+ "## 3.3 可视化训练过程\n",
287
+ "\n",
288
+ "VisualDL使用参考:[官方教程](https://aistudio.baidu.com/aistudio/projectdetail/1990920)\n",
289
+ "\n",
290
+ "我们可以参照如图所示的步骤,开启visualdl,然后查看训练过程中的指标变化。\n",
291
+ "\n",
292
+ "<div align=\"center\">\n",
293
+ " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/284485f03165482493e8399c3277a422ec5520382c564e2b9e1d5c0e113588e1\" width=\"60%\" height=\"60%\" />\n",
294
+ "</div>"
295
+ ]
296
+ },
297
+ {
298
+ "cell_type": "markdown",
299
+ "metadata": {},
300
+ "source": [
301
+ "## 3.4 挑选满意的权重上传至Huggingface\n",
302
+ "\n",
303
+ "参数解释:\n",
304
+ "> * **upload_dir**:我们需要上传的文件夹目录。\n",
305
+ "> * **repo_name**:我们需要上传的repo名称,最终我们会上传到 https://huggingface.co/{你的用户名}/{你指定的repo名称}, 例如: https://huggingface.co/peteli/hometown.\n",
306
+ "> * **pretrained_model_name_or_path**:训练该模型所使用的基础模型。\n",
307
+ "> * **prompt**:搭配该权重需要使用的Prompt文本。\n",
308
+ "<div align=\"center\">\n",
309
+ " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/9036aa64b12d4d1db0b64f685aa98c600aad8f984cbf4ef592bef4c4e6db11e4\" width=\"60%\" height=\"60%\" />\n",
310
+ "</div>"
311
+ ]
312
+ },
313
+ {
314
+ "cell_type": "code",
315
+ "execution_count": null,
316
+ "metadata": {
317
+ "scrolled": true,
318
+ "tags": []
319
+ },
320
+ "outputs": [],
321
+ "source": [
322
+ "from utils import upload_lora_folder\n",
323
+ "upload_dir = \"lora_outputs\" # 我们需要上传的文件夹目录\n",
324
+ "repo_name = \"hometown\" # 我们需要上传的repo名称\n",
325
+ "pretrained_model_name_or_path = \"runwayml/stable-diffusion-v1-5\" # 训练该模型所使用的基础模型\n",
326
+ "prompt = \"a sea of lavender and gold flowers in the world of fairy tales is my hometown\" # 搭配该权重需要使用的Prompt文本\n",
327
+ "\n",
328
+ "upload_lora_folder(\n",
329
+ " upload_dir=upload_dir,\n",
330
+ " repo_name=repo_name,\n",
331
+ " pretrained_model_name_or_path=pretrained_model_name_or_path,\n",
332
+ " prompt=prompt, \n",
333
+ ")"
334
+ ]
335
+ },
336
+ {
337
+ "cell_type": "markdown",
338
+ "metadata": {},
339
+ "source": [
340
+ "# 4.效果展示\n",
341
+ "\n",
342
+ "#### 图1:\n",
343
+ "\n",
344
+ "![](https://ai-studio-static-online.cdn.bcebos.com/b521bc52e4bb4e63b98a2f32a9fd65cbde84088594084e548e25931339a34e0b)\n",
345
+ "\n",
346
+ "#### 图2:\n",
347
+ "\n",
348
+ "![](https://ai-studio-static-online.cdn.bcebos.com/fb2518e3c06b4da5a41973cf6dc81684020e9efd3bfc4b9289163beb774dc530)\n",
349
+ "\n",
350
+ "#### 图3:\n",
351
+ "\n",
352
+ "![](https://ai-studio-static-online.cdn.bcebos.com/b845766ce8b944aead7430146be1ab1fd721456bdce14fe698b7c6978555fa36)\n"
353
+ ]
354
+ }
355
+ ],
356
+ "metadata": {
357
+ "kernelspec": {
358
+ "display_name": "Python 3",
359
+ "language": "python",
360
+ "name": "py35-paddle1.2.0"
361
+ },
362
+ "language_info": {
363
+ "codemirror_mode": {
364
+ "name": "ipython",
365
+ "version": 3
366
+ },
367
+ "file_extension": ".py",
368
+ "mimetype": "text/x-python",
369
+ "name": "python",
370
+ "nbconvert_exporter": "python",
371
+ "pygments_lexer": "ipython3",
372
+ "version": "3.7.4"
373
+ },
374
+ "toc-autonumbering": false,
375
+ "toc-showcode": false,
376
+ "toc-showtags": false
377
+ },
378
+ "nbformat": 4,
379
+ "nbformat_minor": 4
380
+ }