{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " \n", "
\n", "\n", "
【飞桨黑客松-AIGC - DreamBooth LoRA】This is my HomeTown
\n", "\n", "
\n", "\n", "
🔥🔥🔥 前往官网了解题目详情、报名、冲大奖、玩创意！👉👉👉 https://github.com/PaddlePaddle/Paddle/issues/50631#task105
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 1. 背景介绍\n", "\n", "

紫金花海
\n", "
\n", "
The Purple Flower Sea
\n", "
\n", " 紫金苑景区，金昌市民习惯称为“紫金花海”，位于金昌市区的北部，新华路街道天津路与泰安路交叉口北150米，总面积有700多亩。是按照景城一体化理念打造的香草花卉生态休闲旅游景区。\n", "

\n", "\n", "![](https://ai-studio-static-online.cdn.bcebos.com/ebc26abd8a7949139e19167ccc9aae6d39709727b1b9448f82fafde6adfcf4f5)\n", "\n", "
\n", " \n", " \n", " 微风袭来\n", " 荡漾起的紫色“海浪”映入眼帘\n", " 沁人心脾的清香\n", " 淡淡地飘过每个角落\n", " 小憩在花海旁\n", " 倾听着花的物语\n", " 骑着单车\n", " 穿梭在紫色海洋\n", " 掠过小湖\n", " 静静享受紫色连波的浪漫\n", " 走过小桥\n", " 感受着紫色田园的美好\n", "\n", "![#pic_center](https://ai-studio-static-online.cdn.bcebos.com/e20c7bbeb85349058fab2ad7c4a8718df4461d0a9a8b4bc8bb0ed6b3bd561db1)\n", " \n", "
\n", " \n", " \n", " 被花海围绕的湖边\n", " 倒映出一张张欢乐的笑脸\n", " 穿过小小的别墅庄园\n", " 来到充满爱意的婚礼广场\n", " 浪漫而惬意的气氛\n", " 像是走进了童话世界\n", " 蜜蜂在“海浪”中辛勤游荡\n", " 蝴蝶在“海浪”中嬉戏打闹\n", " 小鸟在“海浪”中合奏吟唱\n", " 人儿在“海浪”中忘记烦恼\n", " 一个个穿梭在“海浪”的身影\n", " 仿佛都在诉说着这别致的一切\n", "\n", " ![](https://ai-studio-static-online.cdn.bcebos.com/5b2415cf91a24d1583c0108cd528c89b7bdb8ca4218c4c0f9999f6cc2542f0ab)\n", " \n", "
\n", " \n", " \n", " 迎着初秋的微风\n", " 远离城市的喧嚣\n", " 游荡在这梦幻浪漫的紫色海洋\n", " 把一切的不开心都遗忘\n", " 欢迎大家来紫色小城游玩噢~\n", "![](https://ai-studio-static-online.cdn.bcebos.com/ef94e24c6cd9482ab169e03b96c5883618f9d5fde7f746dc884cf6bc86504244)\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2. 准备工作 \n", "## 2.1 环境安装 \n", " \n", "```sh\n", "pip install \"paddlenlp>=2.5.2\" \"ppdiffusers>=0.11.1\" safetensors --user\n", "```\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# 请运行这里安装所需要的依赖环境！！\n", "!pip install \"paddlenlp==2.5.2\" safetensors \"ppdiffusers>=0.11.1\" --user" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2.2 [Hugging Face Space](https://huggingface.co/) 注册和登录 \n", "\n", "题目要求将模型上传到 Hugging Face，需要先注册、登录。\n", "\n", "* **注册和登录：https://huggingface.co/join** \n", "
\n", " \n", "
\n", "* **获取登录 Token**\n", "
\n", " \n", " \n", " \n", "
\n", "\n", "* **Aistudio 登录 Huggingface Hub**\n", "
\n", " \n", "
\n", " \n", "Tips：为了方便我们之后上传权重，我们需要登录 Huggingface Hub，想要了解更多的信息我们可以查阅 [官方文档](https://huggingface.co/docs/huggingface_hub/quick-start)。\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2023-03-11T05:39:13.575730Z", "iopub.status.busy": "2023-03-11T05:39:13.575259Z", "iopub.status.idle": "2023-03-11T05:39:14.194481Z", "shell.execute_reply": "2023-03-11T05:39:14.193575Z", "shell.execute_reply.started": "2023-03-11T05:39:13.575699Z" }, "scrolled": true, "tags": [] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "b5d6b75aa009457784dcba9d1597d9c3", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HTML(value='
首先，我们需要将所需训练的图片上传到aistudio上的文件夹, 我们可以通过👉**拖拽上传** 的方式，将我们所需的图片上传至指定的文件夹。\n", "* 在这里，我们已经在👉**hometown**文件夹准备好了如下所示的5张图片。\n", "![](https://ai-studio-static-online.cdn.bcebos.com/cb72e87e227b4da18b6a0eba01da507ab055bcaa30904c1382e0878c781c043b)\n", "![](https://ai-studio-static-online.cdn.bcebos.com/6a0f4b223b2747c7af9e4d98129f65b2328c90036c3c479f92febab318db6028)\n", "![](https://ai-studio-static-online.cdn.bcebos.com/f8a2589d91834d24b4e05cbc0c01fd41f67ea102f6f4462eb34080eabf8a455f)\n", "![](https://ai-studio-static-online.cdn.bcebos.com/60ee699e7ee344dab55c5bed6f2901c9c0bc176db47941cebf82860a4eeab01b)\n", "![](https://ai-studio-static-online.cdn.bcebos.com/b3e37a3182434966ae83d8e82fdd55df96e5b1c945bf41edad808a4d36cb327b)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3.2 训练参数调整\n", "\n", " 在训练过程中，我们可以尝试修改训练的默认参数，下面将从三个方面介绍部分参数。\n", "\n", "👉主要修改的参数：\n", "> * **pretrained_model_name_or_path** ：想要训练的模型名称或者本地路径的模型，例如：`\"runwayml/stable-diffusion-v1-5\"`，更多模型可参考 [PaddleNLP 文档](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/ppdiffusers#ppdiffusers%E6%A8%A1%E5%9E%8B%E6%94%AF%E6%8C%81%E7%9A%84%E6%9D%83%E9%87%8D)。\n", "> * **instance_data_dir**：训练图片所在的文件夹目录，我们可以将图片上传至aistudio项目。\n", "> * **instance_prompt**：训练所使用的 `Prompt` 文本。\n", "> * **resolution**：训练时图像的分辨率，建议为 `512`。\n", "> * **output_dir**：训练过程中，模型保存的目录。\n", "> * **checkpointing_steps**：每隔多少步保存模型，默认为`100`步。\n", "> * **learning_rate**：训练使用的学习率，当我使用 `LoRA` 训练模型的时候，我们需要使用更大的学习率，因此我们这里使用 `1e-4` 而不是 `2e-6`。\n", "> * **max_train_steps**：最大训练的步数，默认为`500`步。\n", " \n", "👉可选修改的参数：\n", "> * **train_batch_size**：训练时候使用的 `batch_size`，当我们的GPU显存比较大的时候可以加大这个值，默认值为`4`。\n", "> * **gradient_accumulation_steps**：梯度累积的步数，当我们GPU显存比较小的时候还想模拟大的训练批次，我们可以适当增加梯度累积的步数，默认值为`1`。\n", "> * **seed**：随机种子，设置后可以复现训练结果。\n", "> * **lora_rank**：`LoRA` 层的 `rank` 值，默认值为4，最终我们会得到 **3.5MB** 的模型，我们可以适当修改这个值，如：`32、64、128、256` 等。\n", "> * **lr_scheduler**：学习率衰减策略，可以是`\"linear\", \"constant\", \"cosine\"`等。\n", "> * **lr_warmup_steps**：学习率衰减前，`warmup` 到最大学习率所需要的步数。\n", " \n", "👉训练过程中评估使用的参数：\n", "> * **num_validation_images**：训练的过程中，我们希望返回多少张图片，默认值为`4`张图片。\n", "> * **validation_prompt**：训练的过程中我们会评估训练的怎么样，因此我们需要设置评估使用的 `prompt` 文本。\n", "> * **validation_steps**：每隔多少个 `steps` 评估模型，我们可以查看训练的进度条，知道当前到了第几个 `steps`。\n", " \n", "🔥**Tips:**\n", " 训练过程中会每隔 `validation_steps` 将生成的图片保存到 `{你指定的输出路径}/validation_images/{步数}.jpg`\n", "\n", "
\n", " \n", "
\n", " \n", "👉权重上传的参数：\n", "> * **push_to_hub**: 是否将模型上传到 `huggingface hub`，默认值为 `False`。\n", "> * **hub_token**: 上传到 `huggingface hub` 所需要使用的 `token`，如果我们已经登录了，那么我们就无需填写。\n", "> * **hub_model_id**: 上传到 `huggingface hub` 的模型库名称， 如果为 `None` 的话表示我们将使用 `output_dir` 的名称作为模型库名称。\n", "\n", "在下面的例子中，由于我们前面已经登录了，因此我们可以开启 **push_to_hub** 按钮，将最终训练好的模型同步上传到 `huggingface.co`\n", " \n", "当我们开启`push_to_hub`后，等待程序运行完毕后会自动将权重上传到这个路径 https://huggingface.co/{你的用户名}/{你指定的输出路径} ，例如： https://huggingface.co/peteli/hometown\n", "\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "!python train_dreambooth_lora.py \\\n", " --pretrained_model_name_or_path=\"runwayml/stable-diffusion-v1-5\" \\\n", " --instance_data_dir=\"./hometown\" \\\n", " --output_dir=\"lora_outputs\" \\\n", " --instance_prompt=\"a sea of purple and gold flowers\" \\\n", " --resolution=512 \\\n", " --train_batch_size=1 \\\n", " --gradient_accumulation_steps=1 \\\n", " --checkpointing_steps=100 \\\n", " --learning_rate=1e-4 \\\n", " --lr_scheduler=\"constant\" \\\n", " --lr_warmup_steps=0 \\\n", " --max_train_steps=1000 \\\n", " --seed=0 \\\n", " --lora_rank=4 \\\n", " --push_to_hub=False \\\n", " --validation_prompt=\"a sea of lavender and gold flowers in the world of fairy tales is my hometown\" \\\n", " --validation_steps=100 \\\n", " --num_validation_images=4" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3.3 可视化训练过程\n", "\n", "VisualDL使用参考：[官方教程](https://aistudio.baidu.com/aistudio/projectdetail/1990920)\n", "\n", "我们可以参照如图所示的步骤，开启visualdl，然后查看训练过程中的指标变化。\n", "\n", "
\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3.4 挑选满意的权重上传至Huggingface\n", "\n", "参数解释：\n", "> * **upload_dir**：我们需要上传的文件夹目录。\n", "> * **repo_name**：我们需要上传的repo名称，最终我们会上传到 https://huggingface.co/{你的用户名}/{你指定的repo名称}， 例如： https://huggingface.co/peteli/hometown.\n", "> * **pretrained_model_name_or_path**：训练该模型所使用的基础模型。\n", "> * **prompt**：搭配该权重需要使用的Prompt文本。\n", "
\n", " \n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "from utils import upload_lora_folder\n", "upload_dir = \"lora_outputs\" # 我们需要上传的文件夹目录\n", "repo_name = \"hometown\" # 我们需要上传的repo名称\n", "pretrained_model_name_or_path = \"runwayml/stable-diffusion-v1-5\" # 训练该模型所使用的基础模型\n", "prompt = \"a sea of lavender and gold flowers in the world of fairy tales is my hometown\" # 搭配该权重需要使用的Prompt文本\n", "\n", "upload_lora_folder(\n", " upload_dir=upload_dir,\n", " repo_name=repo_name,\n", " pretrained_model_name_or_path=pretrained_model_name_or_path,\n", " prompt=prompt, \n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 4.效果展示\n", "\n", "#### 图1：\n", "\n", "![](https://ai-studio-static-online.cdn.bcebos.com/b521bc52e4bb4e63b98a2f32a9fd65cbde84088594084e548e25931339a34e0b)\n", "\n", "#### 图2：\n", "\n", "![](https://ai-studio-static-online.cdn.bcebos.com/fb2518e3c06b4da5a41973cf6dc81684020e9efd3bfc4b9289163beb774dc530)\n", "\n", "#### 图3：\n", "\n", "![](https://ai-studio-static-online.cdn.bcebos.com/b845766ce8b944aead7430146be1ab1fd721456bdce14fe698b7c6978555fa36)\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "py35-paddle1.2.0" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" }, "toc-autonumbering": false, "toc-showcode": false, "toc-showtags": false }, "nbformat": 4, "nbformat_minor": 4 }