|
<!DOCTYPE html> |
|
<html> |
|
<head> |
|
<title>WebSD | Home</title> |
|
<meta charset="utf-8" /> |
|
<meta |
|
name="viewport" |
|
content="width=device-width, initial-scale=1, shrink-to-fit=no" |
|
/> |
|
<link |
|
rel="stylesheet" |
|
href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-beta.2/css/bootstrap.min.css" |
|
integrity="sha384-PsH8R72JQ3SOdhVi3uxftmaW6Vc51MKb0q5P2rRUpPvrszuE4W1povHYgTpBfshb" |
|
crossorigin="anonymous" |
|
/> |
|
<link |
|
rel="stylesheet" |
|
href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" |
|
/> |
|
<link rel="stylesheet" href="/assets/css/main.css" /> |
|
<link rel="stylesheet" href="/assets/css/group.css" /> |
|
|
|
|
|
<meta |
|
http-equiv="origin-trial" |
|
content="Agx76XA0ITxMPF0Z8rbbcMllwuxsyp9qdtQaXlLqu1JUrdHB6FPonuyIKJ3CsBREUkeioJck4nn3KO0c0kkwqAMAAABJeyJvcmlnaW4iOiJodHRwOi8vbG9jYWxob3N0Ojg4ODgiLCJmZWF0dXJlIjoiV2ViR1BVIiwiZXhwaXJ5IjoxNjkxNzExOTk5fQ==" |
|
/> |
|
<meta |
|
http-equiv="origin-trial" |
|
content="AnmwqQ1dtYDQTYkZ5iMtHdINCaxjE94uWQBKp2yOz1wPTcjSRtOHUGQG+r2BxsEuM0qhxTVnuTjyh31HgTeA8gsAAABZeyJvcmlnaW4iOiJodHRwczovL21sYy5haTo0NDMiLCJmZWF0dXJlIjoiV2ViR1BVIiwiZXhwaXJ5IjoxNjkxNzExOTk5LCJpc1N1YmRvbWFpbiI6dHJ1ZX0=" |
|
/> |
|
<script src="dist/tvmjs_runtime.wasi.js"></script> |
|
<script src="dist/tvmjs.bundle.js"></script> |
|
</head> |
|
<body> |
|
<div class="container"> |
|
|
|
<div |
|
class="header d-flex flex-column flex-md-row justify-content-md-between" |
|
> |
|
<a href="/" id="navtitle"> |
|
<img |
|
src="/assets/img/logo/mlc-logo-with-text-landscape.svg" |
|
height="70px" |
|
alt="MLC" |
|
id="logo" |
|
/> |
|
</a> |
|
<ul id="topbar" class="nav nav-pills justify-content-center"> |
|
<li class="nav-item"> |
|
<a class="nav-link active" href="/"> Home </a> |
|
</li> |
|
|
|
<li class="nav-item"> |
|
<a |
|
class="nav-link" |
|
href="https://github.com/mlc-ai/web-stable-diffusion" |
|
> |
|
Github |
|
</a> |
|
</li> |
|
</ul> |
|
</div> |
|
|
|
|
|
|
|
|
|
<h1 id="web-stable-diffusion">Web Stable Diffusion </h1> |
|
|
|
<p> |
|
This project brings stable diffusion models to web browsers. |
|
<strong>Everything runs inside the browser with no need of server |
|
support.</strong> |
|
To our knowledge, this is the the world’s first stable diffusion |
|
completely running on the browser. Please check out our |
|
<a href="https://github.com/mlc-ai/web-stable-diffusion">GitHub repo</a> |
|
to see how we did it. There is also a |
|
<a href="#text-to-image-generation-demo">demo</a> which you can try out. |
|
</p> |
|
<li> |
|
WebGPU spec does comes with FP16 support already, but the |
|
implementation does not yet support this feature at this moment. As a |
|
result, the memory consumption of running the demo is about 7GB. For |
|
Apple silicon Mac with only 8GB of unified memory, it may take longer |
|
(a few minutes) to generate an image. This demo may also work for Mac |
|
with AMD GPU. |
|
</li> |
|
<h3 id="demo">Demo</h3> |
|
|
|
<script> |
|
var tvmjsGlobalEnv = tvmjsGlobalEnv || {}; |
|
</script> |
|
|
|
<script type="module"> |
|
import init, { |
|
TokenizerWasm, |
|
} from "./dist/tokenizers-wasm/tokenizers_wasm.js"; |
|
|
|
var initialized = false; |
|
async function getTokenizer(name) { |
|
if (!initialized) { |
|
await init(); |
|
} |
|
const jsonText = await ( |
|
await fetch( |
|
"https://huggingface.co/" + name + "/raw/main/tokenizer.json" |
|
) |
|
).text(); |
|
return new TokenizerWasm(jsonText); |
|
} |
|
|
|
tvmjsGlobalEnv.getTokenizer = getTokenizer; |
|
</script> |
|
<div> |
|
Select Model: <select name="model" id="modelId"> |
|
<option selected="selected">Stable-Diffusion-1.5</option> |
|
<option disabled>Stable-Diffusion-XL</option> |
|
</select> |
|
</div> |
|
<script src="dist/stable_diffusion.js"></script> |
|
|
|
<div> |
|
Input prompt: |
|
<input |
|
name="inputPrompt" |
|
id="inputPrompt" |
|
type="text" |
|
value="A photo of an astronaut riding a horse on mars" |
|
size="77" |
|
/> |
|
<br /> |
|
Negative prompt (optional): |
|
<input |
|
name="negativePrompt" |
|
id="negativePrompt" |
|
type="text" |
|
value="" |
|
size="77" |
|
/> |
|
</div> |
|
|
|
<div> |
|
Select scheduler - |
|
<select name="scheduler" id="schedulerId"> |
|
<option value="0">Multi-step DPM Solver (20 steps)</option> |
|
<option value="1">PNDM (50 steps)</option> |
|
</select> |
|
|
|
<br /> |
|
|
|
Render intermediate steps (may slow down execution) - |
|
<select name="vae-cycle" id="vaeCycle"> |
|
<option value="-1">No</option> |
|
<option value="2">Run VAE every two UNet steps after step 10</option> |
|
</select> |
|
|
|
<div id="progress"> |
|
<label id="gpu-tracker-label"></label><br /> |
|
<label id="progress-tracker-label"></label><br /> |
|
<progress |
|
id="progress-tracker-progress" |
|
max="100" |
|
value="100" |
|
></progress> |
|
</div> |
|
<button onclick="tvmjsGlobalEnv.asyncOnGenerate()">Generate</button> |
|
</div> |
|
<script> |
|
function updateSchedulerBasedOnModel() { |
|
var selectedModel = document.getElementById('modelId').value; |
|
var schedulerDropdown = document.getElementById('schedulerId'); |
|
|
|
if (selectedModel === 'Stable-Diffusion-XL') { |
|
schedulerDropdown.value = '2'; |
|
|
|
for (var i = 0; i < schedulerDropdown.options.length; i++) { |
|
schedulerDropdown.options[i].disabled = (schedulerDropdown.options[i].value !== '2'); |
|
} |
|
} else { |
|
schedulerDropdown.value = '0'; |
|
|
|
for (var i = 0; i < schedulerDropdown.options.length; i++) { |
|
schedulerDropdown.options[i].disabled = (schedulerDropdown.options[i].value !== '0' && schedulerDropdown.options[i].value !== '1'); |
|
} |
|
} |
|
} |
|
|
|
window.onload = function() { |
|
updateSchedulerBasedOnModel(); |
|
document.getElementById('modelId').addEventListener('change', updateSchedulerBasedOnModel); |
|
} |
|
</script> |
|
<div> |
|
<canvas id="canvas" width="512" height="512"></canvas> |
|
</div> |
|
<div id="log"></div> |
|
|
|
<p> |
|
This project brings stable diffusion models to web browsers. |
|
<strong |
|
>Everything runs inside the browser with no need of server |
|
support.</strong |
|
> |
|
To our knowledge, this is the the world’s first stable diffusion |
|
completely running on the browser. Please check out our |
|
<a href="https://github.com/mlc-ai/web-stable-diffusion">GitHub repo</a> |
|
to see how we did it. There is also a |
|
<a href="#text-to-image-generation-demo">demo</a> which you can try out. |
|
</p> |
|
|
|
<p> |
|
<img |
|
src="img/fig/browser-screenshot.png" |
|
alt="Browser screenshot" |
|
width="100%" |
|
/> |
|
</p> |
|
|
|
<p> |
|
We have been seeing amazing progress through AI models recently. Thanks |
|
to the open-source effort, developers can now easily compose open-source |
|
models together to produce amazing tasks. Stable diffusion enables the |
|
automatic creation of photorealistic images as well as images in various |
|
styles based on text input. These models are usually big and |
|
compute-heavy, which means we have to pipe through all computation |
|
requests to (GPU) servers when developing web applications based on |
|
these models. Additionally, most of the workloads have to run on a |
|
specific type of GPUs where popular deep-learning frameworks are readily |
|
available. |
|
</p> |
|
|
|
<p> |
|
This project takes a step to change that status quo and bring more |
|
diversity to the ecosystem. There are a lot of reasons to get some (or |
|
all) of the computation to the client side. There are many possible |
|
benefits, such as cost reduction on the service provider side, as well |
|
as an enhancement for personalization and privacy protection. The |
|
development of personal computers (even mobile devices) is going in the |
|
direction that enables such possibilities. The client side is getting |
|
pretty powerful. For example, the latest MacBook Pro can have up to 96GB |
|
of unified RAM that can be used to store the model weights and a |
|
reasonably powerful GPU to run many of the workloads. |
|
</p> |
|
|
|
<p> |
|
Wouldn’t it be fun to directly bring the ML models to the client, have |
|
the user open a browser tab, and instantly run the stable diffusion |
|
models on the browser? This project provides the first affirmative |
|
answer to this question. |
|
</p> |
|
|
|
<h2 id="text-to-image-generation-demo">Text to Image Generation Demo</h2> |
|
|
|
<p> |
|
Because WebGPU is not yet fully stable, nor have there ever been such |
|
large-scale AI models running on top of WebGPU, so we are testing the |
|
limit here. It may not work in your environment. So far, we have only |
|
tested it on Mac with M1/M2 GPUs in Chrome Canary (a nightly build of |
|
Chrome) because WebGPU is quite new. We have tested on Windows and it |
|
does not work at this moment due to possible driver issues. We |
|
anticipate the support broadens as WebGPU matures. Please check out the |
|
<a href="#instructions">use instructions</a> and |
|
<a href="#notes">notes</a> below. |
|
</p> |
|
|
|
<h3 id="instructions">Instructions</h3> |
|
|
|
<p> |
|
If you have a Mac computer with Apple silicon, here are the instructions |
|
for you to run stable diffusion on your browser locally: |
|
</p> |
|
|
|
<ul> |
|
<li> |
|
Install |
|
<a href="https://www.google.com/chrome/canary/">Chrome Canary</a>, a |
|
developer version of Chrome that enables the use of WebGPU. |
|
</li> |
|
<li> |
|
Launch Chrome Canary. |
|
<strong |
|
>You are recommended to launch from terminal with the following |
|
command:</strong |
|
> |
|
<div class="language-shell highlighter-rouge"> |
|
<div class="highlight"> |
|
<pre |
|
class="highlight" |
|
><code>/Applications/Google<span class="se">\ </span>Chrome<span class="se">\ </span>Canary.app/Contents/MacOS/Google<span class="se">\ </span>Chrome<span class="se">\ </span>Canary <span class="nt">--enable-dawn-features</span><span class="o">=</span>disable_robustness |
|
</code></pre> |
|
</div> |
|
</div> |
|
<p> |
|
This command turns off the robustness check from Chrome Canary that |
|
slows down image generation to times. It is not necessary, but we |
|
strongly recommend you to start Chrome with this command. |
|
</p> |
|
</li> |
|
<li> |
|
Enter your prompt, click “Generate” – we are ready to go! The image |
|
generation will start after downloading and fetching the model |
|
parameters to local cache. The download may take a few minutes, only |
|
for the first run. The subsequent refreshes and runs will be faster. |
|
</li> |
|
<li> |
|
Feel free to enter different prompts as well as negative prompts to |
|
generate the image you want. |
|
</li> |
|
<li> |
|
We provide an option to render images for the intermediate steps of |
|
UNet stage. Select “Run VAE every two UNet steps after step 10” for |
|
“Render intermediate steps” and click “Generate” again, and you will |
|
see how an image gets generated along the process. |
|
</li> |
|
</ul> |
|
|
|
|
|
|
|
<h3 id="notes">Notes</h3> |
|
|
|
<ul> |
|
|
|
<li> |
|
Please check out our |
|
<a href="https://github.com/mlc-ai/web-stable-diffusion" |
|
>GitHub repo</a |
|
> |
|
for running the same shader flow locally on your GPU device through |
|
the native driver. Right now, there are still gaps (e.g., without |
|
launching Chrome from command line, Chrome’s WebGPU implementation |
|
inserts bound clips for all array index access, such that |
|
<code class="language-plaintext highlighter-rouge">a[i]</code> becomes |
|
<code class="language-plaintext highlighter-rouge" |
|
>a[min(i, a.size)]</code |
|
>, which are not optimized out by the downstream shader compilers), |
|
but we believe it is feasible to close such gaps as WebGPU dispatches |
|
to these native drivers. |
|
</li> |
|
</ul> |
|
|
|
<h2 id="disclaimer">Disclaimer</h2> |
|
|
|
<p> |
|
This demo site is for research purposes only. Please conform to the |
|
<a href="https://huggingface.co/runwayml/stable-diffusion-v1-5#uses" |
|
>uses of stable diffusion models</a |
|
>. |
|
</p> |
|
</div> |
|
|
|
|
|
|
|
<script type="text/javascript" src="/assets/js/srcset-polyfill.js"></script> |
|
</body> |
|
</html> |
|
|
