Paperspaceで Stable Diffusion WebUI Forge を動かす

2024年2月11日 15:29

最近「Stable Diffusion WebUI Forge」が話題です。試してみたいので、Paperespaceで動かしてみます。手順は、Stable Diffusion WebUI と同じでや
ってみます。

Stable Diffusion Webuiを動かしたときの記事はこちら

Stable Diffusion WebUI Forge のダウンロード

新しいプロジェクト、ノートブックを作成し、設定用のファイル(webui2.ipynb)をアップロードする、という一連の手順は、前回と同様に、@javacommonsさんの記事（【Paperspace】Stable Diffusion Web UI を月額定額で使う )に沿って進めます。

ダウンロードするパスが
https://github.com/AUTOMATIC1111/stable-diffusion-webui
から
https://github.com/lllyasviel/stable-diffusion-webui-forge.git
に変わるので、設定ファイルを変更しました。（stable-diffusion-webui を stable-diffusion-webui-forge に変更）

#(1) AUTOMATIC1111のダウンロード
%cd /notebooks
!rm -rf stable-diffusion-webui-forge
!git clone https://github.com/lllyasviel/stable-diffusion-webui-forge.git


/notebooks
Cloning into 'stable-diffusion-webui-forge'...
remote: Enumerating objects: 36764, done.
remote: Counting objects: 100% (1605/1605), done.
remote: Compressing objects: 100% (362/362), done.
remote: Total 36764 (delta 1308), reused 1473 (delta 1225), pack-reused 35159
Receiving objects: 100% (36764/36764), 43.45 MiB | 34.44 MiB/s, done.
Resolving deltas: 100% (25209/25209), done.

モデルのダウンロードは後にして、起動できるか見てみます。
v1.6 のときと同様に、launchi.py の引数に --reinstall-xformers　を起動コマンドに加えています。

#(3) WebUI起動
!apt update
!apt -y install python3.10
!apt -y install libpython3.10-dev
!apt -y install build-essential
!apt -y install ffmpeg
!curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10
!python3.10 -m pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 torchaudio==2.0.2+cu118 torchtext==0.15.2 torchdata==0.6.1 --extra-index-url https://download.pytorch.org/whl/cu118 -U
!python3.10 -m pip install xformers==0.0.20 triton==2.0.0 -U
!python3.10 -m pip install httpx==0.24.1
#!python3.10 -m pip install insightface -U
!python3.10 -m pip install matplotlib -U
!python3.10 -m pip install ipython -U
from IPython import get_ipython
get_ipython().run_line_magic('matplotlib', 'inline')
%cd /notebooks/stable-diffusion-webui-forge
!python3.10 launch.py --xformers --enable-insecure-extension-access --share --gradio-queue --reinstall-xformers


中略

Launching Web UI with arguments: --xformers --enable-insecure-extension-access --share --gradio-queue --reinstall-xformers
Total VRAM 16273 MB, total RAM 30068 MB
WARNING:xformers:WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
    PyTorch 2.1.2+cu121 with CUDA 1201 (you have 2.0.1+cu118)
    Python  3.10.13 (you have 3.10.13)
  Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
  Memory-efficient attention, SwiGLU, sparse and more won't be available.
  Set XFORMERS_MORE_DETAILS=1 for more details
xformers version: 0.0.23.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 Quadro P5000 : native
VAE dtype: torch.float32
Using pytorch cross attention
==============================================================================
You are running torch 2.0.1+cu118.
The program is tested to work with torch 2.1.2.
To reinstall the desired version, run with commandline flag --reinstall-torch.
Beware that this will cause a lot of large files to be downloaded, as well as
there are reports of issues with training tab on the latest version.

Use --skip-version-check commandline argument to disable this check.
==============================================================================
ControlNet preprocessor location: /notebooks/stable-diffusion-webui-forge/models/ControlNetPreprocessor
Loading weights [15012c538f] from /notebooks/stable-diffusion-webui-forge/models/Stable-diffusion/realisticVisionV51_v51VAE.safetensors
2024-02-10 14:00:26,079 - ControlNet - INFO - ControlNet UI callback registered.
Running on local URL:  http://127.0.0.1:7860
model_type EPS
UNet ADM Dimension 0
Running on public URL: https://1d84438d28b0451a7d.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)
Startup time: 167.1s (prepare environment: 140.8s, import torch: 5.3s, import gradio: 1.7s, setup paths: 1.0s, initialize shared: 0.3s, other imports: 1.9s, load scripts: 4.8s, scripts list_optimizers: 0.2s, create ui: 2.3s, gradio launch: 8.4s).
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
To load target model SD1ClipModel
Begin to load 1 model
Model loaded in 19.0s (load weights from disk: 10.2s, forge load real models: 7.9s, calculate empty prompt: 0.8s).

Public URLにアクセスし、起動画面を見ると、表示項目が増えていて、バージョン表記も異なります。

VAEや、Clip skipの設定は反映済だったので、v1.6の時のような追加設定は不要でした。

SD VAE, Clip skipは反映済

モデルの配置

モデルはＢＤさんのBlazingRealDrive _V03を使わせていただきます。

ローカルにダウンロードして、GoogleDriveに置いて、Paperspaceからgdownでダウンロードします。

gdown のインストール

root@njofuectfw:/notebooks/stable-diffusion-webui-forge# pip install gdown 
Collecting gdown
  Downloading gdown-5.1.0-py3-none-any.whl.metadata (5.7 kB)
Collecting beautifulsoup4 (from gdown)
  Downloading beautifulsoup4-4.12.3-py3-none-any.whl.metadata (3.8 kB)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from gdown) (3.13.1)
Requirement already satisfied: requests[socks] in /usr/local/lib/python3.10/dist-packages (from gdown) (2.31.0)
Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from gdown) (4.66.1)
Collecting soupsieve>1.2 (from beautifulsoup4->gdown)
  Downloading soupsieve-2.5-py3-none-any.whl.metadata (4.7 kB)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests[socks]->gdown) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests[socks]->gdown) (2.8)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests[socks]->gdown) (2.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests[socks]->gdown) (2019.11.28)
Collecting PySocks!=1.5.7,>=1.5.6 (from requests[socks]->gdown)
  Downloading PySocks-1.7.1-py3-none-any.whl (16 kB)
Downloading gdown-5.1.0-py3-none-any.whl (17 kB)
Downloading beautifulsoup4-4.12.3-py3-none-any.whl (147 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 147.9/147.9 kB 11.7 MB/s eta 0:00:00
Downloading soupsieve-2.5-py3-none-any.whl (36 kB)
Installing collected packages: soupsieve, PySocks, beautifulsoup4, gdown
Successfully installed PySocks-1.7.1 beautifulsoup4-4.12.3 gdown-5.1.0 soupsieve-2.5
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@njofuectfw:/notebooks/stable-diffusion-webui-forge#

BlazingRealDrive _V03ダウンロード（アップロード）

gdownでダウンロードを実行しようとするとエラーになりました。これについては様子を見ることにして、ローカルのPCから直接アップロードすることにします。時間はかかりますが、、、

root@njofuectfw:/notebooks/stable-diffusion-webui-forge/models/Stable-diffusion# gdown '1p-wVPn0qRcx-WrDA5bWbVvhG52rpgatb&confirm=t'
Failed to retrieve file url:

        Cannot retrieve the public link of the file. You may need to change
        the permission to 'Anyone with the link', or have had many accesses.
        Check FAQ in https://github.com/wkentaro/gdown?tab=readme-ov-file#faq.

You may still be able to access the file from the browser:

        https://drive.google.com/uc?id=1p-wVPn0qRcx-WrDA5bWbVvhG52rpgatb&confirm=t

but Gdown can't. Please check connections and permissions.

（翌日追記）改めて試したら成功しました。

root@n8ho6dcrde:/notebooks/stable-diffusion-webui-forge/models/Stable-diffusion# gdown '1p-wVPn0qRcx-WrDA5bWbVvhG52rpgatb&confirm=t'
Downloading...
From: https://drive.google.com/uc?id=1p-wVPn0qRcx-WrDA5bWbVvhG52rpgatb&confirm=t
To: /notebooks/stable-diffusion-webui-forge/models/Stable-diffusion/_BRD_V03j.safetensors
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.13G/2.13G [00:20<00:00, 104MB/s]
root@n8ho6dcrde:/notebooks/stable-diffusion-webui-forge/models/Stable-diffusion# gdown '1ShSKeot8YsdQbyEZuZJNw_ivSVbZ_TCf&confirm=t'
Downloading...
From: https://drive.google.com/uc?id=1ShSKeot8YsdQbyEZuZJNw_ivSVbZ_TCf&confirm=t
To: /notebooks/stable-diffusion-webui-forge/models/Stable-diffusion/_BlazingDrive_V13md.safetensors
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.13G/2.13G [00:17<00:00, 119MB/s]
root@n8ho6dcrde:/notebooks/stable-diffusion-webui-forge/models/Stable-diffusion#

VAEのダウンロード（アップロード）

こちらも直接アップロードします。以前よりも早くなった感じです。

画像生成

512 x 768の画像を Restartで50step + Hires.fix も 50step で、約10分くらいです。特に高速になった感じはありません。

masterpiece, ultra high res, absurdres, (photo realistic), A young female fashion model, BREAK, (silk blouse:1.2), (flower print skirt:1.1), flat shoes, (dynamic pose:0.7), simple background,dutch angle, Negative prompt: Painting, sketches logo, watermark, text, (worst quality), (low quality:2), (normal quality:2), lowers.normal quality, skin spots acnes, skin blemishes, agespot, (nsfw:2), ugly face, breasts out, nipple, too much muscle, extra arms, too much fingers, extra legs, fused fingers, wrinkle, Japanese kimono,blush, bold line,outline, Steps: 50, Sampler: Restart, CFG scale: 7, Seed: 3052008151, Size: 512x768, Model hash: c2f257d26b, Model: _BRD_V03j, VAE hash: 735e4c3a44, VAE: vae-ft-mse-840000-ema-pruned.safetensors, Denoising strength: 0.45, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+, Version: f0.0.12-latest-109-g6e71d974 Time taken: 9 min. 52.5 sec. A: 6.17 GB, R: 8.52 GB, Sys: 8.8/15.8916 GB (55.2%)

masterpiece, ultra high res, absurdres, (photo realistic), A young female fashion model, armond eyes, BREAK, (Black leather pants). They are tight and create a cool impression. (A gray knit sweater:1.1). You wear it over the pants and tuck it in front. It is made of a loose material, and makes it look casual. (Black ankle boots). You match the color of the pants and create a sense of unity in your feet. They have heels, and make you look stylish. (A silver necklace and silver earrings). You choose a thin and long necklace and show it subtly on the neck of the sweater. The earrings are large hoop ones, and add accents to your ears. (Short cut). You do not tuck your hair behind your ears, but make it fluffy around your face. (Smoky makeup). You make your skin look matte. You use a brown blush to create a small face effect. You use a black eyeshadow to create a dark shadow on your eyes. You draw a thick black eyeliner. You use a black mascara to volumize your eyelashes. You use a black lip to make it look dark. (dynamic pose:0.7), simple background,dutch angle, Negative prompt: Painting, sketches logo, watermark, text, (worst quality), (low quality:2), (normal quality:2), lowers.normal quality, skin spots acnes, skin blemishes, agespot, (nsfw:2), ugly face, breasts out, nipple, too much muscle, extra arms, too much fingers, extra legs, fused fingers, wrinkle, Japanese kimono,blush, bold line,outline, Steps: 50, Sampler: Restart, CFG scale: 6.5, Seed: 371721034, Size: 512x768, Model hash: c2f257d26b, Model: _BRD_V03j, VAE hash: 735e4c3a44, VAE: vae-ft-mse-840000-ema-pruned.safetensors, Denoising strength: 0.45, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+, Version: f0.0.12-latest-109-g6e71d974 Time taken: 9 min. 52.7 sec. A: 6.19 GB, R: 8.51 GB, Sys: 8.8/15.8916 GB (55.1%)

次の日、同様に生成したところ、2分弱と高速でした。

masterpiece, ultra high res, absurdres, (photo realistic), A young female fashion model, armond eyes, BREAK, (A navy check skirt:1.1). It is above the knee length, and has pleats. It is a classic and elegant item. (A white blouse:1.1). You tuck it into the skirt and have a ribbon on the collar. It is a simple and clean design, and matches the color of the skirt. (A navy knit vest:1.1). You wear it over the blouse and leave it open. It is made of a thick and warm material, and can be worn from winter to spring. (White socks). You contrast them with the color of the skirt and create brightness in your feet. They are made of a thin and smooth material, and are comfortable to wear. (Navy loafers). You match the color of the skirt and create a sense of unity in your feet. They are flat and easy to walk in, and can fit both casual and formal occasions. (Ponytail). You tie your hair high at the back, and make it fluffy. You do not have bangs, and tuck your hair behind your ears. (Cute makeup). You use a white foundation to make your skin look transparent. You apply an orange blush to give a pop of color to your cheeks. You use a blue eyeshadow to add color to your eyes. You draw a thin brown eyeliner. You use a brown mascara to lift your eyelashes. You use an orange lip to make it look lively. (dynamic pose:0.7), simple background,dutch angle, Negative prompt: Painting, sketches logo, watermark, text, (worst quality), (low quality:2), (normal quality:2), lowers.normal quality, skin spots acnes, skin blemishes, agespot, (nsfw:2), ugly face, breasts out, nipple, too much muscle, extra arms, too much fingers, extra legs, fused fingers, wrinkle, Japanese kimono,blush, bold line,outline, Steps: 50, Sampler: Restart, CFG scale: 6.5, Seed: 4264245214, Size: 512x768, Model hash: c2f257d26b, Model: _BRD_V03j, VAE hash: 735e4c3a44, VAE: vae-ft-mse-840000-ema-pruned.safetensors, Denoising strength: 0.45, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+, Version: f0.0.12-latest-109-g6e71d974 Time taken: 1 min. 42.2 sec. A: 5.31 GB, R: 7.12 GB, Sys: 7.7/15.7314 GB (48.7%)

FreeU

FreeU Integrated の設定画面は、以下でした。

数値はそのままにして、Enabledにします。

ほとんど違いが分りません。FreeUのサイトの参考値を設定してみます。

結果は、やや濃い目になりました。

ハマりそうなのでこの辺りで止めておきます。他のパラメーターについても、追々調べます。

（一応対応）Infinite Image Browsingが動かない

生成した画像のブラウジングするInfinite Image Browsingを入れたのですが、ディレクトリが表示されません。

本当は以下のように表示されるはずです。

.env.exampleに以下の記述がありました。

# This document is used for additional control over the security and privacy of IIB. By default, IIB is already sufficiently secure, and you do not need to take any additional actions.
# Copy this file and rename it to ".env"  . Then, fill in the necessary values for your specific use case.
# Remember to never share your .env file with anyone else as it may contain sensitive information.

# This attribute is used for authentication. If you input a key here, it will be validated for authentication purposes.
# It will be prompted to enter your key when you open the extension. If the authentication fails, all your requests will be rejected.
IIB_SECRET_KEY=

# Configuring the server-side language for this extension,
# including the tab title and most of the server-side error messages returned. Options are 'zh', 'en', or 'auto'.
# If you want to configure the language for the front-end pages, please set it on the extension's global settings page.
IIB_SERVER_LANG=auto


# ---------------------------- ACCESS_CONTROL ----------------------------

# Used to configure whether to enable access control to the file system.
# If enabled, only access to the provided pre-set folders (including those provided by sd-webui and manually \
# added to Quick Move or specified via IIB_ACCESS_CONTROL_ALLOWED_PATHS) will be allowed.
# The available options are 'enable', 'disable', and 'auto'.
# The default value is 'auto', which will be determined based on the command-line parameters used to start sd-webui (such as --server-name, --share, --listen).
IIB_ACCESS_CONTROL=auto

# This variable is used to define a list of allowed paths for the application to access when access control mode is enabled. 
# It can be set to a comma-separated string of file paths or directory paths, representing the resources that are allowed to be accessed by the application.
# In addition, if sd_webui_config or sd_webui_dir has been configured, or if you're running this repository as an extension of sd-webui, 
# you can use the following shortcuts (txt2img, img2img, extra, save) as values for the ALLOWED_PATHS variable.
# IIB_ACCESS_CONTROL_ALLOWED_PATHS=save,extra,/output   ...etc

# This variable is used to control fine-grained access control for different types of requests, but only if access control mode is enabled. 
# It can be set to a string value that represents a specific permission or set of permissions, such as "read-only", "write-only", "read-write", or "no-access". 
# This variable can be used to restrict access to certain API endpoints or data sources based on the permissions required by the user. 
# IIB_ACCESS_CONTROL_PERMISSION=read-write

アクセスコントロールがかかっているとのことなので、これを外してみます。
IIB_ACCESS_CONTROL=disable
として .env を /notebooks/stable-diffusion-webui-forge/extensions/sd-webui-infinite-image-browsing/ 以下に置き、再起動したところ、「Launch from Quick Move」が表示され、Pathの追加もできました。

アクセスコントロールを disable にするのは好ましくないですが、Paperspaceは最大でも6時間しか稼働しないこと、特に秘密情報がないことから、当面はこれで運用します。もっと詳細に設定ができるようなので、これは追々調べます。

生成速度の比較

これまで使っていたWebUI v1.7での生成結果は10分弱でした。これと比べてForgeの2分は短いです。しかし、Forgeでも昨日は10分くらいかかっていたので、日によって違うのか、様子を見ます。

(追記）スピードの違いは、Machineの違いによるようです。
Forgeでは、Free-A4000 ( 45GiB RAM , 8CPU, 16GiB GPU ) だと２分くらい、Free-P5000 だと１０分くらい。P5000だと違いが見られません。

この記事が気に入ったらサポートをしてみませんか？