AnimateDiffでエラー!発生した場合の回避方法(Google colab) RuntimeError: Error(s) in loading state_dict for CLIPTextModel

20230720時点のGoogle colaboratory上でAnimateDiffを実行するとエラーが発生するため回避策を記載。
Google colaboratoryリンク
https://colab.research.google.com/github/Linaqruf/sd-notebook-collection/blob/main/AnimateDiff.ipynb

GithubでIssueとして議論されていた。https://github.com/guoyww/AnimateDiff/issues/57

エラー内容

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/content/AnimateDiff/scripts/animate.py", line 180, in <module>
    main(args)
  File "/content/AnimateDiff/scripts/animate.py", line 104, in main
    pipeline.text_encoder = convert_ldm_clip_checkpoint(base_state_dict)
  File "/content/AnimateDiff/animatediff/utils/convert_from_ckpt.py", line 726, in convert_ldm_clip_checkpoint
    text_model.load_state_dict(text_model_dict)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CLIPTextModel:
	Unexpected key(s) in state_dict: "text_model.embeddings.position_ids". 

回避方法

以下のコードを開いて、コードを修正
/content/AnimateDiff/animatediff/utils/convert_from_ckpt.py

text_model.load_state_dict(text_model_dict)
を
text_model.load_state_dict(text_model_dict, strict=False)
にする

def convert_ldm_clip_checkpoint(checkpoint):
    text_model = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14")
    keys = list(checkpoint.keys())

    text_model_dict = {}

    for key in keys:
        if key.startswith("cond_stage_model.transformer"):
            text_model_dict[key[len("cond_stage_model.transformer.") :]] = checkpoint[key]

    text_model.load_state_dict(text_model_dict, strict=False)

    return text_model

動作検証

Pass