Simpson扩散 v2.0

*在与"辛普森一家"相关的图像上进行微调的稳定扩散 v2.0

如果您想了解如何生成您自己的带标题数据集的更多细节，请参阅此 colab

训练是使用Hugging-Face的文本到图像训练的稍作修改的版本完成的 example script

关于

输入文本提示并生成卡通/辛普森风格的图片

自动 webui 检查点

main 文件夹包含一个 .ckpt 文件和一个 .yaml 文件，将它们放入 stable-diffusion-webui "stable-diffusion-webui/models/Stable-diffusion" 文件夹中并用于生成图片

示例代码

from diffusers import StableDiffusionPipeline, LMSDiscreteScheduler
import torch

# this will substitute the default PNDM scheduler for K-LMS  
lms = LMSDiscreteScheduler(
    beta_start=0.00085, 
    beta_end=0.012, 
    beta_schedule="scaled_linear"
)

guidance_scale=8.5
seed=777
steps=50

cartoon_model_path = "Norod78/sd2-simpsons-blip"
cartoon_pipe = StableDiffusionPipeline.from_pretrained(cartoon_model_path, scheduler=lms, torch_dtype=torch.float16)
cartoon_pipe.to("cuda")

def generate(prompt, file_prefix ,samples):
    torch.manual_seed(seed)
    prompt += ", Very detailed, clean, high quality, sharp image"
    cartoon_images = cartoon_pipe([prompt] * samples, num_inference_steps=steps, guidance_scale=guidance_scale)["images"]
    for idx, image in enumerate(cartoon_images):
        image.save(f"{file_prefix}-{idx}-{seed}-sd2-simpsons-blip.jpg")

generate("An oil painting of Snoop Dogg as a simpsons character", "01_SnoopDog", 4)
generate("Gal Gadot, cartoon", "02_GalGadot", 4)
generate("A cartoony Simpsons town", "03_SimpsonsTown", 4)
generate("Pikachu with the Simpsons, Eric Wallis", "04_PikachuSimpsons", 4)

数据集和训练

在 BLIP captioned Simpsons images 上，使用 1xA5000 GPU 在我的家用台式电脑上对 stabilityai/stable-diffusion-2-base 进行了 10,000 次迭代的微调

训练者是 @Norod78