卡通扩散 v2.0

*稳定的Diffusion v2.0，经过各种卡通节目图像的微调

如果您想了解有关如何生成自己的带有标题的数据集的详细信息，请查看此链接 colab

训练使用了Hugging-Face的文本到图像训练的稍微修改版本 example script

关于

输入文本提示并生成卡通图像

自动化1111网络用户界面检查点

main 文件夹包含一个 .ckpt 文件和一个 .yaml 文件，需要将它们放入 stable-diffusion-webui "stable-diffusion-webui/models/Stable-diffusion" 文件夹中，并用于生成图像

示例代码

from diffusers import StableDiffusionPipeline, LMSDiscreteScheduler
import torch

# this will substitute the default PNDM scheduler for K-LMS  
lms = LMSDiscreteScheduler(
    beta_start=0.00085, 
    beta_end=0.012, 
    beta_schedule="scaled_linear"
)

guidance_scale=8.5
steps=50

cartoon_model_path = "Norod78/sd2-cartoon-blip"
cartoon_pipe = StableDiffusionPipeline.from_pretrained(cartoon_model_path, scheduler=lms, torch_dtype=torch.float16)
cartoon_pipe.to("cuda")

def generate(prompt, file_prefix ,samples, seed=42):
    torch.manual_seed(seed)
    prompt += ", Very detailed, clean, high quality, sharp image"
    cartoon_images = cartoon_pipe([prompt] * samples, num_inference_steps=steps, guidance_scale=guidance_scale)["images"]
    for idx, image in enumerate(cartoon_images):
        image.save(f"{file_prefix}-{idx}-{seed}-sd2-cartoon-blip.jpg")

generate("An oil on canvas portrait of Snoop Dogg, Mark Ryden", "01_SnoopDog", 2, 777)
generate("A flemish baroque painting of Kermit from the muppet show", "02_KermitFlemishBaroque", 2, 42)
generate("Gal Gadot in Avatar", "03_GalGadotAvatar", 2, 777)
generate("Ninja turtles, Naoto Hattori", "04_TMNT", 2, 312)
generate("An anime town", "05_AnimeTown", 2, 777)
generate("Family guy taking selfies at the beach", "06_FamilyGuy", 2, 555)
generate("Pikachu as Rick and morty, Eric Wallis", "07_PikachuRnM", 2, 777)
generate("Pikachu as Spongebob, Eric Wallis", "08_PikachuSpongeBob", 2, 42)
generate("An oil painting of Miss. Piggy from the muppets as the Mona Lisa", "09_MsPiggyMonaLisa", 2, 42)
generate("Rick Sanchez in star wars, Dave Dorman", "10_RickStarWars", 2, 42)
generate("An paiting of Southpark with rainbow", "11_Southpark", 2, 777)
generate("An oil painting of Phineas and Pherb hamering on a new machine, Eric Wallis", "12_PhineasPherb", 2, 777)
generate("Bender, Saturno Butto", "13_Bender", 2, 777)
generate("A psychedelic image of Bojack Horseman", "14_Bojack", 2, 777)
generate("A movie poster for Gravity Falls Cthulhu stories", "15_GravityFalls", 2, 777)
generate("A vibrant oil painting portrait of She-Ra", "16_Shira", 2, 512)
#

数据集和训练

在 stabilityai/stable-diffusion-2-base 上对 BLIP captioned cartoon images 进行了 25,000 次微调迭代，使用我家中的1xA5000 GPU进行训练

由 @Norod78 进行训练