ControlNet image sequence 2 image sequence script v2

ControlNet image sequence 2 image sequence script v2

Added TemporalNet controlnet model support. Using this, it is possible to increase consistency between image sequences by inputting images from previous frames.

Download the model file from here and put it in the same location as your other controlnet models.

diff_control_sd15_temporalnet_fp16.safetensors · CiaraRowles/TemporalNet at main (huggingface.co)

To use it, enable 'Use TemporalNet'. Then select the TempralNet ControlNet model in addition to the end of the list of ControlNet models you need.

A weight of 0.6 or less is recommended.

Since the first frame does not have a previous frame to refer to, it is necessary to separately insert an image into the img2img input part.

It is not recommended to use TemporalNet simultaneously with 'Feed previous frame' because of the risk of distorting the image.

This is a script I modified to fit my personal workflow.

Original script : sd-webui-controlnet/movie2movie.py at main · Mikubill/sd-webui-controlnet (github.com)

This script requires the ControlNet extension.

Mikubill/sd-webui-controlnet: WebUI extension for ControlNet (github.com)

Install

1. Put the downloaded script file in the following path. 'stable-diffusion-webui\scripts'

2. Select controlnet I2I sequence-toyxyz from the Script list in the img2img tab

Requirements

OpenCV: pip install opencv-python

First of all, you need to put 1 image into the Img2Img input at the top.

Detail explanation

Feed previous frame: Slightly reduce flickering by inputting the generated previous frame image to Img2Img again. Lowering the denoising strength may reduce the flickering a bit more, but may cause color problems.

Blend color image: Blends a color image sequence with a previous frame image or an image uploaded to Img2Img.

Blend mode: Choose how color images are blended. Personally, Softlight and Overlay were the most useful.

Opacity: Adjusts the opacity of color images to be mixed.

ControlNet-0~n: The path where the image sequence to be input to ControlNet is located.

Color_Image: The path where the color image sequence is located

Output_path : The path where the generated image sequence will be saved. You need to be careful because all existing contents will be deleted. Create a new path if it does not exist.