OpenPose only detects human key points such as positions of the head, arms, etc. You can see the dancing man became a woman, but the outline and hairstyle are preserved. It tends to translate the scene more faithfully. What’s the difference between using Canny edge detection and Openpose? The Canny edge detector extracts the edges of the subject and background alike. Images are generated based on these two conditionings. It is then fed to Stable Diffusion as an extra conditioning together with the text prompt. Keypoints are extracted from the input image using OpenPose, and saved as a control map containing the positions of key points. Input image annotated with human pose detection using Openpose.īelow is the ControlNet workflow using OpenPose. Openpose is a fast human keypoint detection model that can extract human poses like positions of hands, legs, and head. Human pose detection exampleĮdge detection is not the only way an image can be preprocessed. The process of extracting specific information (edges in this case) from the input image is called annotation (in the research article) or preprocessing (in the ControlNet extension). Stable Diffusion ControlNet with Canny edge conditioning.

It is fed into the ControlNet model as an extra conditioning to the text prompt.

An image containing the detected edges is then saved as a control map. Edge detection exampleĪs illustrated below, ControlNet takes an additional input image and detects its outlines using the Canny edge detector. Let me show you two examples of what ControlNet can do: Controlling image generation with (1) edge detection and (2) human pose detection. The extra conditioning can take many forms in ControlNet. It uses text prompts as the conditioning to steer image generation so that you generate images that match the text prompt.ĬontrolNet adds one more conditioning in addition to the text prompt. The most basic form of using Stable Diffusion models is text-to-image. You can use ControlNet along with any Stable Diffusion models.

Difference between the Stable Diffusion depth model and ControlNetĬontrolNet is a neural network model for controlling Stable Diffusion models.

Install ControlNet extension (Windows/Mac).Install ControlNet on Windows PC or Mac.

Scribble It! instaling