The process of extracting specific information (edges in this case) from the input image is called annotation (in the research article) or preprocessing (in the ControlNet extension). Stable Diffusion ControlNet with Canny edge conditioning. It is fed into the ControlNet model as an extra conditioning to the text prompt. An image containing the detected edges is then saved as a control map. Edge detection exampleĪs illustrated below, ControlNet takes an additional input image and detects its outlines using the Canny edge detector. Let me show you two examples of what ControlNet can do: Controlling image generation with (1) edge detection and (2) human pose detection. The extra conditioning can take many forms in ControlNet. It uses text prompts as the conditioning to steer image generation so that you generate images that match the text prompt.ĬontrolNet adds one more conditioning in addition to the text prompt. The most basic form of using Stable Diffusion models is text-to-image. You can use ControlNet along with any Stable Diffusion models. Difference between the Stable Diffusion depth model and ControlNetĬontrolNet is a neural network model for controlling Stable Diffusion models.Install ControlNet extension (Windows/Mac).
