Stable Diffusion SDXL v1.0 and ComfyUI: How to Install and Use

In the world of advanced AI image generation, the convergence of technology reaches new heights with the fusion of Stable Diffusion SDXL v1.0 and the revolutionary ComfyUI interface. ComfyUI provides a user-friendly interface to use the advanced image generation capabilities of SDXL 1.0, allowing users to create more realistic and detailed images with improved ease and efficiency.

The combination of SDXL v1.0 and ComfyUI together enables users to experiment with various parameters and create custom workflows for different use cases. So, when SDXL 1.0 and ComfyUI team up, it’s like they open the door to a world of creativity where anyone can make their own awesome pictures without any hard stuff.

In this article, we will delve into how these two amazing things work together to create fantastic images and how to install and use them together. But firstly, we will have a look at their short introduction and how they are powerful together. So without any further wait, let’s get started!

SDXL v1.0 and ComfyUI: Basic Intro

SDXL v1.0

• SDXL 1.0 is the latest version of the Stable Diffusion XL model released by Stability.ai on July 26, 2023.
• It provides improved image generation capabilities, including the ability to generate legible text within images, better representation of human anatomy, and a variety of artistic styles.
• SDXL 1.0 also has a better understanding of shorter prompts, reducing the need for lengthy text to achieve desired results.

ComfyUI

• ComfyUI is a powerful and modular GUI for Stable Diffusion, allowing users to create advanced workflows using a node/graph interface.
• It fully supports the latest Stable Diffusion models, including SDXL 1.0 and lets users chain together different operations like upscaling, inpainting, and model mixing within a single UI.
• ComfyUI’s node-based workflow builder makes it easy to experiment with different generative pipelines for state-of-the-art results.

How ComfyUI is Different Than Other Stable Diffusion UIs

Unlike other Stable Diffusion interfaces, ComfyUI offers a node-based workflow builder, which provides more flexibility and control over the creation and execution of workflows.

ComfyUI also supports a wider range of operations and models, including the latest SDXL 1.0, which can be seamlessly integrated into the workflows.

So, by using ComfyUI with/in SDXL 1.0, users can save and reload their entire layout of nodes in ComfyUI, enabling them to reuse complex workflows across multiple images.

How to Install and Use Stable Diffusion SDXL v1.0 and ComfyUI Together? Complete Step-by-Step Guide

To install and use ComfyUI and SDXL 1.0 together, you will need to follow these steps:

Step 1: Download SDXL v1.0

You will need to download both the base and the refiner models. These models are essential for the two-step process of generating the image and then refining it. The base model generates noisy latents that are processed by the refiner model to denoise and enhance the image’s detail and sharpness. Links are given below

• SDXL v1.0 Base Model: (sd_xl_base_1.0.safetensors)

https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/tree/main

• SDXL v1.0 Refiner Model: (sd_xl_refiner_1.0.safetensors)

https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/tree/main

After opening these links, scroll down until you find the file names I mentioned in brackets. 

Step 2: Download ComfyUI

Now, you need to download ComfyUI. Go to the GitHub page for ComfyUI. Scroll down until you find WINDOWS under installing and these click the “Direct link to download.” 

• GitHub Link for ComfyUI: https://github.com/comfyanonymous/ComfyUI  

ComfyUI is a node-based user interface created to understand how Stable Diffusion works, and it closely follows the underlying mechanics of generation. After the download is finished, extract or unzip the ComfyUI file using WinRAR or 7-Zip.

Note: Direct download only works for NVIDIA GPUs. For AMD (Linux only) or Mac, check the beginner’s guide to ComfyUI.

Step 3: Install Dependencies

After downloading ComfyUI, you need to install its dependencies. Open your terminal in the ComfyUI folder and install the dependencies. If you have another UI installed and working with its own python venv, you can use that venv to run ComfyUI without installing any dependencies.

Step 4: Copy SDXL 0.1 Base and Refiner Models to the ComfyUI file

Copy the sd_xl_base_1.0.safetensors and sd_xl_refiner_1.0.safetensors files to the ComfyUI file which is present with name ComfyUI_windows_portable file. Locate this file, then follow the following path:

• ComfyUI_windows_portable > ComfyUI > models > checkpoints

Now paste these models into the “Checkpoints” subfolder

Step 5: Choose and Run GPU or CPU Version

When you are done copying and pasting the models, then inside the “ComfyUI” folder, locate the batch files for running the software.

Depending on your hardware, choose the appropriate version:

• For GPU, run the “run Nvidia GPU.bat” file.
• For CPU, run the “run Nvidia CPU.bat” file.

After this, a ComfyUI environment will open itself, but it does not contain workflow yet. It is shown in the following figure:

Step 4: Download and Use SDXL Workflow

Instead of creating a workflow from scratch, you can download a workflow optimised for SDXL v1.0. You can use the popular Sytan SDXL workflow or any other existing ComfyUI workflow with SDXL. The workflow is provided as a .json file which is easily loadable into the ComfyUI environment.

• GitHub Sytan SDXL Workflow Link: 

https://github.com/SytanSD/Sytan-SDXL-ComfyUI

Step 5: Load the Workflow

After downloading, you can load the workflow into ComfyUI. This can be done by dragging and dropping the downloaded .json file into the ComfyUI environment or using the “Load button.”

Step 6: Select Models in Load Model Nodes

Once the workflow is loaded, make sure to select your base and refiner models from the dropdowns in the Load Model nodes.

Step 7: Generate Images

After all the above steps are completed, you should be able to generate SDXL images with one click. Open the ComfyUI software. Familiarise yourself with the UI and the available settings.

Set Up Prompts

• In the “Positive Prompt” field, enter a description of the image you want to create.
• Optionally, enter keywords in the “Negative Prompt” field to exclude specific elements from the image.
• Configure other settings like seed numbers, CFG values, and steps for the sampler.

Click Quene Prompt to Start Generating Images

• Once you are done with prompts and settings, click the “Queue Prompt” button to start image generation.
• The software will start processing your prompt and generate the image.
• Depending on the complexity, the process might take a few minutes. Subsequent image generations tend to be faster.

Review and Edit Images

• Once the processing will be done, two images will be generated as output separately in both models at a time which can save you time.
• Review the image and make any necessary adjustments to your prompt or settings for desired results.

Save and Share

• Save the generated images to your “output” folder using the “SAVE” button.
• Share your creations on social media or use them for personal projects.

Remember, ComfyUI is extensible, and many people have written some great custom nodes for it. You can explore and use these custom nodes/extensions as needed.

Final Verdict

Now that you know how to install SDXL v1.0 and use it with ComfyUI, you’re all set to make amazing artwork. With these tools, you can create artworks that are so good you might even want to sell them! SDXL v1.0 makes cool pictures, and ComfyUI helps you use it easily. So, whether you’re just having fun or thinking about selling your art, these tools help you make awesome stuff that people will love. Try it out, and see what incredible art you can make!

Also, Check out some of the amazing artworks below:

If you want to learn more about SDXL, do check our following articles below:

• Sdxl 0.9; Revolutionizing AI Image Generation with Boundless Creativity
• Stability AI Unveils Major Upgrade to Flagship Image Generator

We hope you enjoyed reading this article. For more, you can stay tuned with us because we DigiAlps LTD, are providing you with the latest information through our articles on AI and more trending topics.