As text-to-image generation becomes more widespread, having intuitive ways to control outputs is increasingly important. One project tackling this challenge is the GLIGEN GUI – an open-source frontend creating an easy experience for the powerful GLIGEN image generation model. To simplify the usage of GLIGEN and enhance its functionality, the mut-ex has developed this intuitive GUI that leverages the power of ComfyUI as its backend. In this article, we will explore this GUI. So, let’s get started!
Table of Contents
What is GLIGEN?
GLIGEN, available at github.com/gligen/GLIGEN, is a framework that revolutionizes text-to-image models by allowing precise object localization. It allows frozen diffusion models like Stable Diffusion to condition generations on an additional spatial context beyond just text. This includes bounding boxes, key points, and other spatial signals. By providing an intuitive way to specify the exact location of objects within generated images, GLIGEN opens up exciting possibilities in various applications, such as image synthesis and artificial intelligence. According to its CVPR paper, GLIGEN significantly outperforms existing layout-to-image baselines. For more details about GLIGEN, please visit the model’s project page.
Streamlining GLIGEN With ComfyUI as GLIGEN GUI
The intuitive GUI developed by mut-ex for GLIGEN is powered by a popular open-source tool, ComfyUI. Significantly, ComfyUI provides a full-featured graphical interface and APIs for diffusion models like Stable Diffusion. Its modular, graph-based architecture made it an ideal foundation for the GLIGEN GUI.
This GUI simplifies the usage of GLIGEN by providing a user-friendly interface that allows users to interact with the system effortlessly. With the intuitive GUI, users can easily draw grounding boxes on a blank canvas, label them with prompts, and further describe their image using text inputs. The GUI also allows users to incorporate LORAs into their generated images, enhancing the level of detail and complexity.
Example Images Generated by GLIGEN GUI
Getting Started with the GLIGEN GUI
To start harnessing the power of the intuitive GUI for GLIGEN, follow these simple steps:
- Install ComfyUI and run it on your system.
- Download the GLIGEN model file, gligen_sd14_textbox_pruned.safetensors, and place it in the ComfyUI/models/gligen directory.
- Install Flask, a prerequisite for running the GUI, by executing the command pip install flask.
- Clone the repository of the intuitive GUI by running git clone https://github.com/mut-ex/gligen-gui.git in your terminal. Navigate to the gligen-gui directory.
- Launch the GUI by running the command flask –app ‘gligen_gui:create_app(8188)’ run –port 5000 in your terminal. Make sure to replace 8188 with the correct port number if your ComfyUI instance is using a different port.
- Open your browser and access http://127.0.0.1:5000/ to start using the intuitive GUI.
How to Use the GUI for GLIGEN
Using the intuitive GUI for GLIGEN is a breeze. Follow these steps to generate your desired images:
- Make sure you have selected a Stable Diffusion 1.5 checkpoint.
- Envision your image by drawing grounding boxes on the blank canvas using your mouse.
- Label the grounding boxes by entering prompts in the corresponding text inputs on the right side of the GUI.
- Use the text input labeled “POSITIVE” to further describe your image, focusing on style and quality tags.
- If you wish to incorporate LORAs, press the “+” button in the LORA section, select the desired LORA, and adjust its strength. You can add multiple LORAs for more complex and detailed images.
- Finally, press the “Queue Prompt” button to submit your prompt to ComfyUI. Once the image is generated, it will appear on the canvas, bringing your vision to life.
Conclusion
Finally, The intuitive GUI developed by mut-ex for GLIGEN, powered by ComfyUI, opens up a world of possibilities in text-to-image generation, with its user-friendly interface, precise object localization, and seamless integration of LORAs, this GUI empowers users to effortlessly create stunning and customized images. Whether you are a developer, researcher, or enthusiast, this intuitive GUI will undoubtedly enhance your text-to-image generation experience.
| Also Read