Digital Product Studio

How To Use BG Changer with IP Adaptor + Masks for Consistent Backgrounds in Every Frame!

If you had generated an AI video using ControlNet, you would surely have experienced that your image as a whole, including the background, changes constantly along with the character which is not what we want as in day to day videos background does not change drastically as compared to these AI generated videos so to tackle this issue JerryDavos the creator of ControlNet LCM brings you BG Changer with IP Adaptor + Masks. We can now change background with one Simple image or use a Sequence of Images (Same number as of controlnet images) to change the background. It uses line art controlnet for the background and generates a consistent background for every frame of the animation.

How To Use

In this article we will break down how to use BG Changer with IP Adaptor + Masks in step by step tutorial

Prerequisites

  • If you are new to this workflow, read this tutorial first to get how things work:
  • you will also need to read this tutorial about Automask

Controlnet posses

To maintain a consistent background, we need three control net poses from the source video.

  • SoftedgePlus (ControlNet Strength – 0.8, Start – 0, End – 0.95)
  • OpenPose (ControlNet Strenth – 0.8, Start – 0, End – 0.8)
  • AutoMask (Mask)

these raw images are obtained by following this tutorial

IP and Background

Now select a character image and one image of desired background, We used the following images.

BG Changer with IP Adaptor
using “2_4-BG-Change-with-IP” workflow

IP settings, Low weights (0.4 – 0.6 )  creates more appealing and better results.

BG Changer with IP Adaptor
using “2_4-BG-Change-with-IP” workflow

Single image with Strength (0.8) and End percent (0.65) was used.

Models and Loras

Now choose which model and loras you want to stylize your video. for this tutorial we are going with IMP Model available on Civitai and for loras we have gone with CyberpunkAI and V_O.

BG Changer with IP Adaptor
BG Changer with IP Adaptor

Prompts and Settings

set the prompts and settings of your desired.

BG Changer with IP Adaptor
using “2_4-BG-Change-with-IP” workflow

Refining

In this stage we will going to refine and upscales the raw images using a desired checkpoints. we used Mistoon Anime Model for cartoonish look, This workflow was used for refinement, and then the old v2.1 face fixer was applied to correct the faces.

Finished Video

Conclusion

using the BG Changer with IP Adaptor + Masks in conjunction with ControlNet LCM offers a powerful and efficient method to create AI-generated videos with stable, consistent backgrounds. This approach is particularly beneficial for creating videos where the background remains static while the character or subject moves, closely mimicking real-life scenarios.

Overall, this workflow offers a sophisticated way to control and maintain consistency in the background of AI-generated videos. It opens up new possibilities for creators to produce high-quality, realistic animations and videos that closely resemble real-life filming, with the added benefit of AI-driven creativity and efficiency.

Also Read:

SUBSCRIBE TO OUR NEWSLETTER

Stay updated with the latest news and exclusive offers!


* indicates required
Picture of Faizan Ali Naqvi
Faizan Ali Naqvi

Research is my hobby and I love to learn new skills. I make sure that every piece of content that you read on this blog is easy to understand and fact checked!

Leave a Reply

Your email address will not be published. Required fields are marked *


The reCAPTCHA verification period has expired. Please reload the page.

Don't Miss Out on AI Breakthroughs!

Advanced futuristic humanoid robot

*No spam, no sharing, no selling. Just AI updates.