In the ever-evolving landscape of digital animation, the introduction of ControlNet Animation LCM represents a significant advancement. Its key feature is the ability to produce smooth, flicker-free animations. Users can easily upload a source video and then craft a prompt or use Loras to style the animation. Additionally, the IPAdapter function allows users to modify the video’s style, including aspects such as characters, objects, or the background. This comprehensive guide will explore the innovative features and workflows of ControlNet Animation – LCM, enabling both beginners and seasoned animators to harness its full potential.
Table of contents
Step 1 – ControlNet Passes Export
Overview
The journey begins with exporting control passes, forming the foundational layer for your animation project. Here’s a step-by-step breakdown: This file contain all the nodes needed for raw frames for the animation
1.Select your video by copying its path into the Input Video Path node.
2. Define where to save your passes by copying the folder address into the Output path node.
3. For longer videos, a batch workflow prevents memory errors, allowing you to process the video in segments.
4. Customize which passes to extract and edit their properties as needed.
5. For high-resolution videos, downscaling to 1k can optimize rendering. Optionally, you can manually resize your video.
The tool automatically saves source images for reference.
ControlNet Usage Tips
- For Face Closeups – Use Line Art and soft edge.
- For Half Body Shots – Use Line Art with Soft edge or Open Pose.
- For Full body shots – Use Open Pose with line Art or Soft edge
- For Architecture or landscape – Use Depth with Soft edge, line art or Canny.
- For Flat Logos, or Typos – Use Normal with Line Art, Soft edge.
At least 2 Passes are required to use raw animation workflow and get a quality render.
Results
It will export your enabled passes in the specified folder
Step 2: Animation Raw
The Heart of the Workflow
In this stage, you’ll bring your animation to life. Whether aiming for a realistic or abstract style, these steps will guide you:
- Load the Raw animation file
Choose the right control net model checkpoints.
- Define the output image dimensions.
- Use prompts for desired output.
- Test various Loras which suits you best.
- Insert directories for ControlNets and output saving.
- Set up your batch ranges and update skip frames accordingly.
Step 3 – Refiner
Enhancing Your Animation
This workflow adds detail and refines the raw animations.
Duplicate the Comfyui tab and Load AnimateDiff Refiner file
- Specify the file paths for the original images in the Purple Directory Nodes section.
- Define the destination path to save the enhanced images.
- Set the range for processing batches.
- Default for Overlapping Frames is 0 (Use 5 or 10 for a cross-fade effect between batches).
- Default Skip Frame value is 0 (Increase this value after each batch).
- Increment Batch Number Naming by 1 with each batch for better organization.
- Default Refiner Upscale is 1.2 (Optimal range for SD models is between 1.2 and 1.8).
- Refer to the Red note for modifications in refiner settings.
- Input specific prompts to improve accuracy.
- Select the desired model checkpoint for the refinement process.
- Loras function as usual (Check compatibility with animatediff by testing with 10 frames).
- Use AnimateDiff Face Fix (option #4) after rendering all frames if the faces need adjustments.
Step 4 – Face Fix
Fine-Tuning Facial Features
In this process, the focus will be on enhancing only the faces, as the refining tool typically works on the entire image. This specialized workflow ensures that the faces receive detailed improvement since they might not achieve the desired quality when the entire image is refined.
- Enter the location of the purple nodes in the Paths directory for the refined images from item #3.
- Enter the path where these images should be saved.
- Define the range of batches to be used for face fix.
- Default setting for skipping frames is zero (this can be disregarded if all images are included in the batch range).
- Organize the images using batch numbering.
- Set the denoise level for refining; the default and recommended value is 0.4.
- Choose the denoise level for face fix, ideally between 0.25 and 0.45, with 0.4 being the standard.
- Provide specific prompts focusing only on faces to enhance accuracy.
- Select the desired model checkpoint for use.
Final Sequencing
Import all the batches processed during the refining stage and sequence them to create the final video, Overlap them properly and fade out the opacity for a smooth transition.
Conclusion
Finally, ControlNet LCM stands out as a complete and advanced tool in the realm of digital animation, offering a wide array of features designed for creating high-quality animations the process, divided into four distinct steps, allows animators to carefully shape their animations from the initial control pass export to the final sequencing.
In summary, ControlNet Animation – LCM offers a platform for animators seeking to produce smooth, flicker-free animations. Whether for beginners or seasoned professionals, this tool opens up new possibilities in the field of digital animation, allowing for the creation of stunning, lifelike animations with ease and precision.
Also Read: