Generating Deepfakes with Stable Diffusion, ControlNet, and LoRA

We propose a different approach to generate deepfake videos based on Stable Diffusion, ControlNet, and Low-Rank Adaptation (LoRA). Stable Diffusion offers us greater control and fine-tuning options in the generation process. Compared to GANs, the proposed technique enables quick and easy modification of the obtained video by using a text prompt, adding or removing details, and altering the style and context of the deepfake. We describe the approach used and the generation pipeline, and then we show the application interface developed for the generation. Finally, we compare the quality of our deepfake generation framework with two other related approaches using two different tools that detect video/image manipulations.