Among the myriad wonders of Stable Diffusion, one of the most popular and captivating applications lies in the realm of generating lifelike people. These AI-crafted portraits possess an uncanny resemblance to photographs, blurring the lines between reality and imagination.
In this post, we embark on a fascinating journey to explore the mechanics behind generating photo-style portrait images. Let’s delve into the secrets of prompts, models, and upscalers that bring these realistic individuals to life with artistic finesse.
Preparing Your Stable Diffusion
Before we embark on this exciting journey of generating lifelike photo-style portraits, let’s make sure you have the right tools at your disposal.
For this part of the tutorial, we’ll be working with the remarkable “RealisticVision” checkpoint model, known for its prowess in creating realistic human images that mirror photographs with stunning accuracy.
To begin, ensure you have a locally installed Stable Diffusion, as it will be the gateway to unleashing the full potential of RealisticVision. If you haven’t already installed Stable Diffusion, fear not! Our friends at 88stacks offer a web-hosted version that provides the same remarkable capabilities without any installation hassles.
The Parameters We’ll Be Using
To bring forth the enchanting world of lifelike photo-style portraits, we must acquaint ourselves with the essential Stable Diffusion parameters that work in symphony to create artistic wonders. With precision and care, let’s configure the following parameters for our exquisite journey:
- Sampling Method: DPM++ SDE Karras
- Sampling Steps: 45
- CFG Scale: 6.5
- Size: 512×768
Now that we have laid the foundation, let’s delve into the artistry of prompts. A well-crafted prompt is the key to unlocking the essence of our photo-style portrait. We will set the stage with a thoughtful prompt and a negative prompt to guide Stable Diffusion in producing captivating imagery.
Prompt:
“photo of woman”
Negative Prompt:
“disfigured, ugly, bad, immature, cartoon, anime, 3d, painting, b&w, nudity, nsfw”
Note: The negative prompts used are for ensuring the photograph is free from bad anatomy, artifacts, and unnatural effects. They are in no way intended to cause harm or enforce an unhealthy standard of conventional beauty.
This is the first image we’ll be generating. While it looks realistic enough, we are not satisfied with how plain looking it is.
If you are, congratulations. You have created your first realistic looking AI image!
Setting Up The Scene
A truly captivating photo-style portrait not only relies on the subject but also on the intricate details that breathe life into the overall composition. As we venture into the art of crafting lifelike images, let’s pay attention to every element that sets the stage for a mesmerizing visual tale.
Lighting
The play of light and shadow can work wonders in infusing emotion and depth into our photo-style portraits. Whether it’s soft, diffused lighting for a dreamy aura or dramatic lighting for a captivating effect, the choice of lighting can be the masterstroke that elevates our creation
Prompt: photo of a woman, soft natural light, sunlight
You can use a variety of different lighting prompts, such as:
- Hard key lighting
- Rembrandt lighting
- Rim lighting
- Soft natural lighting
Remember, lighting tells a story. Whether its strong compositions of dark against light, it pays well to be selective about the lighting you are using.
Camera Used
Just like a painter chooses the right brush, the choice of the virtual camera in Stable Diffusion matters. Experiment with different camera settings to find the perfect angle and perspective that accentuates the subject’s allure.
Prompt: photo of a woman, soft natural light, sunlight, (vintage photography:1.2)
You can use a variety of different camera style prompts, such as:
- DSLR
- Film Grain
- Polaroid
- Vintage Photography
- Analogue Camera Photography
You can use prompt weights (pressing CTRL+ {Arrow Key UP/DOWN}) on the selected prompt to see more of its effects used on the generated image
Coloring
Colors evoke emotions and add personality to our portraits. Delve into a palette that harmonizes with the mood you wish to convey. From vibrant and bold to muted and pastel, the right color selection can speak volumes. Coloring can also affect the clothing and the hair being generated.
Prompt: photo of a woman, soft natural light, sunlight, (vintage photography:1.2), psychedelic coloring
You can use a variety of different coloring prompts, such as:
- Pastel Colors
- Muted Colors
- Psychedelic Coloring
- Gothic Colors
- Natural Earth Colors
You can also use thematic colors from different directors that use distinct colors, such as Wes Anderson, Alexandro Jodorowsky, and Akira Kurosawa. Experiment with different styles of coloring offered by masters in media and see which fits into your photo
Clothing
Attire plays a crucial role in defining the character of our subjects. Dress them in outfits that resonate with their personalities or align with the desired narrative. Whether it’s casual, formal, or inspired by a specific style like Balenciaga’s iconic fashion, clothing choices can shape the entire story.
Prompt: photo of a woman, soft natural light, sunlight, (vintage photography:1.2), (psychedelic coloring:1.2), Boho-chic maxi dress with flowing fabric, embroidered detailing, and fringe accents
You can use a variety of different clothing style prompts, such as:
- Balenciaga Collection
- Cyberpunk techwear streetwear look and clothes
- Sporty tracksuit with black stripes and pastel accents
- Black-tie tuxedo with a tailored fit and velvet lapels
- Asian traditional clothing
At this point you may see some weird, unnatural fragments or artifacts in the images. This is natural, as Stable Diffusion takes up more prompts to use. You can either increase the sampling steps, switch model checkpoints, use LoRAs, or even fix them in Inpainting.
Environment
The backdrop and setting breathe life into our subjects. Transport them to a park, a bustling cityscape, or even an ethereal fantasy realm. The environment sets the stage and complements the subject’s presence, allowing their story to unfold.
Prompt: photo of a woman, soft natural light, sunlight, (vintage photography:1.2), (psychedelic coloring:1.2), Boho-chic maxi dress with flowing fabric, embroidered detailing, and fringe accents, in the desert
We honestly don’t know what we were trying to generate, but it looks like our lovely friend here is ready for the next Burning Man somewhere in the Mojave desert.
Additionally, you can use our prompt tool here to help you brainstorm different prompts for creating scenes for realistic photos.
With these essential elements harmoniously in place, our photo-style portraits transform into living art, capturing hearts and captivating minds. So, let’s set the scene with a touch of magic and let the enchanting ambiance lead us to AI-crafted masterpieces beyond imagination.
Touching Up Defects Using Inpainting
At first glance, you might think inpainting and post-processing are the most daunting steps in our journey. However, fret not, for they are surprisingly straightforward, especially if you’ve installed Stable Diffusion locally and come this far in your creative expedition.
The beauty lies in the familiarity of these tasks for those well-versed in image generators and post-processing tools like Adobe Photoshop. Just like a seasoned artist wields their favorite brush, you too can effortlessly navigate inpainting and post-processing with a touch of finesse.
Let’s dive into an example to understand how to enhance our artwork. Suppose we have an image that appears visually appealing except for a slight deformation in the face.
This was the original image before it was fixed using Inpainting. It had a fractured and anatomically incorrect face. Let’s try to fix that by loading it up on Inpainting.
In the Stable Diffusion UI, click img2img in between txt2img and Extras in the worktab.
Select Inpainting in the subtab below and select the photo you want to upload and edit.
Next, we’ll be painting what is called a “mask” on the image. The mask tells the AI which part of the image they will be regenerating. So for this image, we’ll paint over her face.
Before typing “woman face” in the prompt and clicking generate, it’s important that you change the Inpainting parameters as well.
There’s a lot to learn about how these parameters change the image, but the most important settings you should take note of are the Inpaint area: Only masked and the Inpaint masked settings.
Viola! You should have a new image with the problematic area fixed. You can also use this method with other visual problems like fingers, artifacts, and out-of-place objects. Experiment with the parameters and see how they can change the image as a whole.
Mastering the Craft: Additional Tips and Tricks
As we near the pinnacle of photo-style portrait generation, let’s equip ourselves with a set of powerful tips and tricks to unleash the full potential of our artistic endeavors.
1. ControlNet for Pose Control: To breathe life into our subjects and achieve precise control over their poses, ControlNet is the answer. This remarkable tool empowers us to shape the posture and positioning of our characters, allowing for a captivating interplay of emotions and dynamics in our portraits. With ControlNet, the art of posing becomes an art form in itself, imbuing our creations with a touch of realism that mesmerizes the eye.
2. LoRA for Specialized Designs: For those seeking to infuse their portraits with specialized and specific designs, LoRA comes to the forefront. This exceptional feature supplements model modifiers, granting us the ability to tailor our AI-generated images to meet even the most intricate creative visions. With LoRA in our repertoire, imagination knows no bounds, and our photo-style portraits transcend into realms of pure artistic ingenuity.
3. Different Checkpoint Models, Endless Possibilities: As we traverse the vast expanse of Stable Diffusion, a myriad of checkpoint models beckon us with their unique strengths and creative potential. Don’t be afraid to explore different models, for each one offers a distinct flavor to our artwork. From RealisticVision for lifelike human images to DreamShaper for artistic flair, the diversity of checkpoint models is our palette of possibilities.
4. Embrace Iteration and Experimentation: In the creative realm, iteration is the key to unlocking greatness. Don’t shy away from experimenting with different prompts, parameters, and styles. Each iteration holds the promise of newfound discoveries and unforeseen brilliance. Embrace the process of trial and error, for it is the bridge that connects us to artistic brilliance.
5. Merge Human Expertise and AI Precision: Remember, while AI provides us with a powerful toolkit, it’s the human touch and creative finesse that breathe life into our artwork. Merge the precision of AI with the passion of the human heart, and watch as your photo-style portraits transcend into a realm of soulful expression.
Unraveling the Enchantment of AI-Crafted Portraits
Witnessing the magic of AI technology come alive, we’ve ventured deep into the mechanics of generating lifelike human images that transcend mere pixels on a screen.
From prompts that weave tales of emotion to models that breathe life into our subjects, every stroke of creativity has led us to masterpieces that blur the boundaries of reality and imagination. So, as we stand at the intersection of technology and artistry, let us remember that this journey is only the beginning. Stable Diffusion is an ever-evolving canvas, inviting us to embrace experimentation and forge new paths of expression