Midjourney Meets Stable Diffusion: Revolutionizing Product Photography

Midjourney V6 stands at the forefront of AI-assisted image generation, crafting images brimming with realism that rival traditional photography.

This innovation predominantly addresses the bulk of commercial needs for product imagery within various scenes. A new era dawns where the hassle of capturing specific scene photos becomes a relic of the past.

Yet, the real challenge isn’t just in crafting the scene; it’s in seamlessly integrating the product into it. This is where the prowess of Stable Diffusion becomes invaluable.

Let me guide you through how Midjourney and Stable Diffusion collaborate to forge an exceptionally scenic product image.

1. Scene Creation with Midjourney

In this AI-driven age of photography, a simple product image against a white backdrop is all we require. Consider, for instance, an image of peppermint essential oil.

To accentuate its natural and healthy essence, we must envelop it in an appropriate scene. Utilizing the prompts below, I harnessed Midjourney to generate a fitting backdrop, feeding it the product image for a more authentic replication.

Prompt: Medium: Photo. Subject: A bottle of peppermint essential oil on a moss-covered log, surrounded by peppermint leaves. Emotion: Serene. Lighting: Natural, clear blue sky. Scene: Bubbling brook in the background, lush greenery. Style: Realistic, vibrant colors

Retaining the actual product in the image is crucial, as it provides a placeholder for later enhancements. If you’re curious about crafting such scenes, delve into my earlier Midjourney tutorial focused on product photography prompts:

👉 90+ Midjourney Prompts for Product Photography

2. Erasing the Product

Our next step involves artfully erasing the product in Midjourney using AI-driven techniques, setting the stage to introduce our actual product. This task can be effortlessly accomplished with tools like Photoshop or Adobe Firefly’s Generative Fill. Just select the product in Midjourney, hit “Generate” without any prompts, and watch as Adobe skillfully blends the selected area with its surroundings.

3. Integrating the Product into the Scene

Here, we select the product using Photoshop’s new Object Selection tool or web-based AI tools like Removebg or Clipdrop to eliminate the background.

Perfection isn’t mandatory in this selection, as minor flaws are manageable, given that we will refine them using Stable Diffusion.

In Photoshop, achieving a natural look can be challenging. For instance, when placing the slightly cloudy oil in a natural setting, the transparency of the glass and the play of light and shadow are complex to replicate authentically.

This is precisely where Stable Diffusion’s capabilities shine.

4. Refinement with Stable Diffusion

Choose an appropriate model within Stable Diffusion, preferably an SDXL model for enhanced texture. I opted for a realisticVision model based on SD1.5, filling in the relevant prompts.

The subsequent step involves employing ControlNet’s Depth model. The goal here is to produce a depth map to maintain the product’s original contour, simplifying the restoration of any flawed areas to their initial form.

This depth feature allows Stable Diffusion to blend the product more naturally into the scene, avoiding the rigidity of methods like Canny or Lineart.

I set the Denoising Strength relatively high for a more organic effect.

After generating several iterations, I selected the one that resonated with me.

The outcome? A strikingly realistic bottle, transparent enough to offer glimpses of the background. Although the bottle’s mouth appeared somewhat unnatural, a simple Photoshop mask restored its originality. The final effect is demonstrated below.

Utilizing the SDXL model could have further enhanced the image’s texture. It’s worth noting that the texture from Midjourney V6 might diminish during Stable Diffusion’s repainting, but this can be rectified by overlaying a mask.

Summary

AI is revolutionizing design, unlocking a realm of boundless possibilities. With AI-driven production tools, photographers and post-production artists are liberated from mundane, time-intensive tasks, allowing AI to shoulder this burden.

In the future, designers will pivot more towards harnessing their creativity and ideas, sparing them from the drudgery of repetitive, mechanical operations.

I hope this sharing ignites your imagination and enriches your understanding. Feel free to express your appreciation through likes or shares.