Diffusion-based product imagery

Problem

Generated lifestyle imagery needs to preserve the product itself. A plausible scene is insufficient if the object changes or its lighting is inconsistent with its surroundings.

My contribution

Built an end-to-end image-generation pipeline using Flux and ControlNets. Modified diffusion sampling to improve object reconstruction, followed by intrinsic decomposition for product relighting.

Worked on segmentation with BiRefNet and SAM with YOLO-world, and on classification in low-data settings using CLIP, BLIP2 and Qwen2.5.

Observed outcome

Both the modified diffusion-sampling work and the product-relighting work were adopted into Avataar’s production pipeline. Sampling improved product shape and consistency; relighting improved lighting consistency. These quality improvements are qualitative observations, without a standardized before/after metric.

Pipeline

Combined product-image generation, modified diffusion sampling and intrinsic decomposition in an end-to-end pipeline for product reconstruction and relighting.

Tools

Flux · ControlNet · Diffusion · SAM · CLIP