adobe-research/custom-diffusion
Enables fast and efficient multi-concept customization of text-to-image diffusion models like Stable Diffusion using a few images.
Core Features
Detailed Introduction
Custom Diffusion offers an innovative method for fine-tuning text-to-image diffusion models, such as Stable Diffusion, to incorporate new concepts from a limited set of images (4-20). It achieves rapid training by selectively adjusting only a subset of model parameters, specifically key and value projection matrices in cross-attention layers, resulting in minimal additional storage per concept. A key advantage is its ability to handle multiple concepts simultaneously, allowing for complex generations like combining new objects with new artistic styles. This research, presented at CVPR 2023, significantly enhances the customization capabilities of generative AI models.