Evolutionary Optimization Trumps Adam Optimization on Embedding Space Exploration

arXiv:2511.03913v2 Announce Type: replace-cross Abstract: Deep diffusion models have revolutionized image generation by producing high-quality outputs. However, achieving specific objectives with these models often requires costly adaptations such as fine-tuning, which can be resource-intensive and time-consuming. An alternative approach is inference-time control, which involves optimizing the prompt embeddings to guide the generation process without altering the model weights. We explore prompt