Diffusions in Architecture: Artificial Intelligence and Image Generators
by Matias del Campo, Lev Manovich
Design as a Latent Condition
Daniel Bolojan
Prompt Encoding
Diffusion models are a recent and captivating development in the field of generative AI, offering an iterative approach to generating high‐quality and diverse outputs.
The diffusion process is characterized by a series of discrete steps that progressively refine the output, each step introducing greater levels of detail and complexity. The models involve sampling from learned distributions using text prompts to guide the generation of images.
In text‐to‐image models such as Midjourney, Stable Diffusion, and Disco Diffusion, prompt engineering is employed to fine‐tune the input prompt through a word weighting technique. This approach enables the emphasis or de‐emphasis of certain words, enabling more precise control over the generation process.
However, some design intents and certain design levels cannot be effectively controlled through prompt engineering alone. This limitation arises because complex design ideas cannot always be expressed through a single prompt, and certain design levels cannot be abstracted simultaneously while others retain high resolution.
A more effective approach involves treating each design intent, design concept and level of abstraction separately, fine‐tuning them using prompt engineering, and then combining and layering them in a process that I like to call prompt encoding.
Prompt ...