Hello everyone,

Next Monday (2023/10/09) at noon, Huijie Zhang will be presenting in EECS room 2311. Please fill out the food form before attending, so we can buy enough pizza for everyone.

If you have research to share, please volunteer to present using this link. Currently, there is no one scheduled for 2023/11/06 (two seminars from now). As a token of gratitude, presenters get to choose a customized meal from a selection of local restaurants, as listed here.

All seminar info is available on the SPEECS website, and a Google calendar link with dates/times/presenters is can be found here. If you have any questions, you can contact Zongyu Li or me directly, or email speecs.seminar-requests@umich.edu. Suggestions are always welcome :)

Speaker: Huijie Zhang

Topic: The Emergence of Reproducibility and Consistency in Diffusion Models

Abstract: Recently, diffusion models have emerged as powerful deep generative models, showcasing cutting-edge performance across various applications such as image generation, solving inverse problems, and text-to-image synthesis. These models generate new data (e.g., images) by transforming random noise inputs through a reverse diffusion process. In this work, we uncover a distinct and prevalent phenomenon within diffusion models in contrast to most other generative models, which we refer to as “consistent model reproducibility’’. To elaborate, our extensive experiments have consistently shown that when starting with the same initial noise input and sampling with a deterministic solver, diffusion models tend to produce nearly identical output content. This consistency holds true regardless of the choices of model architectures and training procedures. Additionally, our research has unveiled that this exceptional model reproducibility manifests in two distinct training regimes: (i) “memorization regime,’’ characterized by a significantly overparameterized model which attains reproducibility mainly by memorizing the training data; (ii) “generalization regime,’’ in which the model is trained on an extensive dataset, and its reproducibility emerges with the model’s generalization capabilities. Our analysis provides theoretical justification for the model reproducibility in “memorization regime’’. Moreover, our research reveals that this valuable property generalizes to many variants of diffusion models, including conditional diffusion models, diffusion models for solving inverse problems, and fine-tuned diffusion models. A deeper understanding of this phenomenon has the potential to yield more interpretable and controllable data generative processes based on diffusion models.

Supplementary link: None

Mirror: http://websites.umich.edu/~speecsseminar/presentations/20231009/

Thanks,

Matt Raymond