Presenter

  • Name: Huijie Zhang
  • Affiliation: University of Michigan, Electrical and Computer Engineering
  • Contact: https://www.huijiezh.com

Details

  • Date: Monday, October 9, 2023
  • Time: 12:00 PM
  • Location: EECS, room 2311

Abstract

Recently, diffusion models have emerged as powerful deep generative models, showcasing cutting-edge performance across various applications such as image generation, solving inverse problems, and text-to-image synthesis. These models generate new data (e.g., images) by transforming random noise inputs through a reverse diffusion process. In this work, we uncover a distinct and prevalent phenomenon within diffusion models in contrast to most other generative models, which we refer to as “consistent model reproducibility’’. To elaborate, our extensive experiments have consistently shown that when starting with the same initial noise input and sampling with a deterministic solver, diffusion models tend to produce nearly identical output content. This consistency holds true regardless of the choices of model architectures and training procedures. Additionally, our research has unveiled that this exceptional model reproducibility manifests in two distinct training regimes: (i) “memorization regime,’’ characterized by a significantly overparameterized model which attains reproducibility mainly by memorizing the training data; (ii) “generalization regime,’’ in which the model is trained on an extensive dataset, and its reproducibility emerges with the model’s generalization capabilities. Our analysis provides theoretical justification for the model reproducibility in “memorization regime’’. Moreover, our research reveals that this valuable property generalizes to many variants of diffusion models, including conditional diffusion models, diffusion models for solving inverse problems, and fine-tuned diffusion models. A deeper understanding of this phenomenon has the potential to yield more interpretable and controllable data generative processes based on diffusion models.

Registration

Please fill out this survey to RSVP for pizza!

For any additional questions or inquiries, please contact us at speecs.seminar-requests@umich.edu