[Community Pipeline] Diffusion Posterior Sampling for General Noisy Inverse Problems

[tongdaxu]

What does this PR do?

This PR is a Community Pipeline with update of examples/comumunity/README.md and a new file examples/community/dps_pipeline.py. This pipeline implements the paper [ICLR 2023] Diffusion Posterior Sampling for General Noisy Inverse Problems:

• The paper deals with zero-shot conditional sampling from unconditional diffuision models, with the corrupted signal and corrunpting operator known. It is kind of magical, as it can do super-resolution, deblurring without any any training or finetuning.
• For example, using google's unconditional ddpm on celebahq dataset, we can reconstruct
  • Reconstructed image:
  • 
• from x4 downsampled image:
  • Downsampled image:
  • 
• or we can also reconstruct
  • Reconstructed image:
  • 
• from a heavily blurred image:
  • Gaussian blurred image:
  • 
• Note that everything above does not require any training or finetuning. Unlike ControlNet that needs extra training, it is an entirely zero-shot approach that works with only pre-trained diffusion model.
• The original implementation of the authors (https://github.com/DPS2022/diffusion-posterior-sampling) are built upon OpenAI's guided diffusion and ADM model, which are not supported in diffuser.
• This community pipeline is build upon Google's DDPM as diffuser has no support on ADM. Thus I can not exactly reproduce the results in paper. I have checked the visual results and it looks pretty good (See updated readme). I am a reviewer for [ICLR] in area of generative model, and think my experience is enough to say that this implemetation looks mostly correct to me.

I have paid some attention to the docstr and typechecking, but I am not exactly sure so please correct me if you spot a mistake. I provide test code in main function of "dps_pipeline.py" and how to use it in updated readme.

Thanks for your time!

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@patrickvonplaten and @yiyixuxu

[patrickvonplaten]

Very cool!