[tests] cache non lora pipeline outputs.

[sayakpaul]

What does this PR do?

As discussed.

[DN6]

It seems like we mostly use a single scheduler for testing with a few exceptions (CogVideoX)

The LoRA test where scheduler type matters would be LCM LoRAs, and in those tests it looks like LCM scheduler is set directly?

diffusers/tests/lora/test_lora_layers_sdxl.py

Line 227 in 4067d6c

pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)

So why not refactor this entirely to just work with a single scheduler, and we can use the pipeline class name as the cache key?

[sayakpaul]

I would prefer to keep the scope of this PR fixed to caching only.

[sayakpaul]

For SD and SDXL, we need to test for both schedulers which is why we default to:

diffusers/tests/lora/utils.py

Line 112 in 4345907

scheduler_classes = [DDIMScheduler, LCMScheduler]

When the utils.py was added, we didn't have many models with LoRA support, hence this was the default to test with. But that has changed. I don't think we have to test for both schedulers even for SD and SDXL since we have:

diffusers/tests/lora/test_lora_layers_sdxl.py

Line 223 in 4345907

def test_sdxl_lcm_lora(self):

But two schedulers are also used for CogVideoX as you mentioned. Do you propose to have a separate test class for that? Something like CogVideoXDPMSchedulerLoRATests?

I agree to this suggestion, generally. But would really prefer to do that in a separate PR.

[DN6]

For CogVideo, does the choice of scheduler affect the LoRA features (loading, fusing etc) in any way? We can do it in a separate PR that's fine.

[sayakpaul]

It shouldn't. As I said, I generally agree to your suggestion.