Skip to content

Qwen Image and Chroma pipeline breaks using schedulers that enable flow matching by parameter. #12108

@Vargol

Description

@Vargol

Describe the bug

Several Schedulers support flow matching by using the prediction_type='flow_prediction" e.g.

pipe.scheduler = UniPCMultistepScheduler(prediction_type="flow_prediction", flow_shift=3.16, timestep_spacing='trailing', use_flow_sigmas=True)

However Chroma and Qwen Image will not work with these schedulers failing with the error

ValueError: The current scheduler class <class 'diffusers.schedulers.scheduling_unipc_multistep.UniPCMultistepScheduler'>'s `set_timesteps` does not support custom sigmas schedules. Please check whether you are using the correct scheduler.

Can we have this fixed by either changing the schedulers to have the missing attributes and use them, or by rethinking the way these pipelines handle the timesteps .

Reproduction

import torch
from diffusers import QwenImagePipeline, UniPCMultistepScheduler



pipe = QwenImagePipeline.from_pretrained("Qwen/Qwen-Image",
                                         torch_dtype=torch.bfloat16)
#pipe.scheduler = FlowMatchEulerDiscreteScheduler(shift=3.16, use_beta_sigmas=True)
pipe.scheduler = UniPCMultistepScheduler(prediction_type="flow_prediction", flow_shift=3.16, timestep_spacing='trailing', use_flow_sigmas=True)
pipe.to("mps")
pipe("a nice picture of an rainbow")

Logs

File "/Volumes/SSD2TB/AI/Diffusers/qwenimagelowmem.py", line 84, in <module>
    image = pipe(prompt_embeds=prompt_embeds, prompt_embeds_mask=prompt_embeds_mask, 
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/SSD2TB/AI/Diffusers/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/SSD2TB/AI/Diffusers/lib/python3.11/site-packages/diffusers/pipelines/qwenimage/pipeline_qwenimage.py", line 619, in __call__
    timesteps, num_inference_steps = retrieve_timesteps(
                                     ^^^^^^^^^^^^^^^^^^^
  File "/Volumes/SSD2TB/AI/Diffusers/lib/python3.11/site-packages/diffusers/pipelines/qwenimage/pipeline_qwenimage.py", line 119, in retrieve_timesteps
    raise ValueError(
ValueError: The current scheduler class <class 'diffusers.schedulers.scheduling_unipc_multistep.UniPCMultistepScheduler'>'s `set_timesteps` does not support custom sigmas schedules. Please check whether you are using the correct scheduler.

System Info

  • 🤗 Diffusers version: 0.35.0.dev0
  • Platform: macOS-15.5-arm64-arm-64bit
  • Running on Google Colab?: No
  • Python version: 3.11.13
  • PyTorch version (GPU?): 2.6.0 (False)
  • Flax version (CPU?/GPU?/TPU?): not installed (NA)
  • Jax version: not installed
  • JaxLib version: not installed
  • Huggingface_hub version: 0.34.3
  • Transformers version: 4.52.4
  • Accelerate version: 1.7.0
  • PEFT version: 0.17.0
  • Bitsandbytes version: not installed
  • Safetensors version: 0.5.3
  • xFormers version: not installed
  • Accelerator: Apple M3
  • Using GPU in script?: Yes
  • Using distributed or parallel set-up in script?: No

Who can help?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions