Skip to content

fix: apply fp8 weight scale factors when loading scaled safetensors - #1898

Open
fszontagh wants to merge 2 commits into
leejet:masterfrom
fszontagh:fix/fp8-scale-weight
Open

fix: apply fp8 weight scale factors when loading scaled safetensors#1898
fszontagh wants to merge 2 commits into
leejet:masterfrom
fszontagh:fix/fp8-scale-weight

Conversation

@fszontagh

Copy link
Copy Markdown
Contributor

Summary

fp8 safetensors are dequantised to F16, but the companion per-tensor scale was never applied, so every fp8 weight loaded off by its scale factor. The scales are ~1e-3, so weights end up orders of magnitude too large, activations saturate and generation returns a pure white image. Nothing errors: the run reports 1/1 images saved.

Two spellings ship in the wild and both are now handled, for F8 weights only, so int8_tensorwise checkpoints keep their existing .weight_scale path:

  • <module>.scale_weight (ComfyUI)
  • <module>.weight_scale (diffusers)

.scale_input is an activation hint with no meaning once the weight is materialised, so it is dropped alongside the weight scale. scripts/convert_fp8_scale_to_bf16.py only understood .weight_scale and had the same blind spot, so it is updated to match.

Related Issue / Discussion

None.

Additional Information

Affected checkpoints found locally, both spellings:

checkpoint scale key paired
qwen_2.5_vl_7b_fp8_scaled.safetensors .scale_weight 358
umt5_xxl_fp8_e4m3fn_scaled.safetensors (WAN encoder) .scale_weight 168
qwen3vl_8b_fp8_scaled.safetensors .weight_scale 252

Before/after on two pipelines, same command and seed, measured on the decoded output:

run mean std colours
Qwen-Image-Edit, before 255.0 0.00 1
Qwen-Image-Edit, after 86.4 70.08 177315
WAN 2.1 VACE video, before 255.0 0.00 1
WAN 2.1 VACE video, after 117.4 90.16 11982

For Qwen-Image-Edit the fixed output differs from the same prompt run through the GGUF build of the same encoder by mean 1.43, i.e. the scaling is numerically right rather than merely non-blank.

SD 1.5 was re-run as a regression check and is unchanged. The change is inert on checkpoints that carry no scale tensors.

Checklist

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant