Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 44 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ Join our [Discord](https://discord.gg/rde6eaE5Ta) to engage in discussions with

## News

- 🔥 [2026/09] ⚡ **SANA-Video 2.0 5B 4-Step Preview** is released! The DMD preview generates 720p videos in four denoising steps and supports 5-second and 8-second outputs. See [Online Demo](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo) | [4-Step Weights](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step) | [Doc](https://nvlabs.github.io/Sana/docs/sana_video2/) | [Project](https://nvlabs.github.io/Sana/Video2/).
- 🔥 [2026/08] 🎬 **SANA-Video 2.0** training, inference, model architecture, and 5B 720p checkpoint are released! The 8-second model supports both text-to-video and text-image-to-video generation, with hybrid linear/softmax attention and Attention Residuals. See [Online Demo](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo) | [Project](https://nvlabs.github.io/Sana/Video2/) | [Doc](https://nvlabs.github.io/Sana/docs/sana_video2/) | [Model Zoo](https://nvlabs.github.io/Sana/docs/model_zoo/#sana-video-20) | [Weights](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p).
- 🔥 [2026/08] ⚡ **Sol Engine: Day-One MiniMax-H3 Acceleration** is available! The 33B omni-modal audio+video DiT runs **3.95×** faster on GB200, reached in 4.5 hours of optimization, and up to **4.52×** on hardware that sits on a desk — 3.92× on DGX Spark, 4.52× on GeForce RTX 5090 — with no distillation, no LoRA, and no calibration pass. See [GB200 Blog](https://nvlabs.github.io/Sana/Sol-Engine/H3/) | [On-Device Blog](https://nvlabs.github.io/Sana/Sol-Engine/H3-OnDevice/).
- 🔥 [2026/07] 🌍 **SANA-Streaming** training is released! Includes bidirectional and distillation training. See [Doc](https://nvlabs.github.io/Sana/docs/sana_streaming/).
Expand Down Expand Up @@ -112,7 +113,7 @@ We introduce **SANA**, a series of efficient diffusion models for high-resolutio
- **[SANA-1.5](https://nvlabs.github.io/Sana/Sana-1.5/)**: Efficient training-time and inference-time compute scaling for better quality.
- **[SANA-Sprint](https://nvlabs.github.io/Sana/Sprint/)**: One/few-step generation via sCM distillation, **0.1s per 1024px image** on H100.
- **[SANA-Video/LongSANA](https://nvlabs.github.io/Sana/Video/)**: Efficient video generation with Block Linear Attention / with [LongLive](https://github.com/NVlabs/LongLive).
- **[SANA-Video 2.0](https://nvlabs.github.io/Sana/Video2/)**: 5B and 14B text-to-video/text-image-to-video architectures with hybrid linear/softmax attention and Attention Residuals. Try the [5B 720p online demo](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo) or download the [checkpoint](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p); the 14B config and checkpoint are not included yet.
- **[SANA-Video 2.0](https://nvlabs.github.io/Sana/Video2/)**: 5B and 14B text-to-video/text-image-to-video architectures with hybrid linear/softmax attention and Attention Residuals. Try the [5B 720p 4-step preview](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo), or download the [50-step](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p) and [4-step preview](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step) checkpoints; the 14B config and checkpoint are not included yet.
- **[Sol-RL](https://nvlabs.github.io/Sana/Sol-RL/)**: NVFP4 Rollout, BF16 Training RL achieves **4.64× faster convergence**.
- **[SANA-WM](https://nvlabs.github.io/Sana/WM/)**: 2.6B parameter controllable world model, generating 720p, 1-minute video worlds with 6-DoF camera control.
- **[SANA-Streaming](https://nvlabs.github.io/Sana/Streaming/)**: 2B real-time streaming video-to-video editing for 720p, minute-scale videos.
Expand Down Expand Up @@ -148,7 +149,7 @@ cd Sana && ./environment_setup.sh sana
This sample was generated from the public 5B checkpoint with seed 4. The result
contains 193 frames at 24 FPS in a 1280 × 736 bucket (8.04 seconds).

Try your own prompt in the [SANA-Video 2.0 5B 720p online demo](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo), or reproduce the verified sample below with the exact release command.
Try your own prompt in the [SANA-Video 2.0 5B 720p 4-step preview](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo), or reproduce the original 50-step sample below with its exact release command.

<p align="center">
<a href="https://huggingface.co/datasets/Efficient-Large-Model/Sana-assets/resolve/main/Video2/assets/release-demo/sana_video2_5b_720p_rooster.mp4">
Expand Down Expand Up @@ -185,6 +186,47 @@ bash inference_video_scripts/inference_sana_video.sh \
--work_dir output/sana_video2_t2v_720p_demo
```

The online preview supports 5-second (81 frames at 16 FPS) and 8-second
(193 frames at 24 FPS) outputs, and defaults to RL LoRA scale 0.7. Run the
T2V-only full DMD checkpoint at its native RL scale 1.0 with the 5-second
profile using:

```bash
bash inference_video_scripts/inference_sana_video.sh \
--np 1 \
--config configs/sana_video2/SanaVideo2_5B_720p.yaml \
--model_path hf://Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step/checkpoints/SANA_Video_2.0_5B_720p_4step.pth \
--txt_file=asset/samples/sana_video2_5b_720p_demo.txt \
--task=t2v \
--model.image_size=480 \
--custom_height_width='[736,1280]' \
--sampling_algo=fastvideo_dmd_4step \
--generator_sigma_profile=sana_shift6_dpm \
--cfg_scale=1.0 \
--flow_shift=1.0 \
--motion_score=20 \
--negative_prompt=None \
--num_frames=81 \
--step=4 \
--fps=16 \
--seed=4 \
--work_dir output/sana_video2_t2v_720p_4step_preview
```

The verified online seed-4 preview below uses the same prompt, temporal profile,
and motion score, with the Space default RL LoRA scale 0.7 (1280 × 736,
81 frames, 16 FPS, 5.06 seconds). The model card records its exact API call.

<p align="center">
<a href="https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step/resolve/main/demo/sana_video2_5b_720p_4step_rooster_seed4.mp4">
<img src="https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step/resolve/main/demo/sana_video2_5b_720p_4step_rooster_seed4_poster.png" width="90%" alt="SANA-Video 2.0 5B four-step preview: a cartoon rooster holding a beer bottle in a floral vintage room"/>
</a>
</p>

<p align="center">
<a href="https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step/resolve/main/demo/sana_video2_5b_720p_4step_rooster_seed4.mp4">▶ Watch or download the verified four-step video</a>
</p>

### Inference with 🧨 diffusers

```python
Expand Down
3 changes: 2 additions & 1 deletion asset/docs/model_zoo.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,12 @@ ______________________________________________________________________

### [SANA-Video 2.0](https://nvlabs.github.io/Sana/Video2/)

[Documentation](https://nvlabs.github.io/Sana/docs/sana_video2/) · [Online demo](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo)
[Documentation](https://nvlabs.github.io/Sana/docs/sana_video2/) · [4-step preview](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo)

| Model | Reso | pth link | diffusers | Precision | Description |
| --- | --- | --- | --- | --- | --- |
| SANA-Video 2.0 5B | 720p | [SANA-Video_2.0_5B_720p](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p) | --- | bf16 inference | 8s T2V/TI2V ReFL model (193 frames at 24 FPS) |
| SANA-Video 2.0 5B 4-step preview | 720p | [SANA-Video_2.0_5B_720p_4step](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step) | --- | bf16 inference | T2V-only DMD preview (81 frames at 16 FPS or 193 frames at 24 FPS) |
| SANA-Video 2.0 14B | 480p | Coming soon | Coming soon | bf16 | 40-layer hybrid-attention model |

______________________________________________________________________
Expand Down
76 changes: 73 additions & 3 deletions asset/docs/sana_video2.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,10 @@ for 720p, 8-second generation. The 14B config and checkpoint are not included
yet.

> **Release status:** Explore the [project page](https://nvlabs.github.io/Sana/Video2/),
> try the [5B 720p online demo](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo),
> or download the [checkpoint](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p).
> try the [4-step 5B 720p preview](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo),
> or download the [50-step](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p)
> and [4-step preview](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step)
> checkpoints.
> The 14B checkpoint is coming soon.

## Architecture
Expand All @@ -36,6 +38,7 @@ Both variants use:
| Model | Resolution | Checkpoint | Precision |
| --- | --- | --- | --- |
| SANA-Video 2.0 5B | 720p, 193 frames at 24 FPS | [SANA-Video_2.0_5B_720p](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p) | BF16 inference |
| SANA-Video 2.0 5B 4-step preview | 720p, 81 frames at 16 FPS or 193 frames at 24 FPS | [SANA-Video_2.0_5B_720p_4step](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step) | BF16 inference |
| SANA-Video 2.0 14B | 480p | Coming soon | BF16 |

The released 5B checkpoint was jointly post-trained for text-to-video (T2V)
Expand All @@ -44,10 +47,16 @@ merged model `state_dict`; optimizer, scheduler, and standalone LoRA state are
not included. The checkpoint preserves the source EMA tensor values and is
cast to BF16 by the inference entry point.

The 4-step research preview is a full-model DMD EMA export initialized from
the merged ReFL step-500 model. It is T2V-only and is not a LoRA adapter. The
release artifact contains model tensors only, without optimizer or scheduler
state.

## Code layout

- Model: `diffusion/model/nets/sana_video2.py`
- Transformer blocks: `diffusion/model/nets/sana_video2_blocks.py`
- Four-step sampler: `diffusion/scheduler/fastvideo_dmd_sampler.py`
- Training: `train_video_scripts/train_video_ivjoint_chunk.py`
- Inference: `inference_video_scripts/inference_sana_video.py`
- Configs: `configs/sana_video2/`
Expand Down Expand Up @@ -85,7 +94,7 @@ with classifier-free guidance 8, flow shift 12, and 50 sampling steps.
The following sample was generated from the public checkpoint with seed 4. The
encoded result is 1280 × 736, 193 frames, 24 FPS, and 8.04 seconds long.

[Try SANA-Video 2.0 5B 720p online](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo), or reproduce this sample with the exact command below.
[Try the SANA-Video 2.0 5B 720p 4-step preview online](https://huggingface.co/spaces/Efficient-Large-Model/sana-video2-5b-720p-demo), or reproduce the original 50-step sample below with its exact command.

<video controls muted loop playsinline poster="https://huggingface.co/datasets/Efficient-Large-Model/Sana-assets/resolve/main/Video2/assets/release-demo/sana_video2_5b_720p_rooster_poster.png" style="width: 100%;">
<source src="https://huggingface.co/datasets/Efficient-Large-Model/Sana-assets/resolve/main/Video2/assets/release-demo/sana_video2_5b_720p_rooster.mp4" type="video/mp4">
Expand Down Expand Up @@ -121,6 +130,67 @@ bash inference_video_scripts/inference_sana_video.sh \
--work_dir output/sana_video2_t2v_720p_demo
```

### Four-step text-to-video preview

The DMD preview uses four fixed stochastic stages, CFG 1, BF16 latent noise,
and the `sana_shift6_dpm` sigma profile. It supports a 5-second profile
(81 frames at 16 FPS) and an 8-second profile (193 frames at 24 FPS), and does
not support first-frame conditioning. The online preview defaults to the
5-second profile and RL LoRA scale 0.7. The `flow_shift` argument is retained
for CLI consistency but is not applied by this sampler. Model construction
stays at the source tower's 480 setting; `custom_height_width` controls the
actual 736×1280 latent and MP4 dimensions.

```bash
bash inference_video_scripts/inference_sana_video.sh \
--np 1 \
--config configs/sana_video2/SanaVideo2_5B_720p.yaml \
--model_path hf://Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step/checkpoints/SANA_Video_2.0_5B_720p_4step.pth \
--txt_file=asset/samples/sana_video2_5b_720p_demo.txt \
--task=t2v \
--model.image_size=480 \
--custom_height_width='[736,1280]' \
--sampling_algo=fastvideo_dmd_4step \
--generator_sigma_profile=sana_shift6_dpm \
--cfg_scale=1.0 \
--flow_shift=1.0 \
--motion_score=20 \
--negative_prompt=None \
--num_frames=81 \
--step=4 \
--fps=16 \
--seed=4 \
--work_dir output/sana_video2_t2v_720p_4step_preview
```

The local command above runs the full checkpoint at its native RL scale 1.0.
The verified online sample below uses the same prompt, schedule, duration,
motion score, and seed, with the Space default RL LoRA scale 0.7 (1280 × 736,
81 frames, 16 FPS, 5.06 seconds). Reproduce those exact online settings with:

```python
from pathlib import Path

from gradio_client import Client

prompt = Path("asset/samples/sana_video2_5b_720p_demo.txt").read_text().strip()
video_path, seed = Client("Efficient-Large-Model/sana-video2-5b-720p-demo").predict(
prompt=prompt,
video_duration="5 seconds",
rl_lora_scale=0.7,
motion_score=20,
seed=4,
randomize_seed=False,
api_name="/generate",
)
```

<video controls muted loop playsinline poster="https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step/resolve/main/demo/sana_video2_5b_720p_4step_rooster_seed4_poster.png" style="width: 100%;">
<source src="https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step/resolve/main/demo/sana_video2_5b_720p_4step_rooster_seed4.mp4" type="video/mp4">
</video>

[Open or download the verified four-step video](https://huggingface.co/Efficient-Large-Model/SANA-Video_2.0_5B_720p_4step/resolve/main/demo/sana_video2_5b_720p_4step_rooster_seed4.mp4)

### Text-image-to-video

Each line in `asset/samples/sample_i2v.txt` contains a prompt and an input-image
Expand Down
1 change: 1 addition & 0 deletions diffusion/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
__version__ = "0.2.1.dev0"

from diffusion.scheduler.dpm_solver import DPMS
from diffusion.scheduler.fastvideo_dmd_sampler import FastVideoDMD4Step
from diffusion.scheduler.flow_euler_sampler import ChunkFlowEuler, FlowEuler, LTXFlowEuler
from diffusion.scheduler.iddpm import Scheduler
from diffusion.scheduler.longlive_flow_euler_sampler import LongLiveFlowEuler
Expand Down
134 changes: 134 additions & 0 deletions diffusion/scheduler/fastvideo_dmd_sampler.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# Copyright 2026 NVIDIA CORPORATION & AFFILIATES
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0
"""Four-stage stochastic sampler used by the SANA-Video 2.0 DMD preview.

The schedule and transition below intentionally mirror the deployment-aligned
rollout used during DMD training. It is not a shortened DPM-Solver schedule:
each stage predicts ``x0`` and, except for the terminal stage, re-noises it at
the next fixed sigma with a fresh draw from the caller's generator.
"""

from __future__ import annotations

from typing import Any

import torch

WAN_LITERAL_GENERATOR_SIGMA_PROFILE = "wan_literal"
SANA_SHIFT6_DPM_GENERATOR_SIGMA_PROFILE = "sana_shift6_dpm"

_GENERATOR_SIGMA_PROFILES: dict[str, tuple[float, ...]] = {
WAN_LITERAL_GENERATOR_SIGMA_PROFILE: (1.0, 0.75, 0.5, 0.25),
SANA_SHIFT6_DPM_GENERATOR_SIGMA_PROFILE: (
0.9998332262,
0.9471688271,
0.8568977118,
0.6663702130,
),
}


def resolve_fastvideo_dmd_generator_sigmas(profile: str | None) -> tuple[float, ...]:
"""Resolve a named, validated four-stage DMD generator schedule."""

if profile not in _GENERATOR_SIGMA_PROFILES:
choices = ", ".join(sorted(_GENERATOR_SIGMA_PROFILES))
raise ValueError(f"Unknown FastVideo DMD generator sigma profile {profile!r}; " f"choose one of: {choices}")
return _GENERATOR_SIGMA_PROFILES[profile]


class FastVideoDMD4Step:
"""Deployment sampler for the four-step SANA-Video DMD student."""

def __init__(
self,
model: torch.nn.Module,
condition: torch.Tensor,
*,
cfg_scale: float = 1.0,
generator_sigma_profile: str = SANA_SHIFT6_DPM_GENERATOR_SIGMA_PROFILE,
model_kwargs: dict[str, Any] | None = None,
) -> None:
if float(cfg_scale) != 1.0:
raise ValueError("FastVideoDMD4Step was trained and validated with cfg_scale=1.0")
self.model = model
self.condition = condition
self.cfg_scale = float(cfg_scale)
self.generator_sigma_profile = generator_sigma_profile
self.sigmas = resolve_fastvideo_dmd_generator_sigmas(generator_sigma_profile)
self.model_kwargs = dict(model_kwargs or {})
self.last_trace: dict[str, Any] | None = None

def sample(
self,
latent: torch.Tensor,
*,
steps: int = 4,
generator: torch.Generator | None = None,
) -> torch.Tensor:
"""Run exactly four model evaluations and three stochastic transitions."""

if int(steps) != 4:
raise ValueError(f"FastVideoDMD4Step requires exactly 4 steps, received {steps}")
if latent.ndim < 1 or latent.shape[0] < 1:
raise ValueError(f"latent must include a positive batch dimension, got {latent.shape}")

current = latent
physical_sigmas = (*self.sigmas, 0.0)
embedding_timesteps: list[int] = []
transition_noise_draws = 0

for sigma, next_sigma in zip(physical_sigmas[:-1], physical_sigmas[1:], strict=True):
model_timestep = torch.full(
(current.shape[0],),
1000.0 * sigma,
device=current.device,
dtype=torch.float32,
)
embedding_timesteps.append(int(1000.0 * sigma))
velocity = self.model(current, model_timestep, self.condition, **self.model_kwargs)
if not isinstance(velocity, torch.Tensor):
raise TypeError(
"FastVideoDMD4Step expects the model to return a tensor, " f"received {type(velocity).__name__}"
)
if velocity.shape != current.shape:
raise RuntimeError(
"SANA velocity output shape mismatch: "
f"latent={tuple(current.shape)} prediction={tuple(velocity.shape)}"
)

pred_x0 = current - sigma * velocity
if next_sigma > 0.0:
noise = torch.randn(
current.shape,
device=current.device,
dtype=current.dtype,
generator=generator,
)
current = (1.0 - next_sigma) * pred_x0 + next_sigma * noise
transition_noise_draws += 1
else:
current = pred_x0

self.last_trace = {
"generator_sigma_profile": self.generator_sigma_profile,
"physical_sigmas": list(physical_sigmas),
"embedding_timesteps": embedding_timesteps,
"transition_noise_draws": transition_noise_draws,
"flow_shift_applied": False,
"latent_dtype": str(latent.dtype).removeprefix("torch."),
}
return current
Loading
Loading