feat(dspark): add Ascend NPU support for Qwen3.5-4B DSpark training

[curnane-lab]

Summary

This PR adds Ascend NPU training support for DSpark on Qwen3.5-4B.

Note on scope: The branch currently contains two commits. The first commit (a2f18ea, "feat: DSpark trainer") is borrowed from the preceding DSpark trainer PR and provides the base DSpark implementation. This PR's own incremental change is the second commit (5776d51), which adds the NPU example script and the flex_attention -> SDPA fallback in the trainer.

What is added (incremental)

1. NPU training launcher

• examples/run_qwen3.5_4b_dspark_online_npu.sh
  • Sets ASCEND_RT_VISIBLE_DEVICES and PYTORCH_NPU_ALLOC_CONF.
  • Uses --attention-backend sdpa and --target-model-backend hf (HF backend always surfaces last_hidden_states, which DSpark's L1 / confidence losses require).
  • Uses HCCL via torchrun --standalone.

2. Trainer NPU fallback

• scripts/train_dspark.py
  • Auto-detects Ascend NPU and falls back from flex_attention to sdpa when the default backend would fail on NPU.

DSpark background (for context)

DSpark = SpecForge's DFlash block-diffusion drafter + EAGLE-style Markov & confidence heads, trained with:

• Cross-entropy against ground-truth next tokens.
• L1 distribution distillation using the target model's final hidden state.
• Confidence-head BCE against the empirical per-token accept rate.

The base trainer implementation is in the preceding commit (a2f18ea). This PR only layers the NPU enablement on top.

Usage

export TARGET_MODEL_PATH=/path/to/Qwen3.5-4B
export TRAIN_DATA_PATH=/path/to/train.jsonl
bash examples/run_qwen3.5_4b_dspark_online_npu.sh 0,1,2,3,4,5,6,7

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://sgl-fru7574.slack.com/archives/C09784E3EN6 to discuss your PR.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!