VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation

Abstract

Few-step distillation offers efficient generation with diffusion and flow models, but preserving quality with only a few sampling steps remains challenging. Distribution-matching methods via score distillation provide a strong foundation, yet they rely on accurately tracking the student’s changing distribution. At intermediate generation steps in few-step sampling, student-score estimation can be difficult, and commonly used estimators may describe a different distribution from the one used to update the student.

Flow matching allows a more flexible approach to supervision through velocity fields constructed from the student’s sampling process. Building on this perspective, we introduce Velocity Adversarial Distillation (VAD), which uses a learned critic to compare student and teacher fields and turn their disagreement into feedback for learning. VAD unifies score distillation, Fisher-based matching, and moment matching, also explaining existing heuristic MMD updates and delivering new training objectives.

We further develop VA-TDM, which follows trajectory distribution matching: providing supervision for partial denoising along the student trajectory. Beyond this, VA-TDM combines transition-aware online velocity learning and dual-form VAD supervision across intermediate student transitions. Extensive results show that VA-TDM achieves state-of-the-art four-step performance in both text-to-image and text-to-video generation across various backbones and scales effectively to 33B MiniMax-H3. In particular, VA-TDM-H3 notably outperforms the few-step Fast-H3 and is closer to the original many-step H3 according to human preference.

Method

Why velocity instead of score?

Trajectory Distribution Matching (TDM) supervises every intermediate transition of a few-step student, so each step learns a partial denoising task. Score-based updates, however, need the score of the student's perturbed intermediate output. The student's clean prediction and its retained residual are correlated, so perturbing an intermediate output is not the same as adding fresh Gaussian noise to the clean prediction: direct estimation can involve high-variance targets, while clean-target denoising can estimate a different distribution from the one being supervised. Velocity targets can be defined through paths with coupled endpoints, which lets us build student fields that respect the actual sampling process.

Velocity Adversarial Distillation (VAD)

A vector-valued critic \(\mathbf f\) compares the teacher velocity \(\mathbf u_\tau\) with a student velocity \(\bar{\mathbf v}^F_{\theta,\tau}\). The critic maximizes

whose optimum is exactly the velocity residual \(\mathbf f^\star_F=\mathbf u_\tau-\bar{\mathbf v}^F_{\theta,\tau}\). The student opposes the critic, and the game admits a Nash equilibrium with \(\bar{\mathbf v}^F_{\theta^\star,\tau}=\mathbf u_\tau\) and \(\mathbf f^\star\equiv\mathbf 0\).

VA-TDM: distillation along the student trajectory

• Dual-form supervision with a shared backbone. A-form and B-form games are optimized together, \(\mathcal L_{\mathrm{mix}}=\lambda_m\mathcal L^{A}_{\mathrm{mix}}+(1-\lambda_m)\mathcal L^{B}_{\mathrm{mix}}\), with two online-field heads on one backbone. Sharing reduces cost and also improves quality.
• Transition-aware online fields. Samples from different student steps follow different distributions even at the same noise level, so field estimation is conditioned on the transition being supervised.
• Path / flow-product mixing. \(\mathcal L^F_{\mathrm{mix}}=(1-\lambda)\mathcal L^F_{\mathrm{path,sg}}+\lambda\mathcal L^F_{\mathrm{fp,sg}}\) gives an explicit knob for balancing the adversarial game.
• Regularization-corrected critic. The online field is regularized toward the teacher velocity, and the critic is rescaled to \(\mathbf f^F_\psi=(1+\lambda_r)(\mathbf u_\tau-\mathbf h^F_\psi)\) accordingly.
• On-trajectory ODE regression warm-up. Before VAD, the student regresses multi-step teacher ODE targets from its own rollouts, which directly targets the intermediate transitions VA-TDM optimizes.

No extra cost is added at inference: the student still samples in four steps.

BibTeX

@article{luo2026vatdm,
  title   = {VA-TDM: A Unified View for Improving Few-Step Diffusion Models
             by Velocity Adversarial Distillation},
  author  = {Luo, Yihong and Pang, Tianyu and Hu, Tianyang},
  journal = {arXiv preprint},
  year    = {2026}
}