---
abstract: |
Despite the rapid progress of video generation models, the role of data in influencing motion is poorly understood. We present Motive (MOTIon attribution for Video gEneration), a motion-centric, gradient-based data attribution framework that scales to modern, large, high-quality video datasets and models. We use this to study which fine-tuning clips improve or degrade temporal dynamics. Motive isolates temporal dynamics from static appearance via motion-weighted loss masks, yielding efficient and scalable motion-specific influence computation. On text-to-video models, Motive identifies clips that strongly affect motion and guides data curation that improves temporal consistency and physical plausibility. With Motive-selected high-influence data, the method improves both motion smoothness and dynamic degree on VBench, achieving a 74.1% human-preference win rate compared with the pretrained base model. To the authors' knowledge, this is the first framework to attribute motion rather than visual appearance in video generative models and to use it to curate fine-tuning data.
author:
- Xindi Wu
- Despoina Paschalidou
- Jun Gao
- Antonio Torralba
- Laura Leal-Taixé
- Olga Russakovsky
- Sanja Fidler
- Jonathan Lorraine
title: Motion Attribution for Video Generation
---
# Abstract
Despite the rapid progress of video generation models, the role of data in influencing motion is poorly understood. We present Motive (MOTIon attribution for Video gEneration), a motion-centric, gradient-based data attribution framework that scales to modern, large, high-quality video datasets and models. We use this to study which fine-tuning clips improve or degrade temporal dynamics. Motive isolates temporal dynamics from static appearance via motion-weighted loss masks, yielding efficient and scalable motion-specific influence computation. On text-to-video models, Motive identifies clips that strongly affect motion and guides data curation that improves temporal consistency and physical plausibility. With Motive-selected high-influence data, the method improves both motion smoothness and dynamic degree on VBench, achieving a 74.1% human-preference win rate compared with the pretrained base model. To the authors' knowledge, this is the first framework to attribute motion rather than visual appearance in video generative models and to use it to curate fine-tuning data.
# Introduction
Motion is the defining element of videos. Unlike image generation, which produces a single frame, video generative models capture how objects move, interact, and obey physical constraints [@wiedemer2025video; @kang2024far]. Yet even with the rapid progress of video generation, a fundamental question remains:\
> *Which training data influence the motion in generated videos?*
**Why it matters.** Diffusion models are data-driven, and their progress has tracked the scaling of data and compute [@saharia2022photorealistic; @nichol2021improved; @ho2022video; @peebles2023scalable]. Prior work [@blattmann2023stable; @kaplan2020scaling; @ravishankar2025scaling] shows that training data shapes key generative properties, including visual quality [@rombach2022high], semantic fidelity [@namekata2024emerdiff], and compositionality [@wu2025compact; @favero2025compositional]. Motion is no exception. *Motion* refers to temporal dynamics captured by optical flow, including trajectories, deformations, camera movement, and interactions. If generated motion reflects the data distribution that shaped the model, then attributing motion to influential training clips provides a direct lens on why a model moves the way it does and enables targeted data selection for desired dynamics. High-quality data often matters most in fine-tuning, where large pretraining corpora are inaccessible and carefully selected clips can have an outsized impact. Motion-specific attribution is therefore especially valuable in the fine-tuning regime, where the goal is to identify which clips most influence temporal coherence and physical plausibility.
**Why existing methods are limited for motion.** Prior diffusion data attribution focuses on images and explains static content. Extending these methods to videos naïvely collapses motion into appearance, missing the temporal structure that distinguishes videos from images. Three challenges drive this gap: (i) localizing motion so attribution focuses on dynamic regions rather than static backgrounds, (ii) scaling to sequences since gradients must integrate across time, and (iii) capturing temporal relations like velocity, acceleration, and trajectory coherence that single-frame attribution cannot measure. Addressing motion attribution requires methods that explicitly model temporal structure, rather than treating time as an additional spatial axis.
**Our method.** We introduce Motive, a motion attribution framework for video generation models that isolates motion-specific influence. Motive computes gradients with motion-aware masking. As a result, the attribution signal emphasizes dynamic regions rather than static appearance. Efficient approximations make the method practical for large, high-quality datasets and video generative models. Our scores trace generated motion back to training clips, enabling targeted curation and improving motion quality when used to guide fine-tuning. Our key contributions are:
1. Proposing a scalable gradient-based attribution approach for video generation models that is computationally efficient, even at the scale of modern, high-quality data and large generative models (§`\ref{sec:method:1}`{=latex}).
2. Addressing a video-specific bias by correcting frame-length effects in gradient magnitudes, ensuring fair attribution across clips of different durations (§`\ref{sec:method:video-bias}`{=latex}).
3. Introducing an attribution that emphasizes temporal dynamics to trace which training clips most strongly influence motion quality (§`\ref{sec:method:2}`{=latex}).
4. Showing that we improve motion smoothness and dynamic degree on VBench and in human evaluation (§`\ref{sec:experiment}`{=latex}), matching, or surpassing, full-dataset fine-tuning performance with only $10\%$ of the data, and outperforming motion-unaware attribution baselines (Tables `\ref{tab:vbench-wan}`{=latex} and `\ref{tab:human_eval_extended}`{=latex}).
# Background {#sec:background}
Notation and extended related work are in App. §`\ref{app:notation}`{=latex} and §`\ref{sec:app:related}`{=latex}.
## Video Generation with Diffusion and Flow-Matching Models {#subsec:diffusion}
**Diffusion and flow matching in latent space.** Let $p_{\boldsymbol{\theta}}(\mathbf{v} \mid \mathbf{c})$ be a conditional generator with parameters $\boldsymbol{\theta}$, where $\mathbf{v} \in \mathbb{R}^{F \times H \times W \times 3}$ is a clip of height $H$, width $W$, and $F$ frames, and $\mathbf{c}$ denotes conditioning such as text or other multimodal metadata (e.g., fps, depth, pose). We operate in VAE latents: $\mathbf{h}=E(\mathbf{v})$ and train a denoiser or velocity field on noisy latents. A noise scheduler supplies time-dependent coefficients $(\alpha_{t},\sigma_{t})$ controlling signal and noise scales, and the forward noising is: $$\smash{
\mathbf{z}(t, \boldsymbol{\epsilon}) \;=\; \alpha_{t}\,\mathbf{h} \;+\; \sigma_{t}\,\boldsymbol{\epsilon},
\quad \boldsymbol{\epsilon} \sim \mathcal{N}(0,\mathbf{I}), \quad t \in \{1,\ldots,T\}.
}
\label{eq:forward}$$ *Denoising diffusion* [@ho2020denoising] trains a network $\boldsymbol{\epsilon}_{\boldsymbol{\theta}}(\mathbf{z},\mathbf{c},t)$ to predict the injected noise: $$\smash{
\mathcal{L}_{\textnormal{diff}}(\boldsymbol{\theta};\mathbf{v},\mathbf{c})
\;=\;
\mathbb{E}_{t,\boldsymbol{\epsilon}}\!\left[
\left\lVert
\boldsymbol{\epsilon}_{\boldsymbol{\theta}}(\mathbf{z}(t, \boldsymbol{\epsilon}),\mathbf{c},t) - \boldsymbol{\epsilon}
\right\rVert_2^2
\right].
}
\label{eq:ldiff}$$ *Flow matching* [@lipman2022flow; @albergo2023stochastic] learns a time-dependent vector field $\mathbf{f}_{\boldsymbol{\theta}}(\mathbf{z}_t,\mathbf{c},t)$ that matches the instantaneous velocity $\dot{\mathbf{z}}=\tfrac{\textnormal{d}}{\textnormal{d}t}\mathbf{z}$ induced by a chosen interpolant: $$\smash{
\mathcal{L}_{\textnormal{flow}}(\boldsymbol{\theta};\mathbf{v},\mathbf{c})
\;=\;
\mathbb{E}_{t,\boldsymbol{\epsilon}}\!\left[
\left\lVert
\mathbf{f}_{\boldsymbol{\theta}}(\mathbf{z}(t, \boldsymbol{\epsilon}),\mathbf{c},t) - \dot{\mathbf{z}}(t, \boldsymbol{\epsilon})
\right\rVert_2^2
\right].
}
\label{eq:lflow}$$ Both objectives train time-indexed predictors over the latent space by integrating over $t$ and $\boldsymbol{\epsilon}$, thus gradient-based methods like attribution share similar challenges.
**From images to video for generation.** Adding a temporal axis materially changes modeling and training. Generation must capture spatial appearance and temporal dynamics such as object and camera motion, deformations, and interactions. Modern systems extend image backbones with temporal capacity, for example, 3D U-Nets or 2D U-Nets augmented with temporal attention, causal or sliding-window context, and factorized space-time blocks, often trained in a latent-video VAE that compresses frames while preserving temporal cues. Training departs from images along several axes, which we address in §`\ref{sec:method}`{=latex}: (i) *Compute and storage.* Longer sequences multiply the cost of sampling timesteps, noise draws, and frames, motivating fixed-timestep or small-subset estimators that reduce variance without prohibitive cost (§`\ref{sec:method:1}`{=latex}). (ii) *Variable horizon.* Clips vary in $F$ and frame rate (§`\ref{sec:method:video-bias}`{=latex}). (iii) *Time-specific failure modes.* Typical artifacts include inconsistent trajectories, temporal flicker, identity drift, and physically implausible dynamics despite sharp individual frames (§`\ref{sec:method:2}`{=latex}).
**Motion representations in videos.** We denote our video as $\mathbf{v}=[\mathbf{f}_{f}]_{f=1}^{F}$ with $\mathbf{f}_{f}\in\mathbb{R}^{H\timesW\times 3}$ being the $f$-th frame. We represent motion via optical flow between consecutive frames: $\mathbf{F}_{f}:\{1,\ldots,H\}\times\{1,\ldots,W\}\to\mathbb{R}^2$, where each flow vector in $\mathbb{R}^2$ encodes the horizontal displacement $\mathrm{d}w$ and vertical displacement $\mathrm{d}h$ of a pixel. The motion magnitude is $M_{\!f}(h,w)=\|\mathbf{F}_{f}(h,w)\|_2$. The $M_{\!f}$ over frames $f$ and pixels $h,w$ summarizes the amount and spatial layout of motion in a clip, which we use to provide masks in our motion-weighted loss in §`\ref{sec:method}`{=latex}.
```{=latex}
\providecommand{\annleftshift}{-3.0em}
```
```{=latex}
\providecommand{\annrightshift}{ 3.0em}
```
## Data Attribution {#subsec:attribution}
Data attribution measures how individual training samples affect a model's predictions [@bae2024training]. A classic approach to data attribution is to use influence functions [@koh2017understanding]. Intuitively, the influence of a training sample measures: if we upweight this training example, how much would the model's prediction on a test datum change? Consider a loss function $\mathcal{L}(\boldsymbol{\theta};\mathbf{x})$ and a test sample $\mathbf{x}_{\textnormal{test}}$, the influence of a training point $\mathbf{x}_{n}$ can be quantified as: $$\begin{aligned}
I(\mathbf{x}_{n},\mathbf{x}_{\textnormal{test}})
&~=~
- \nabla_{\boldsymbol{\theta}}\mathcal{L}(\boldsymbol{\theta};\mathbf{x}_{\textnormal{test}})^{\!\top}
\,\mathbf{H}_{\boldsymbol{\theta}}^{-1}\,
\nabla_{\boldsymbol{\theta}}\mathcal{L}(\boldsymbol{\theta};\mathbf{x}_{n}), \quad
\mathbf{H}_{\boldsymbol{\theta}}
=\tfrac{1}{N}\sum\nolimits_{n=1}^{N}
\nabla_{\boldsymbol{\theta}}^{2}\mathcal{L}(\boldsymbol{\theta};\mathbf{x}_{n}),
\end{aligned}$$ where the inverse Hessian captures the curvature of the loss landscape, yet computing or storing it is infeasible at modern model and dataset scales. Thus, practical methods (e.g., TracIn [@pruthi2020estimating] and TRAK [@park2023trak]) approximate influence via gradient inner products or gradient feature projections.
**Attribution in diffusion models.** Diffusion training aggregates gradients over timesteps $t$ and noise draws $\boldsymbol{\epsilon}$, where gradient norms vary systematically with $t$, producing a timestep bias where examples aligned with large-norm timesteps appear spuriously influential. Diffusion-ReTrac [@xie2024data] reduces this bias by normalizing gradients and sub-sampling $t$ and $\boldsymbol{\epsilon}$ for influence. Let $\mathcal{L}_{\textnormal{diff}}$ denote the diffusion loss, and with the sampled-timestep-and-noise set $\mathcal{T}$, we compute a cosine-style score for normalized test and train gradients:
normalized test gradients normalized training gradients
Averaging gradients over $(t,\boldsymbol{\epsilon})$ stabilizes estimates, and normalization mitigates timestep-induced scale effects. Attribution quality is also sensitive to the measurement function used to score examples, such as denoising loss versus likelihood proxies [@zheng2023intriguing].
**Why vanilla attribution is insufficient for videos.** Naïvely applying gradient-based attribution to video diffusion risks treating appearance and motion alike, overemphasizing appearance (objects, textures, backgrounds) while overlooking dynamics [@park2025concept; @tulyakov2018mocogan]. Its cost grows with clip length, sampled timesteps, noise draws, and gradient dimensionality, making naïve methods impractical at modern video scales. To improve motion, we need attribution that suppresses static appearance, emphasizes motion-specific signals, and remains efficient (§`\ref{sec:method}`{=latex}). Motion is distributed across frames and entangled with static cues, thus influence cannot be assigned frame-independently.
# Method {#sec:method}
We formalize the problem in §`\ref{sec:formulation}`{=latex} and develop a practical framework for motion attribution in video diffusion models with four components: scalable gradient computation (§`\ref{sec:method:1}`{=latex}), frame-length bias fix (§`\ref{sec:method:video-bias}`{=latex}), motion-aware weighting (§`\ref{sec:method:2}`{=latex}), and target data selection (§`\ref{sec:method:3}`{=latex}). We provide efficiency analysis (§`\ref{sec:method:4}`{=latex}) demonstrating scalability to billion-parameter models and large-scale video datasets.
```{=latex}
\begin{figure*}[t]
\includegraphics[width=\textwidth]{fig/pipeline.pdf}
\caption{\textbf{Motive.}
\textit{Top.} Motion-gradient computation (\S\ref{sec:method:2}) has three steps: (1) detect motion with AllTracker, (2) compute motion-magnitude patches, (3) apply loss-space motion masks to focus gradients on dynamic regions.
\textit{Bottom.} Our method (\S\ref{sec:method:1}) is made scalable via a single-sample variant with common randomness and a projection, computed for each pair of training and query data, aggregated (\S\ref{sec:method:3}) for a final ranking, and eventually used to select fine-tuning subsets.}
\label{fig:pipeline}
\end{figure*}
```
## Problem Formulation {#sec:formulation}
We study data attribution for motion in the fine-tuning setting. Let $\mathcal{D}_{\textnormal{ft}} = \{(\mathbf{v}_{n}, \mathbf{c}_{n})\}_{n=1}^{N}$ be the fine-tuning corpus. Given a query video $(\hat{\mathbf{v}},\hat{\mathbf{c}})$, we assign to each training clip $(\mathbf{v}_{n},\mathbf{c}_{n})$ a motion-aware influence score $I(\mathbf{v}_{n}, \hat{\mathbf{v}}; \boldsymbol{\theta})$ that explains how it contributes to the dynamics observed in $\hat{\mathbf{v}}$. The score should satisfy: (i) *predictivity*, rankings correlate with observed changes from fine-tuning on the most influential subsets; (ii) *efficiency*, scales to modern video generators, such as forgoing explicit Hessian inversion, expensive per-data integration, or prohibitive storage. To do this, we augment the influence target defined in `\eqref{eq:diffusion-influence}`{=latex} to be (a) lower variance for stable rankings with feasible levels of compute, (b) more scalable to store, and (c) motion-centric.
**Fine-tuning Subset Selection.** For a budget $K\llN$, we get a motion-influential subset by ranking scores and taking the top-$K$ examples. For multiple query motions, we combine selections as described in §`\ref{sec:method:3}`{=latex}. The resulting subsets serve as candidates for motion-centric fine-tuning.
## Scalable Gradient-based Attribution {#sec:method:1}
We make attribution scalable for modern, large video datasets and models via inverse-Hessian approximations, lower-variance gradient-similarity estimators, low-cost single-sample estimators, and a Fastfood projection for tractable storage.
**Approximating the inverse-Hessian.** Computing exact inverse-Hessian-vector products is infeasible for modern neural networks. We estimate influence via gradient similarity, using an identity preconditioner for the inverse Hessian [@koh2017understanding; @pruthi2020estimating; @park2023trak].
**Common randomness for stable rankings.** To reduce variance without changing the target, we evaluate train and test gradients under the same $(t,\boldsymbol{\epsilon})$ pairs and average over a small set $\mathcal{T}$ [@xie2024data; @lin2024diffusion]. This paired averaging stabilizes rankings compared to independent draws:
**Single-sample variant for reduced compute.** We fix a single $t_{\textnormal{fix}}$ and a shared draw $\boldsymbol{\epsilon}_{\textnormal{fix}}\sim\mathcal{N}(0,\mathbf{I})$ for all train--test pairs. Sharing $(t_{\textnormal{fix}},\boldsymbol{\epsilon}_{\textnormal{fix}})$ allows low enough variance for the efficient single-sample estimator to maintain relative ordering [@xie2024data; @lin2024diffusion]. The estimator becomes:
**Structured projection for reduced storage.** To operate at model scale, we apply a Johnson--Lindenstrauss projection via Fastfood [@le2014fastfood] and then normalize. Let
$$\mathbf{P} \in \mathbb{R}^{D' \times D}
\;\;\text{implemented as}\;\;
\mathbf{P} :=\tfrac{1}{\xi\sqrt{D'}}\, \mathbf{S} \mathbf{Q} \mathbf{G} \boldsymbol{\Pi} \mathbf{Q} \mathbf{B},$$ where $\mathbf{Q}$ is the Walsh--Hadamard matrix, $\mathbf{B}$ is a diagonal Rademacher matrix, $\boldsymbol{\Pi}$ is a random permutation, $\mathbf{G}$ is a diagonal Gaussian scaling, and $\mathbf{S}$ is a diagonal rescaling, and $\xi$ normalizes the variance. The projected, normalized gradient is:
$$\tilde{\mathbf{g}}\big(\boldsymbol{\theta}, \mathbf{x}\big)
~:=~
\frac{
\mathbf{P}\,\nabla_{\boldsymbol{\theta}}\mathcal{L}_{\textnormal{diff}}(\boldsymbol{\theta}, \mathbf{x},t_{\textnormal{fix}},\boldsymbol{\epsilon}_{\textnormal{fix}})
}{
\|\mathbf{P}\,\nabla_{\boldsymbol{\theta}}\mathcal{L}_{\textnormal{diff}}(\boldsymbol{\theta}, \mathbf{x},t_{\textnormal{fix}},\boldsymbol{\epsilon}_{\textnormal{fix}})\|
}.$$ Then the influence score is the dot product of normalized projected gradients (i.e., cosine similarity) in $\mathbb{R}^{D'}$:
$$\label{eq:diffusion-influence-tilde}
I_{\textnormal{diff}}^{3}(\mathbf{x}_{n},\mathbf{x}_{\textnormal{test}})
~=~
\eqnmarkbox[NavyBlue]{tildegtest}{
\tilde{\mathbf{g}}\!\big(\boldsymbol{\theta};\,\mathbf{x}_{\textnormal{test}}\big)^{\!\top}}
\;\;
\eqnmarkbox[OliveGreen]{tildegtrain}{
\tilde{\mathbf{g}}\!\big(\boldsymbol{\theta};\,\mathbf{x}_{n}\big)}.$$ `\annotate[xshift=\annleftshift ,yshift=-0.0em]{below,left }{tildegtest}{projected, normalized test gradient }`{=latex} `\annotate[xshift=\annrightshift,yshift=-0.1em]{below,right}{tildegtrain}{projected, normalized training gradient}`{=latex}
This keeps compute $\mathcal{O}(D'\logD')$ for projection and $\mathcal{O}(D')$ per dot product, with storage $\mathcal{O}(|\mathcal{D}|\,D')$, while staying close to the ranking behavior of full-gradient cosine similarity [@park2023trak].
## Video-specific Frame-length Bias Fix {#sec:method:video-bias}
Raw gradient magnitudes depend on the number of frames $F$ in the video $\mathbf{v}$, thereby biasing scores toward longer videos. We correct this by normalizing for frame count before the projection--normalization step: $$\smash{
\nabla_{\boldsymbol{\theta}}\mathcal{L}_{\textnormal{diff}}(\boldsymbol{\theta};\,\mathbf{v},t_{\textnormal{fix}},\boldsymbol{\epsilon}_{\textnormal{fix}})
\;\leftarrow\;
\frac{1}{F}\,
\nabla_{\boldsymbol{\theta}}\mathcal{L}_{\textnormal{diff}}(\boldsymbol{\theta};\,\mathbf{v},t_{\textnormal{fix}},\boldsymbol{\epsilon}_{\textnormal{fix}}).
}$$ We still apply $\ell_2$ normalization in `\eqref{eq:diffusion-influence-tilde}`{=latex}, further stabilizing scales across examples. Together, single-timestep, common randomness, projection, and frame-length correction form a compact, scalable estimator that we use throughout. However, naïve video-level attribution conflates appearance with motion, often ranking clips high due to shared backgrounds or objects, while offering little insight into dynamics.
## Motion Attribution {#sec:method:2}
To move beyond whole-video influence, we introduce motion attribution, which isolates the contribution of training data to temporal dynamics. Unlike video-level attribution, which treats each clip as a single unit and conflates appearance with motion, motion attribution reweights per-location gradients using motion masks, assigning influence via dynamic behavior rather than static content.
**Motion Masking Attribution.** Motion is what distinguishes video diffusion from image diffusion. Our goal is to understand how training data shapes motion in video diffusion models. Prior work has emphasized architectural or algorithmic changes for motion modeling [@peebles2023scalable; @blattmann2023stable; @guo2023animatediff], many of the largest generative gains have instead come from scaling and curating massive video corpora, which in turn enable impressive motion synthesis results in video diffusion models [@ho2022video; @wan2025; @tan2024vidgen; @yang2024cogvideox]. Yet we lack tools that quantify how specific training clips shape particular motion patterns. We address this by attributing motion back to data via motion-weighted gradients, which yields actionable signals for targeted data selection, artifact diagnosis, and selective fine-tuning.
**Motion Detection and Latent Space Mapping.** Given a video $\mathbf{v} \in \mathbb{R}^{F \times H \times W \times 3}$ with $F$ frames of resolution $H \times W$, we first encode it into the VAE latent space as $\mathbf{h} = E(\mathbf{v}) \in \mathbb{R}^{F \times \nicefrac{H}{s} \times \nicefrac{W}{s} \times C}$, with downsampling factor $s = 8$ and $C = 16$ following the **`Wan2.1`** backbone used in our experiments.
For motion computation, we use AllTracker [@harley2025alltracker] to extract motion information in pixel space: $A = \mathcal{A}(\mathbf{v}) \in \mathbb{R}^{F \times H \times W \times 4}$, where the first two channels contain optical flow maps $A_{:,:,:,0:2}$ indicating pixel displacement between frames, and the remaining channels $A_{:,:,:,2:4}$ encode visibility and confidence scores. We extract displacement vectors at each pixel location as: $$\label{eq:displacement-vector}
\smash{
\mathbf{D}_{\!f}(h,w) = (A_{f,h,w,0}, A_{f,h,w,1}) = (\mathrm{d}w, \mathrm{d}h).
}$$ We then bilinearly downsample motion quantities from $(H,W)$ to the latent grid $\big(\tfrac{H}{s},\tfrac{W}{s}\big)$ so that our masking lives where gradients are computed.
**Motion-Weighted Gradient Computation.** We define the motion magnitude at each location as: $M_{\!f}(h,w)
\!=\!\|\mathbf{D}_{\!f}(h,w)\|_2$. To obtain comparable motion weights across frames and pixels, we min--max normalize over all frames and pixels, ensuring values lie in $[0,1]$: $$\label{eq:motion-weight}
\mathbf{W}(f,h,w)
=
\frac{M_{\!f}(h,w)-m_{\min}}
{m_{\max}-m_{\min}+\zeta},$$ where $m_{\min}=\min_{f',h',w'}M_{\!f}(h',w')$, $m_{\max}=\max_{f',h',w'}M_{\!f}(h',w')$, and $\zeta=10^{-6}$ ensures a positive denominator. This normalization mitigates bias from absolute motion scale, yielding weights that emphasize relative motion saliency rather than raw magnitude, following prior practice in video saliency detection [@fang2013video]. Let $(\tilde{h},\tilde{w})$ index the latent grid. We obtain latent-aligned weights by bilinear downsampling: $$\label{eq:motion-downsample}
\smash{
\tilde{\mathbf{W}}(f,\tilde{h},\tilde{w})
~=~\operatorname{Bilinear}\!\left(\mathbf{W}(\cdot, \cdot,\cdot),F,~\tfrac{H}{s},~\tfrac{W}{s}\right).
}$$ We compute per-location squared error at fixed $(t_{\textnormal{fix}},\boldsymbol{\epsilon}_{\textnormal{fix}})$ at each frame $f$ and \`\`latent pixel" $(\tilde{h},\tilde{w})$:
$$\label{eq:per-location-error}
\begin{aligned}
\tilde{\mathcal{L}}_{\boldsymbol{\theta},\mathbf{v},\mathbf{c}}(f, \tilde{h},\tilde{w})
~=~\Big(\,&[\boldsymbol{\epsilon}_{\boldsymbol{\theta}}(\mathbf{z}(\mathbf{v}, t_{\textnormal{fix}}, \boldsymbol{\epsilon}_{\textnormal{fix}}),t_{\textnormal{fix}},\mathbf{c})]_{f,\tilde{h},\tilde{w}}
- [\boldsymbol{\epsilon}_{\textnormal{target}}(t_{\textnormal{fix}},\boldsymbol{\epsilon}_{\textnormal{fix}})]_{f,\tilde{h},\tilde{w}}\,\Big)^2,
\end{aligned}$$ and define the motion-weighted loss by averaging over frames and latent spatial locations: $$\label{eq:motion-loss}
\mathcal{L}_{\textnormal{mot}}(\boldsymbol{\theta};\mathbf{v},\mathbf{c})
~=~\frac{1}{F_{\mathbf{v}}} \operatorname*{mean}_{f,\tilde{h},\tilde{w}}
\Big[\tilde{\mathbf{W}}_{\mathbf{v},\mathbf{c}}(f,\tilde{h},\tilde{w})\cdot \tilde{\mathcal{L}}_{\boldsymbol{\theta},\mathbf{v},\mathbf{c}}(f, \tilde{h},\tilde{w})\Big].$$ Notably, when $\tilde{\mathbf{W}}$ is all ones, this recovers the standard objective with no motion emphasis. The $\nicefrac{1}{F_{\mathbf{v}}}$ factor corrects for frame-length bias and $F_{\mathbf{v}}$ signifies how the number of frames may be video-dependent. The corresponding motion-weighted gradient for attribution is: $$\begin{aligned}
\label{eq:diffusion-influence-motion}
I_{\textnormal{mot}}\!(\mathbf{v}_{n},\!\hat{\mathbf{v}})
&\!=\!
\tilde{\mathbf{g}}_{\textnormal{mot}}\!(\boldsymbol{\theta}\!,\!\hat{\mathbf{v}})^{\!\top}\!
\tilde{\mathbf{g}}_{\textnormal{mot}}\!(\boldsymbol{\theta}\!,\!\mathbf{v}_{n}), \quad
\textnormal{where } \mathbf{g}_{\textnormal{mot}} \!:=\! \nabla_{\!\boldsymbol{\theta}}\mathcal{L}_{\textnormal{mot}}
\textnormal{ and } \tilde{\mathbf{g}}_{\textnormal{mot}}\!(\boldsymbol{\theta}\!,\!\mathbf{v}) \!:=\!
\frac{
\mathbf{P}\mathbf{g}_{\textnormal{mot}}\!(\boldsymbol{\theta}\!,\!\mathbf{v}\!,\!t_{\textnormal{fix}},\!\boldsymbol{\epsilon}_{\textnormal{fix}})
}{
\|\mathbf{P}\mathbf{g}_{\textnormal{mot}}\!(\boldsymbol{\theta}\!,\!\mathbf{v}\!,\!t_{\textnormal{fix}},\!\boldsymbol{\epsilon}_{\textnormal{fix}})\|
}.
\end{aligned}$$
Loss-space masking leaves forward noising and generation unchanged and reweights only attribution, avoiding interactions between motion weighting and noise injection. In contrast, our motion-aware attribution emphasizes dynamic regions and de-emphasizes static backgrounds, so rankings identify training clips that most strongly shape the model's motion rather than appearance.
## Most Influential Fine-tuning Subset Selection {#sec:method:3}
**Goal.** Given a query clip $(\hat{\mathbf{v}},\hat{\mathbf{c}})$, we compute a motion-aware attribution value for each fine-tuning sample $(\mathbf{v}_{n},\mathbf{c}_{n}) \in \mathcal{D}_{\textnormal{ft}}$ using $I_{\textnormal{mot}}\!\left(\mathbf{v}_{n},\,\hat{\mathbf{v}}\right)$ (`\eqref{eq:diffusion-influence-motion}`{=latex}). We construct a fine-tuning set $\mathcal{S}$ for one or many query videos $\hat{\mathbf{v}}$.
**Single-query-point fine-tuning selection.** For a budget of $K$ data points, we select the $K$ highest-scoring examples. In practice, $K$ is chosen as a percentile of the dataset size (e.g., top 1--10%), ensuring the subset scales consistently across datasets.
**Multi-query-point fine-tuning selection: aggregating attribution scores.** For $Q$ queries, we adopt the majority voting approach from ICONS [@wu2024icons] and aggregate motion-aware influence scores across queries by percentile thresholding and voting. A sample receives a vote if the score is above the percentile cutoff $\tau$ for that query. The consensus score of a candidate $\mathbf{v}_{n}$ is the total number of queries that vote for it. We then rank all training samples by $\operatorname{MajVote}(\mathbf{v}_{n})$ and select the top-$K$ to form the fine-tuning subset. This formulation emphasizes samples that are consistently influential across multiple queries, without requiring cross-query calibration of raw scores: $$\label{eq:majority-vote}
\begin{aligned}
\operatorname{MajVote}_{n}
&\!=\!
\sum\nolimits_{q=1}^{Q}\!\mathbb{I}\!\big[\,
I_{\textnormal{mot}}(\mathbf{v}_{n},\hat{\mathbf{v}}_q) \!>\! \tau
\,\big], \quad
\mathcal{S}_{\textnormal{vote}}(K)
\!=\!
\bigl\{\mathbf{v}_{n} | \mathbf{v}_{n}
\text{ in top-}K \text{ by } \operatorname{MajVote} \bigr\}.
\end{aligned}$$
## Computational Efficiency Analysis {#sec:method:4}
**Gradient Compute.** Naïvely averaging over timesteps and noise for every example costs $\mathcal{O}(|\mathcal{D}|\,|\mathcal{T}|\,B)$, where $B$ is a single forward+backward cost and $|\mathcal{T}|$ is the number of sampled $t, \boldsymbol{\epsilon}$ per data. Using a single sample reduces this to $\mathcal{O}(|\mathcal{D}|\,B)$, which is key to keeping the cost reasonable for modern video datasets and models, while reusing a sample across data allows low enough variance for stable rankings. Projection adds $\mathcal{O}(D' \log D')$ per example using Fastfood [@le2014fastfood], negligible relative to a backward pass.
**Gradient Storage.** Storing full gradients is $\mathcal{O}(|\mathcal{D}|\,D)$. We instead store only projected vectors, $\mathcal{O}(|\mathcal{D}|\,D')$, plus the structured Fastfood state, $\mathcal{O}(D)$. Since $D'$ is typically orders of magnitude smaller than $D$, this transformation makes storage tractable for billion-parameter models.
**Data Ranking Compute.** Influence computation in `\eqref{eq:diffusion-influence-tilde}`{=latex} is an inner product in $\mathbb{R}^{D'}$, so evaluating all train examples against a query is $\mathcal{O}(|\mathcal{D}|\,D')$, and sorting is $\mathcal{O}(|\mathcal{D}|\log|\mathcal{D}|)$.
```{=latex}
\begin{figure*}[t]
\includegraphics[width=\textwidth]{fig/fig2.png}
\caption{\textbf{Motion Attribution Samples} with Wan2.1-T2V-1.3B.
\textit{Top}: Query clips showing float (\textit{left}) and roll (\textit{right}) motions.
\textit{Middle}:
Top-ranked positive training samples identified by Motive~with high influence scores.
\textit{Bottom}: Negative influence samples with minimal, camera-only motion, or cartoon-style content that conflict with target motions. Videos are included in the supplementary material.}
\label{fig:influence_examples}
\end{figure*}
```
**Additional Motion-Emphasis Compute.** Motion-specific overhead primarily stems from AllTracker mask extraction with complexity $\mathcal{O}(|\mathcal{D}| \cdot H \cdot W \cdot F)$ for clip length $F$ and frame resolution $H \times W$. Masks are extracted once, cached, and negligible relative to gradient cost. We provide a detailed runtime breakdown in App. `\ref{app:runtime}`{=latex}.
# Experiment {#sec:experiment}
## Setup {#sec:experiment:setup}
**Fine-tuning Datasets.** We evaluate our motion attribution framework on two large-scale video datasets: **`VIDGEN-1M`** [@tan2024vidgen] and **`4DNeX-10M`** [@chen20254dnex], both of which offer diverse motion patterns, rich temporal dynamics, and complex scenes. For our experiments, we use 10k videos from both datasets, which provide sufficient scale and diversity to thoroughly evaluate motion attribution methods across different temporal patterns and video generation scenarios.
**Motion Query Data.** To evaluate our motion attribution, we curate a set of query videos representing distinct motion patterns and scenarios. Our query dataset consists of videos spanning $10$ motion categories, with a focus on object dynamics: *compress, bounce, roll, explode, float, free fall, slide, spin, stretch, swing*. Five videos, totaling $50$ queries, represent each motion type. These videos are chosen for their clear, isolated motions, serving as a basis for evaluating attribution quality and downstream motion generation. Details are in App. `\ref{app:motion_query_data}`{=latex}.
**Model & Baselines.** We use pretrained models **`Wan2.1-T2V-1.3B`** and **`Wan2.2-TI2V-5B`**, with additional results on **`LTX-2B`** in App. `\ref{sec:add_experiments}`{=latex}. Our baselines: Base model: pretrained, no fine-tuning; Full fine-tuning: approximate upper bound using the complete dataset; Random selection: uniform sampling; Motion magnitude: selects videos with the highest average motion magnitude; V-JEPA embeddings: selects most representative videos of motion patterns using V-JEPA [@assran2025v] features, capturing high-level motion semantics; and Ours w/o motion masking: influence of the entire video level without motion-specific masking.
```{=latex}
\begin{figure*}[!t]
\includegraphics[width=\textwidth]{fig/fig3.png}
\caption{\textbf{Qualitative Comparisons.}
We compare four motion scenarios across the base model (Wan2.1-T2V-1.3B), or fine-tuned with random selection and Motive. Our approach produces more realistic motion. Videos are included in the supplementary material.
}
\label{fig:qualitative_results}
\end{figure*}
```
```{=latex}
\begin{table*}[t!]
\caption{\textbf{VBench Evaluation.} Performance comparison on VBench~\citep{huang2024vbench} across different baselines (all values in $\%$, higher is better). All selection methods use $10\%$ of training data; our method uses majority vote aggregation (\S\ref{sec:method:3}) across motion queries. MM: motion masking. Subj.: Subject Consistency, Bg.: Background Consistency, Mot.: Motion Smoothness, Dyn.: Dynamic Degree, Aesth.: Aesthetic Quality, Img.: Imaging Quality.
}
\label{tab:vbench-wan}
\small
\renewcommand{\arraystretch}{1.12}
\setlength{\tabcolsep}{4pt}
\resizebox{0.95\textwidth}{!}{%
\begin{tabular}{@{}l*{6}{r}@{\vrule}*{6}{r}@{}}
\toprule
& Wan2.1-T2V-1.3B & Wan2.2-TI2V-5B \\
\cmidrule(lr){2-7} \cmidrule(lr){8-13}
Method & Subj. & Bg. & Mot. & Dyn. & Aesth. & Img. & Subj. & Bg. & Mot. & Dyn. & Aesth. & Img. \\
\midrule
Base & 95.3 & 96.4 & 96.3 & 39.6 & 45.3 & \textbf{65.7} & 94.9 & 96.4 & 97.5 & 42.0 & 44.4 & 65.5 \\
Full FT & 95.9 & \textbf{96.6} & 96.3 & 42.0 & 45.0 & 63.9 & \textbf{95.3} & 96.5 & 97.5 & 45.3 & 44.8 & \textbf{66.2} \\
Random & 95.3 & 96.6 & 96.3 & 41.3 & 45.7 & 65.1 & 94.7 & 96.2 & 97.3 & 41.6 & 44.6 & 65.2 \\
Motion mag. & 95.6 & 96.2 & 95.7 & 40.1 & 45.1 & 63.2 & 95.0 & 96.3 & 97.4 & 44.9 & 45.0 & 65.1 \\
V-JEPA & 95.7 & 96.0 & 95.6 & 41.6 & 44.9 & 62.7 & 95.2 & 96.4 & 97.3 & 45.6 & 44.9 & 64.8 \\\midrule
Ours w/o MM & 95.4 & 96.1 & 96.3 & 43.8 & 45.7 & 63.2 & 94.9 & 96.5 & 97.4 & 43.8 & 45.2 & 64.8 \\
\rowcolor{ggreen!10}
\textbf{Ours (Motive)} & \textbf{96.3} & 96.1 & 96.3 & \textbf{47.6} & \textbf{46.0} & 64.6 & 95.1 & \textbf{96.6} & \textbf{97.6} & \textbf{48.3} & \textbf{45.6} & 65.5 \\
\bottomrule
\end{tabular}
}
\end{table*}
```
**Benchmark.** We evaluate our attribution using VBench [@huang2024vbench] metrics across six dimensions: subject consistency, background consistency, motion smoothness, dynamic degree, aesthetic quality, and imaging quality. Motion smoothness and dynamic degree are our primary targets for temporal dynamics, while other metrics help maintain visual quality. We use custom evaluation prompts, following VBench's descriptive style, for 10 motion types, with $5$ prompts each, to assess our framework's effectiveness on target motions.
**Implementation Details.** We finetune base models with our Motive-selected high-quality video data following the official and DiffSynth-Studio implementation. During fine-tuning, we update only the DiT backbone while freezing the T5 text encoder and VAE. All models are trained at a resolution of $480\times832$ with a learning rate of $1 \times 10^{-5}$. Specialist models are trained on single motion category selected data, while generalist models use aggregated selections (both with top $10\%$ selection from **`VIDGEN-1M`** [@tan2024vidgen] or **`4DNeX`** [@chen20254dnex]). Compute and runtime details are provided in App. `\ref{app:runtime}`{=latex}.
## Main Results {#sec:main_results}
**High-influence selection and negative filtering.** Fig. `\ref{fig:influence_examples}`{=latex} shows our motion-aware attribution ranks clips with clear, physically grounded dynamics and downranks those with little transferable motion. For rolling and floating, positives show continuous trajectories and smooth temporal evolution (turbulent water, planetary rotation). Negatives are mostly static footage, camera-only motion, or cartoons whose simplified kinematics do not transfer. This pattern holds across motion categories and aligns with the quantitative gains below.
**Qualitative improvements across motion types.** Fig. `\ref{fig:qualitative_results}`{=latex} compares the base **`Wan2.1-T2V-1.3B`** model, finetuned with random selection or Motive with equal data budgets, across $4$ scenarios. Our method yields higher motion fidelity and temporal consistency than baselines, especially for complex deformation and physics-driven motion.
**Quantitative Results.** We evaluate our approach using VBench [@huang2024vbench], demonstrating consistent improvements in motion when fine-tuning with attribution-selected data. As shown in Tab. `\ref{tab:vbench-wan}`{=latex}, Motive consistently achieves the highest dynamic degree scores across both models: $47.6\%$ on **`Wan2.1-T2V-1.3B`** and $48.3\%$ on **`Wan2.2-TI2V-5B`**, significantly outperforming random selection ($41.3\%$ and $41.6\%$) and whole video attribution ($43.8\%$ for both models). Our method also excels in aesthetic quality ($46.0\%$ and $45.6\%$), while maintaining competitive motion smoothness ($96.3\%$ and $97.6\%$). Notably, using only $10\%$ of training data, our approach surpasses the full fine-tuning on dynamic degree ($42.0\%$ and $45.3\%$) on both models, demonstrating superior performance of motion-specific attribution for targeted fine-tuning. Additional results on **`LTX-2B`** are in App. `\ref{sec:add_experiments}`{=latex}; motion magnitude distribution analysis is in App. `\ref{sec:analysis}`{=latex}.
```{=latex}
\begin{figure*}[!t]
\includegraphics[width=\textwidth]{fig/fig5.png}
\caption{\textbf{Impact of Frame-Length Normalization on Motion Attribution.} Comparison of top-ranked samples for floating motion query. \textbf{Left}: With proper frame-length normalization, top samples consistently exhibit floating motion (waves, floating objects, surfing). \textbf{Right}: Without normalization, rankings are biased by video length, resulting in no coherent patterns among top samples.
}
\label{fig:frame_norm}
\end{figure*}
```
## Human Evaluation
```{=latex}
\begin{wraptable}[10]{r}{0.45\textwidth}
\small
\begin{tabular}{@{}lccc@{}}
\toprule
\textbf{Method} & \textbf{Win ($\%$)} & \textbf{Tie ($\%$)} & \textbf{Loss ($\%$)} \\
\midrule
Ours vs. Base & $74.1$ & $12.3$ & $13.6$ \\ %
Ours vs. Random & $58.9$ & $12.1$ & $29.0$ \\ %
Ours vs. Full FT & $53.1$ & $14.8$ & $32.1$ \\ %
Ours vs. w/o MM & $46.9$ & $20.0$ & $33.1$ \\ %
\bottomrule
\end{tabular}
\caption{\textbf{Human evaluation.} Pairwise comparisons across $50$ videos with $17$ participants ($850$ total). Win, tie, and loss rates show where our method is preferred, rated equal, or outperformed.}
\label{tab:human_eval_extended}
\end{wraptable}
```
Automated scores can miss perceptual motion quality, so we run a human evaluation pairwise comparison protocol: participants view two generated videos and choose which shows better motion. We recruit $17$ annotators and evaluate $10$ motion categories. For each category, we prepare $5$ test cases and compare our method to baselines across three pairings, yielding a balanced set of judgments. Presentation order is randomized, and ties are allowed. We report win rate (fraction our method is preferred), tie rate, and overall preference. As shown in Tab. `\ref{tab:human_eval_extended}`{=latex}, annotators favor our attribution-guided selection: $74.1\%$ win rate vs. the base model and $53.1\%$ vs. the full fine-tuned model, showing perceptually meaningful motion improvements.
## Ablations {#subsec:ablations}
```{=latex}
\begin{wrapfigure}{r}{0.45\textwidth}
\begin{tikzpicture}
\node (img11) {\includegraphics[width=.9\linewidth]{fig/1_projection_correlation_analysis.pdf}};
\node[left=of img11, rotate=90, anchor=center, xshift=0.2cm, yshift=-1.0cm] {Spearman Correlation $\rho$};
\node[below=of img11, node distance=0cm, xshift=0.1cm, yshift=1.3cm, font=\color{black}] {Projection Dimension};
\end{tikzpicture}
\caption{\textbf{Projection dimension analysis.} Spearman correlation between projected and full gradients shows rapid improvement with projection dimension, with $512$ providing a strong trade-off between accuracy and efficiency.}
\label{fig:projection_correlation}
\end{wrapfigure}
```
**Projected Gradients Preserve Influence Rankings.** Comparing full gradients for attribution is infeasible at a billion-parameter scale. We reduce dimensionality with structured random projections that preserve influence geometry, ablating $D' \!\in\! \{128,\dots,2048\}$ against the full-gradient baseline. We assess ranking preservation via Spearman correlation with unprojected scores (Fig. `\ref{fig:projection_correlation}`{=latex}). Small projections preserve rankings poorly: $D'\!=\!128$ yields $\rho\!=\!46.9\%$. Preservation improves with size: $D'\!=\!512$ reaches $\rho\!=\!74.7\%$. Beyond that, gains are marginal while cost rises: $D'\!=\!1024$ ($\rho\!=\!75.7\%$) and $D'\!=\!2048$ ($\rho\!=\!76.1\%$). Thus, $D'\!=\!512$ offers the best trade-off, scaling to large models while maintaining quality.
**Single-timestep attribution.** Using a single timestep avoids the cost of averaging across timesteps while closely matching the multi-timestep baseline. With fixed $t\!=\!751$ (midpoint of the $1000$-step trajectory), we obtain $\rho\!=\!66\%$ agreement with ground truth computed using $10$ evenly-spaced timesteps based on flow matching schedule. High timesteps (early denoising) heavily obscure motion with noise; low timesteps (late denoising) operate on nearly formed videos, where gradients reflect fine details. $t\!=\!751$ (mid-denoising) balances influence ranking correlation and compute efficiency. Averaging multiple timesteps yields minimal gains, a single fixed timestep is therefore sufficient for variance-reduced, scalable attribution.
**Frame-Length Normalization.** As in the Wan training protocol, we standardize all videos to 81 frames at 16 fps (satisfying the $4n{+}1$ constraint) for fair attribution across clips of different raw lengths. Without standardization, gradient-based scores correlate strongly with video length rather than motion quality ($\rho\!=\!78.0\%$), leading to longer clips ranking higher regardless of dynamics. Standardizing frames reduces spurious length correlations by $54.0\%$ while preserving motion-based correlation, so rankings reflect motion rather than duration. As in Fig. `\ref{fig:frame_norm}`{=latex}, normalization clarifies motion-specific patterns. For floating queries with frame-length normalization (left), top-ranked samples consistently show wave dynamics, floating objects, and surfing, all matching the target motion. Without normalization (right), top samples lack coherent similarity because rankings are driven by clip length, harming motion-relevant training example identification.
# Conclusion
We address a central and underexplored question in video diffusion: which training clips influence the motion in generated videos? We introduced Motive, a motion-aware data attribution framework for video diffusion that isolates temporal dynamics from static appearance. By tracing generated motion back to influential training clips, our method enables targeted data curation that improves motion quality with a fraction of the data. As video models scale, such data-level understanding will be key to diagnosing failure modes and building more controllable generative systems.
# Acknowledgements {#acknowledgements .unnumbered}
We thank the following people (listed alphabetically by last name) for their helpful discussions, feedback, or participation in human studies: Allison Chen, Sanghyuk Chun, Zhiwei Deng, Amaya Dharmasiri, Xingyu Fu, Will Hwang, Yifeng Jiang, Amogh Joshi, Pang Wei Koh, Chen-Hsuan Lin, Huan Ling, Tiffany Ling, Shaowei Liu, Zhengyi Luo, Rafid Mahmood, Kaleb S. Newman, Julian Ost, Zeeshan Patel, Davis Rempe, Rulin Shao, Esin Tureci, Anya Tsvetkov, Jiachen T. Wang, Sheng-Yu Wang, Tingwu Wang, Zian Wang, Hongyu Wen, Jon Williams, Mengzhou Xia, Donglai Xiang, Yilun Xu, William Yang, and Haotian Zhang.
# Notation {#app:notation}
::: {#tab:notation}
**Symbol** **Description**
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------
*Acronyms and Basic Notation*
VAE Variational Autoencoder
DiT Diffusion Transformer backbone
$\mathbf{I}$ Identity matrix
*Video Generation*
$p_{\boldsymbol{\theta}}(\mathbf{v} \mid \mathbf{c})$ Conditional video generator with parameters $\boldsymbol{\theta}$
$\mathbf{v} \in \mathbb{R}^{F \times H \times W \times 3}$ Video clip with frames $F$, height $H$, width $W$
$\mathbf{c}$ Conditioning signal such as text or multimodal metadata
$\boldsymbol{\theta}$ Trainable model parameters
$f \in \{1,\ldots,F\}$ Frame index
$h \in \{1,\ldots,H\}$, $w \in \{1,\ldots,W\}$ Spatial indices for height and width respectively
$\tilde{h}$, $\tilde{w}$ Latent grid indices
*Datasets*
$\mathcal{D} = \{(\mathbf{v}_{n}, \mathbf{c}_{n})\}_{n=1}^{N}$ Training corpus with size $N$ and index $n$
$\mathcal{D}_{\textnormal{ft}} \subseteq \mathcal{D}$ Fine-tuning dataset
$\mathcal{S} \subseteq \mathcal{D}$ Selected influential subset
$k \in \{1,\ldots,K\}$ The selected subset size
$Q$ Number of query data
$q \in \{1,\ldots,Q\}$ Query index
$\mathbf{x}$ Generic input data pair
$\mathbf{x}_{\textnormal{test}}$, $\mathbf{x}_{n}$ Test/query pair and training pair
$\hat{\mathbf{v}}$, $\hat{\mathbf{c}}$ Query video and its conditioning
*Latent Space and Diffusion Components*
$E$ VAE encoder
$\mathbf{h} = E(\mathbf{v}) \in \mathbb{R}^{F \times (H/s) \times (W/s) \times C}$ Latent video with spatial factor $s$ and channels $C$
$\mathbf{z}$ Noisy latent variable used in diffusion or flow matching
$\boldsymbol{\epsilon} \sim \mathcal{N}(0,\mathbf{I})$ Gaussian noise
$\boldsymbol{\epsilon}_{\boldsymbol{\theta}}(\mathbf{z},\mathbf{c},t)$ Predicted noise network in diffusion training
$\mathbf{f}_{\boldsymbol{\theta}}(\mathbf{z},\mathbf{c},t)$ Time-indexed vector field in flow matching
$\dot{\mathbf{z}}$ Time derivative of the latent trajectory
$\alpha_{t}, \sigma_{t}$ Scheduler signal and noise scales at timestep $t$
$\boldsymbol{\epsilon}_{\textnormal{target}}$ Target noise or velocity used for supervision
$t \in \{1,\ldots,T\}$ Diffusion or flow-matching timestep, with total timesteps $T$
$t_{\textnormal{fix}}$, $\boldsymbol{\epsilon}_{\textnormal{fix}}$ Fixed timestep and shared noise draw used for low-variance gradients
: Glossary and notation.
:::
::: {#tab:notation-attribution}
**Symbol** **Description**
------------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------
*Attribution and Influence*
$I(\mathbf{v}_{n},\hat{\mathbf{v}};\boldsymbol{\theta})$ Influence score between a train clip and a query clip
$I_{\textnormal{mot}}(\mathbf{v}_{n},\hat{\mathbf{v}};\boldsymbol{\theta})$ Motion-aware influence score
$\operatorname{TopK}(\cdot)$ Top-$K$ operator for selecting highest scores
$\operatorname{MajVote}(\cdot)$ Majority-vote aggregation across queries
$\tau$ Percentile cutoff for voting
$\rho$ Spearman correlation coefficient
*Loss Functions*
$\mathcal{L}$ Generic loss
$\mathcal{L}_{\textnormal{diff}}(\boldsymbol{\theta};\mathbf{v},\mathbf{c}), \mathcal{L}_{\textnormal{flow}}(\boldsymbol{\theta};\mathbf{v},\mathbf{c})$ Diffusion and flow-matching objective
$\mathcal{L}_{\textnormal{mot}}(\boldsymbol{\theta};\mathbf{v},\mathbf{c})$ Motion-weighted objective used for attribution
$\tilde{\mathcal{L}}$ Per-location squared error in latent space
$\mathbf{g}$, $\tilde{\mathbf{g}}$ Gradient and its projected version
$\mathbf{g}_{\textnormal{mot}}$, $\tilde{\mathbf{g}}_{\textnormal{mot}}$ Motion-weighted gradient and its projection
$\mathbf{H}_{\boldsymbol{\theta}}$ Hessian with respect to $\boldsymbol{\theta}$
*Motion Representations*
$\mathcal{A}(\mathbf{v}) = A$ AllTracker motion extraction
$A \in \mathbb{R}^{F \times H \times W \times 4}$ Motion tensor containing flow, visibility, and confidence
$\mathbf{D}_{\!f}(h,w)$ Displacement vector at frame $f$ and location $(h,w)$
$M_{\!f}(h,w)$ Motion magnitude at a location, computed from the displacement
$\mathbf{W}(f,h,w) \in [0,1]$ Normalized motion weights used to mask per-location losses
$\zeta$ Small numerical bias for stability (e.g., $10^{-6}$)
*Projections and Computational Details*
$D$, $D'$ Full and projected gradient dimensions
$\mathbf{P} \in \mathbb{R}^{D' \times D}$ Projection matrix used for Fastfood-style JL projection
$\xi$ Variance normalization constant for projection
$\mathcal{T}$ Set of sampled $(t,\boldsymbol{\epsilon})$ pairs for gradient estimation
$B$ Unit compute cost used in complexity accounting
: Glossary and notation (continued).
:::
# Extended Related Work {#sec:app:related}
## Data Attribution {#data-attribution}
Understanding how individual training examples shape model behavior has been a long-standing goal. Modern data attribution methods fall into two main groups [@hammoudeh2024training]: retraining-based methods (e.g., leave-one-out [@cook1982residuals; @jia2021scalability], downsampling (also known as subsampling or counterfactual influence) [@feldman2020neural], Shapley-value [@wang2024data; @wang2024rethinking]) and gradient-based methods (influence-function family, including Influence Functions [@koh2017understanding; @lorraine2024jacnet], TracIn [@pruthi2020estimating], and TRAK [@park2023trak]). Influence functions provide a principled framework by approximating the effect of removing a training point. TracIn [@pruthi2020estimating] and TRAK [@park2023trak] make attribution feasible at scale. While effective for classification, these assume a mapping between training gradients and predictions, which becomes more complex in generative models.
Data attribution traces how individual training examples (or subsets) influence a model's predictions or behavior. Formally, it assigns an attribution score to each training sample, estimating the extent to which that sample contributes (positively or negatively) to the model's output on a given test query or behavior. Influence data attribution is an example of nested optimization [@vicol2022implicit; @lorraine2024scalable] with other examples including differentiable games [@balduzzi2018mechanics; @lorraine2021lyapunov; @lorraine2022complex], hyperparameter optimization [@raghu2021meta; @lorraine2018stochastic; @mehta2024improving; @lorraine2020optimizing], and variance-reduced gradient estimation for diffusion-based teacher--student distillation [@bettencourt2026carv]. Before diffusion models, attribution methods were applied to supervised learning tasks such as classification and regression, where influence functions [@koh2017understanding; @lorraine2022task] and scalable approximations such as TracIn [@pruthi2020estimating], TRAK [@park2023trak], and TDA [@bae2024training] quantified the impact of training examples on downstream predictions. Recent work adapted data attribution to diffusion models [@georgiev2023journey; @zheng2023intriguing; @wang2025fast; @wang2024data; @lin2024diffusion; @brokman2024montrage; @kwon2023datainf], where iterative denoising introduces timestep-dependent bias. @mlodozeniec2024influence propose scalable approximations, while @xie2024data identify timestep-induced artifacts and normalization schemes. Concept-TRAK [@park2025concept] extends attribution to concepts by reweighting gradients with concept-specific rewards, enabling attribution to semantic factors. @wang2023evaluating instead design a customization-based benchmark for text-to-image models, where models are fine-tuned on exemplar images with novel tokens and attribution is evaluated by whether it can recover the responsible exemplars. However, these are limited to image diffusion, which captures static appearance but not temporal dynamics.
## Motion in Video Generation
Video diffusion extends image generation to time, requiring coherent motion across frames [@ho2022video; @blattmann2023stable; @peebles2023scalable; @wan2025; @agarwal2025cosmos]. A large body of work builds temporal structure via attention layers [@wu2023tune], control signals [@chen2023control; @zhang2023controllable], feature correspondences [@geyer2023tokenflow; @bao2023latentwarp; @wang2024cove], or consistency distillation [@wang2023videolcm; @zhou2024upscale]. Recent work has highlighted the challenge of decoupling motion from appearance in video diffusion transformers, in which spatial and temporal information become entangled within the model's representations [@shi2025decouple]. However, understanding which training clips influence specific motion patterns in generated videos remains an open challenge.
In parallel, motion has long been studied using optical flow and correspondence, from classical formulations [@horn1981determining; @lucas1981iterative] to modern approaches such as RAFT [@teed2020raft], which improve accuracy and generalization. These priors are often repurposed during generation to guide dynamics, but they do not explain which training examples shaped a model's motion behavior. Our work addresses both gaps by introducing a motion-aware data attribution framework specifically designed for video diffusion. We use motion-weighted gradients that disentangle temporal dynamics from static appearance, enabling us to trace generated motion patterns back to the most influential training clips.
# Additional Experiments {#sec:add_experiments}
## Results on Additional Video Generation Models
We further test our framework on additional video generation architectures beyond the **`Wan`** models shown in Sec. `\ref{sec:experiment}`{=latex} (Tab. `\ref{tab:vbench-wan}`{=latex}). We evaluate Motive on **`LTX-2B`** [@HaCohen2024LTXVideo], which represents a different architectural design and training paradigm. The results in Tab. `\ref{tab:app-vbench-ltx}`{=latex} demonstrate that Motive works effectively across different model architectures and scales.
```{=latex}
\begin{table*}[t!]
\caption{\textbf{VBench Evaluation on LTX-2B.} Performance comparison on VBench~\citep{huang2024vbench} across different baselines (all values in $\%$, higher is better). All selection methods use $10\%$ of training data; MM: motion masking.
}
\label{tab:app-vbench-ltx}
\small
\renewcommand{\arraystretch}{1.12}
\setlength{\tabcolsep}{5pt}
\resizebox{\textwidth}{!}{%
\begin{tabular}{@{}l*{6}{c}@{}}
\toprule
& LTX-2B Video \\
\cmidrule(lr){2-7}
Method & Subject Consist. & Background Consist. & Motion Smooth. & Dynamic Degree & Aesthetic Qual. & Imaging Qual. \\
\midrule
Base & 93.8 & 95.2 & 94.6 & 36.5 & 32.1 & 48.2 \\
Full FT & 94.4 & \textbf{95.7} & 94.8 & 38.9 & 32.6 & \textbf{49.1} \\
Random & 93.7 & 95.4 & 94.7 & 37.8 & \textbf{32.8} & 48.4 \\
Motion mag.& 94.1 & 95.3 & 95.2 & 39.6 & 32.4 & 47.9 \\
V-JEPA & 94.2 & 95.5 & 95.1 & 40.2 & 31.9 & 47.5 \\
\addlinespace[2pt]
\cmidrule(lr){1-7}
Ours w/o MM& 94.3 & 95.6 & 95.3 & 41.8 & \textbf{32.8} & 48.7 \\
\rowcolor{ggreen!10}
\textbf{Ours (Motive)}
& \textbf{94.8} & 95.6 & \textbf{95.5} & \textbf{45.1} & 32.7 & 48.6 \\
\bottomrule
\end{tabular}
}
\end{table*}
```
# Analysis {#sec:analysis}
Motion DistributionDistribution Across Motion Binsis not simply selecting ``motion-rich" clips. Our influence scores are computed via gradients, and training videos are considered influential only when they directly improve the model’s ability to generate the target motion dynamics, not because they contain more motion overall.
## Motion Distribution Analysis
**Motive is not simply selecting \`\`motion-rich\" clips:** The key distinction is that our influence scores are computed via gradients, and training videos are considered influential only when they directly allow the model to lower the loss, improving the model's ability to generate the target motion dynamics, not because they contain more motion overall.
To empirically validate this, we further analyze the distribution of motion magnitudes in our selected data. We compute the mean motion magnitude for 10k videos in the **`VIDGEN`** dataset and compare the distributions of the top 10% (highest influence scores) and the bottom 10% (lowest influence scores).
As shown in Fig. `\ref{fig:motion_distribution}`{=latex}, the top 10% selected videos have a mean motion magnitude of 3.85, which is only 4.3% higher than the bottom 10% (3.69), despite representing opposite extremes of influence scores. The analysis also shows that within the moderate-motion range (bins 3, 4, and 5), the top 10% of positive-influence samples outnumber the bottom 10% of negative-influence samples. Yet, both groups also appear in low-motion bins (0-2) and high-motion bins (6-9).
This distribution pattern shows that high-influence videos selected by Motive span the entire motion spectrum, not just high-motion regions. Many high-motion videos receive low influence scores, while numerous influential videos exhibit modest or even low motion magnitude. These findings show that our motion attribution approach captures training influence, focusing on motion rather than simply acting as a motion-saliency filter.
4DNEX Influence HeatmapVIDGEN Influence HeatmapCross-motion influence overlap across datasets. Heatmaps showing the percentage overlap of top-100 influential training samples across motion categories for (a) 4DNEX and (b) VIDGEN datasets. Each cell (i, j) shows the percentage of motion category i’s influential data (aggregated from 5 queries per category) that also appears in motion category j’s top-100 influential samples. The asymmetric nature of the matrices (e.g., bounce→float ≠ float→bounce) arises because different motion categories have different numbers of unique influential videos, leading to directional overlap percentages. Consistent high-overlap pairs (e.g., bounce-float: 44.4%/46.3%) and low-overlap pairs (e.g., free fall-stretch: 12.8%/12.7%) across datasets validate that these influence patterns reflect fundamental aspects of motion representation in video generation models.
## Cross-Motion Influence Patterns
To analyze cross-motion influence patterns, we examine the percentage overlap of top-$100$ influential training data across different motion categories in both the 4DNEX and VIDGEN datasets. As described in §`\ref{sec:experiment}`{=latex}, we use $5$ query samples to identify the top-100 most influential training videos, aggregating results across queries. As shown in Fig. `\ref{fig:cross_motion_overlap}`{=latex}, both datasets exhibit remarkably similar patterns, with mean overlaps of $24.0\%$ and $24.3\%$, respectively, indicating a moderate degree of sharing of influential data across motion categories.
Both datasets consistently identify the same high-overlap pairs: bounce-float ($44.4\%$/$46.3\%$), compress-float ($40.1\%$/$34.0\%$), and compress-spin ($36.9\%$/$39.6\%$), suggesting these motions share fundamental characteristics that the model learns from similar training examples. Conversely, low-overlap pairs such as free fall-stretch ($12.8\%$/$12.7\%$) and float-slide ($14.0\%$/$10.9\%$) indicate more specialized influential data for mechanically dissimilar motions. The influence matrices are asymmetric because the number of unique influential samples shared among the $5$ query samples varies by motion category. The similar cross-motion influence patterns observed across both the 4DNEX and VIDGEN datasets demonstrate that these relationships are generalizable across different video datasets and reflect dynamic similarity.
# Additional Method Details {#app:algorithm}
**Tracker-agnostic scope.** We treat the motion estimator as a pluggable source of saliency rather than a training dependency. Given displacement magnitudes, we construct latent-space weights via bilinear mapping and normalization. Our implementation supports alternative estimators (such as dense optical flow or point tracking) with identical interfaces, enabling users to swap AllTracker without modifying the attribution code.
**Model-agnostic scope.** Our attribution only requires per-example gradients under matched $(t_{\textnormal{fix}},\boldsymbol{\epsilon}_{\textnormal{fix}})$, and therefore applies to both diffusion and flow-matching objectives. The score reduces to a gradient inner product under a fixed preconditioner; the generator architecture affects gradient statistics but not the definition of influence. In practice, replacing the denoiser or velocity field leaves the weighting and aggregation unchanged.
**Algorithm Summary.** For completeness, Algorithm `\ref{alg:motion_attribution}`{=latex} summarizes the full Motive pipeline, detailing the computation of motion-weighted gradients, projection into low-dimensional space, and the subsequent influence-based ranking and selection of training clips.
```{=latex}
\begin{algorithm}[t]\caption{Motive: Motion-Aware Data Attribution Framework}
\label{alg:motion_attribution}
\begin{algorithmic}[1]
\Require fine-tuning corpus $\mathcal{D}_{\textnormal{ft}} = \{(\mathbf{v}_n, \mathbf{c}_n)\}_{n=1}^{N}$, query video $(\hat{\mathbf{v}}, \hat{\mathbf{c}})$, fixed $(t_{\textnormal{fix}}, \boldsymbol{\epsilon}_{\textnormal{fix}})$, projection matrix $\mathbf{P}$
\Ensure Motion-aware influence scores $\{I_{\textnormal{mot}}(\mathbf{v}_n, \hat{\mathbf{v}})\}_{n=1}^{N}$
\For{$(\mathbf{v}_n, \mathbf{c}_n) \in \mathcal{D}_{\textnormal{ft}}$}
\State $A_n \leftarrow \textsc{AllTracker}(\mathbf{v}_n)$ \Comment{Extract per-pixel flow displacements $\mathbf{D}_{\!f}$ (Eq.~\ref{eq:displacement-vector})}
\State Downsample and normalize to latent-space motion mask $\mathbf{W}_n$ (Eqs.~\ref{eq:motion-weight}--\ref{eq:per-location-error})
\State Evaluate motion-weighted loss $\mathcal{L}_{\textnormal{mot}}(\boldsymbol{\theta}; \mathbf{v}_n, \mathbf{c}_n)$ (Eq.~\ref{eq:motion-loss})
\State Compute motion gradient $\mathbf{g}_{\textnormal{mot}}(\boldsymbol{\theta}, \mathbf{v}_n, t_{\textnormal{fix}}, \boldsymbol{\epsilon}_{\textnormal{fix}}) = \nabla_{\boldsymbol{\theta}} \mathcal{L}_{\textnormal{mot}}(\boldsymbol{\theta}; \mathbf{v}_n, \mathbf{c}_n, t_{\textnormal{fix}}, \boldsymbol{\epsilon}_{\textnormal{fix}})$
\State Normalize by frame length: $\mathbf{g}_{\textnormal{mot}} \leftarrow \mathbf{g}_{\textnormal{mot}} /F$
\State Project motion gradient: $\tilde{\mathbf{g}}_{\textnormal{mot}}(\boldsymbol{\theta}, \mathbf{v}_n) := \frac{\mathbf{P}\mathbf{g}_{\textnormal{mot}}(\boldsymbol{\theta}, \mathbf{v}_n, t_{\textnormal{fix}}, \boldsymbol{\epsilon}_{\textnormal{fix}})}{\|\mathbf{P}\mathbf{g}_{\textnormal{mot}}(\boldsymbol{\theta}, \mathbf{v}_n, t_{\textnormal{fix}}, \boldsymbol{\epsilon}_{\textnormal{fix}})\|}$ (Eq.~\ref{eq:diffusion-influence-motion})
\EndFor
\State Compute query gradient: $\tilde{\mathbf{g}}_{\textnormal{mot}}(\boldsymbol{\theta}, \hat{\mathbf{v}})$ using the same procedure for $(\hat{\mathbf{v}}, \hat{\mathbf{c}})$
\For{$n = 1, \dots, N$}
\State $I_{\textnormal{mot}}(\mathbf{v}_n, \hat{\mathbf{v}}) = \tilde{\mathbf{g}}_{\textnormal{mot}}(\boldsymbol{\theta}, \hat{\mathbf{v}})^{\top} \tilde{\mathbf{g}}_{\textnormal{mot}}(\boldsymbol{\theta}, \mathbf{v}_n)$ (Eq.~\ref{eq:diffusion-influence-motion})
\EndFor
\State Rank all training clips by $I_{\textnormal{mot}}(\mathbf{v}_n, \hat{\mathbf{v}})$ and select top-$K$ influential samples using majority vote aggregation (Eq.~\ref{eq:majority-vote}):
\[
\mathcal{S} = \mathcal{S}_{\textnormal{vote}}(K) = \bigl\{\mathbf{v}_n | \mathbf{v}_n \text{ in top-}K \text{ by } \operatorname{MajVote} \bigr\}
\]
\State \Return $\mathcal{S}$
\end{algorithmic}
\end{algorithm}
```
# Additional Experiment Details {#app:exp_details}
## Hyperparameter Settings {#app:hyperparameter}
For reproducibility, we document the hyperparameters used throughout attribution, subset selection, and fine-tuning. Where values were not explicitly tuned, we adopted defaults from DiffSynth-Studio and the official Wan repository for Wan models and from Diffusion-Pipe for LTX models.
**Attribution.** Motion-aware influence estimation is computed at a single fixed timestep, the midpoint of the denoising trajectory, which strongly correlates with multi-timestep averaging. A shared Gaussian draw $\boldsymbol{\epsilon}_{\textnormal{fix}} \sim \mathcal{N}(0,\mathbf{I})$ is used across all training--query pairs to reduce stochastic variance. Gradients are projected from dimension $D = \num{1418996800}$ to $D' = 512$ using a Fastfood Johnson--Lindenstrauss projection $\mathbf{P}$ selected via the search in Fig.`\ref{fig:projection_correlation}`{=latex} to balance performance and storage. Motion weights $\mathbf{W}$ are computed from AllTracker flow magnitudes $M_{\!f}$, min--max normalized to $[0,1]$ with a small bias $\zeta = 10^{-6}$. All computations use bfloat16 precision for memory efficiency.
**Subset Selection & fine-tuning.** For any number of query points, we select top-$10$% data of the datasets. We finetune the backbone while freezing both the text encoder [@raffel2020exploring] and the VAE. The input resolution is fixed to $480 \times 832$ pixels. We use a learning rate of $1 \times 10^{-5}$ and the AdamW optimizer [@loshchilov2017decoupled] following the DiffSynth-Studio defaults. We train the models for $1$ epoch, repeating the dataset $50$ times.
**Evaluation.** The test set consists of the same $10$ motion categories as the query set, but with different visual appearances. We provide the prompt samples below.
```{=latex}
\begin{example}[{\small Samples from Query Set}]\small
We illustrate representative prompts from our query set used to generate query videos with Veo-3.
\begin{itemize}
\item \textbf{compress}: ``A slice of sandwich bread flattened by a flat metal plate, steady camera, soft studio lighting, plain backdrop; emphasize air pockets collapsing.''
\item \textbf{bounce}: ``A ping-pong ball bouncing on a white table, steady side camera, neutral light, seamless backdrop; emphasize consistent bounce height and timing.''
\item \textbf{roll}: ``A spool of thread rolling from left to right, close-up static camera, bright studio light; highlight axle rotation and smooth travel.''
\item \textbf{explode}: ``A single balloon bursting into fragments, captured in high-speed slow motion with a fixed camera, bright even lighting, seamless background; emphasize outward debris and air release.''
\item \textbf{float}: ``A foam cube floating on the surface of water, static overhead camera, bright light, clean tank; emphasize buoyancy and slight rocking.''
\end{itemize}
\end{example}
```
`\label{fig:query-data-samples}`{=latex}
```{=latex}
\begin{example}[{\small Samples from Test Set}]\small
We illustrate representative prompts from our test set that our fine-tuned models use to generate test videos.
\begin{itemize}
\item \textbf{compress}: ``A rubber ball being compressed under a flat press, filmed with a stationary camera. Bright, shadow-free lighting and a clean background emphasize the deformation as it flattens.''
\item \textbf{bounce}: ``A basketball bouncing vertically on a wooden court plank, unmoving camera, balanced indoor lighting, plain wall background; clearly show deformation at impact.''
\item \textbf{roll}: ``A bike tire rolling freely on a stand, static side camera, indoor neutral light; show uniform rotation without wobble.''
\item \textbf{explode}: ``A fragile glass ornament breaking apart mid-air, fixed camera, bright controlled lighting, plain backdrop; capture shards and reflections crisply.''
\item \textbf{float}: ``A green leaf floating gently on perfectly still water in a transparent tank, fixed top-down camera, bright even lighting; emphasize surface tension ripples.''
\end{itemize}
\end{example}
```
`\label{fig:test-data-samples}`{=latex}
## Details on Motion Query Data {#app:motion_query_data}
A small, controlled set of query videos is constructed to isolate specific motion primitives while minimizing confounding factors (e.g., textured backgrounds, uncontrolled camera motion). Such clean and consistent clips are challenging to obtain from natural data sources. To address this, we synthesize the query set using Veo-3 [@veo3] and apply a strict post-generation screening for physical plausibility and generation realism. We target ten motion types: *compress, bounce, roll, explode, float, free fall, slide, spin, stretch, swing*. For each category, we retain $5$ query samples, yielding a total of $50$ queries. This scale provides adequate coverage of the motion taxonomy used in our evaluations while maintaining tractable attribution computation. We further provide a few examples of the generation prompts and the generated video query set in Fig. `\ref{fig:veo3}`{=latex}.
{#fig:veo3 width="\\linewidth"}
**Rationale for synthetic queries.** The query set is not used as training data; instead, it specifies targets for attribution and for multi-query aggregation. Synthetic generation offers controllability that is difficult to achieve at scale with web videos. This design yields near-realistic yet standardized stimuli aligned with our goal of probing motion-specific influence.
# Discussion {#app:discussion}
## Runtime {#app:runtime}
::: {#tab:runtime}
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
**Component** **Complexity** **Runtime** **Notes**
--------------------------- ------------------------------------------------------------------ ---------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gradient computation $\mathcal{O}(B)$ per sample Query: $\sim54$ seconds `\newline `{=latex}Training: $\sim150$ hours $1$ A100 GPU; `\newline `{=latex}Single forward+backward pass; `\newline `{=latex}training is dominant cost but amortized over all queries; `\newline`{=latex} embarrassingly parallel (with $64$ GPUs, $\sim2.3$ hours)
Projection $\mathcal{O}(|\mathcal{D}| \cdot D' \log D')$ $\sim1.97$ seconds per sample $D' = 512$
Influence computation $\mathcal{O}(|\mathcal{D}| \cdot D')$ $\sim46$ milliseconds per query $1$ query $\times$ 10k training samples
Majority-vote aggregation $\mathcal{O}(|\mathcal{D}| \cdot q)$ $\sim139$ milliseconds $50$ queries $\times$ 10k samples
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
: **Runtime Breakdown.** Detailed computational complexity and runtime for each component of our motion attribution framework on 10k training samples with **`Wan2.1-T2V-1.3B`** model.
:::
All training runs are conducted on $4$-$8$ NVIDIA A100 GPUs. We provide a detailed runtime breakdown from our experiments on 10k training samples with **`Wan2.1-T2V-1.3B`** model to address scalability concerns. The key insight is that the dominant cost of our pipeline is computing per-training-sample gradients, which is done once and can then be reused for all subsequent queries. Each training clip's gradient is projected into a compact $512$-dimensional vector, and adding a new query requires only (i) a single backward pass to obtain its own projected $512$-dimensional gradient and (ii) computing cosine similarity between the query vector and stored training vectors, which is exceptionally lightweight (on the order of seconds). Thus, the computational burden does not scale with the number of queries but only with the size of the training set. As shown in Tab. `\ref{tab:runtime}`{=latex}, the dominant cost is the one-time training data gradient computation for 10k samples ($\sim150$ hours on 1 A100), which is amortized across all queries. Once computed, adding a new query requires only $\sim54$ seconds (gradient computation) + $46$ms (influence computation) = $\sim$``{=html}54 seconds total. The training data gradient computation is embarrassingly parallel and can be reduced to $\sim\!\num{2.3}$ hours with $64$ GPUs.
**Runtime comparison with baselines.** We compare the computational cost of our method with the baseline approaches for processing 10k training samples on a single GPU (Table `\ref{tab:runtime-baseline}`{=latex}). While our method requires more upfront computation than the baseline approaches, this cost is amortized across all queries, and the computed gradients can be reused for multiple selection queries, making it practical for large-scale data curation scenarios.
```{=latex}
\resizebox{0.9\linewidth}{!}{
\begin{tabular}{lccccc}
\toprule
Method &
Random &
Motion Magnitude &
Optical Flow &
V-JEPA &
Ours \\
\midrule
Total for 10k ($1$ GPU) &
$<1$ second &
$\sim5.5$ hours &
$\sim5.7$ hours &
$\sim3$ hours &
$\sim150$ hours \\
\bottomrule
\end{tabular}
}
```
`\label{tab:runtime-baseline}`{=latex}
## Limitations
Gradient-based attribution is computationally expensive, requiring a high upfront cost for per-sample gradient computation (see §`\ref{app:runtime}`{=latex}), though this cost is amortized across queries. Our analysis treats each video as a whole unit, thereby avoiding collapsing motion into frame-level appearance, but it risks overlooking the fact that only certain segments may carry motion-relevant information. Highly informative intervals can be diluted when averaged with static or redundant portions of the same clip. This suggests an open direction toward finer-grained attribution at the motion-segment or motion-event level, which could yield more precise insights into how different phases of a trajectory shape motion learning. Another limitation is that our motion masks may overemphasize camera-only motion; we detect this by spatial uniformity of $\mathbf{W}$ and down-weight such clips, but a full disentanglement of ego and object motion remains future work.
Additionally, our framework does not explicitly account for classifier-free guidance (CFG), which is widely used in practice to steer video generation but introduces discrepancies between training-time attribution and inference-time dynamics. As a result, our influence estimates may not fully capture how guidance alters motion behavior. In addition, while attribution-driven fine-tuning improves targeted motion quality, it may introduce trade-offs in the base model's capabilities. This underscores the need for future work to balance targeted motion adaptation with the preservation of broader generative capabilities.
## Future Directions
**Improving the attribution pipeline.** Several directions can strengthen our motion attribution methodology. *Tracker-robust motion saliency*: replace or ensemble AllTracker with alternative estimators and use its confidence/visibility channels to weight masks. *Self-generated video queries*: use model-generated videos as queries to trace problematic motion patterns (e.g., unrealistic physics) back to training data, enabling iterative diagnostics and targeted motion improvement.
**Scaling data curation.** *Closed-loop data curation*: move from one-shot ranking to active selection: iteratively attribute, finetune, and re-attribute, or replace simple majority voting with learned query weights. *Sophisticated finetuning*: move to more sophisticated finetuning setups, such as multi-student distillation [@song2024multi].
**Extending to new domains.** *Other modalities*: extend our methodology to other modalities, including world models [@zhu2025astra], audio [@evans2025stable; @richter2025score], or video+audio [@wiedemer2025video].
**Safety and governance.** Use negative-influence filtering to suppress undesirable or unsafe dynamics, document curator choices, and audit motion behaviors that our framework exposes.
# Visualization {#app:visualization}
## Motion Visualization {#app:motion-visualization}
To provide intuition for the behavior of our motion-weighted loss, we visualize the motion magnitude as an overlay. We compute per-pixel motion magnitude using optical flow and apply motion-based weighting that preserves the appearance of dynamic regions while attenuating static backgrounds. This motion overlay directly illustrates the spatial weighting applied by our motion loss during training.
Fig. `\ref{fig:motion_examples}`{=latex} presents representative frames from our dataset, comparing original frames with their corresponding motion overlays for seven distinct video samples. These visualizations show that the motion-weighted loss preferentially emphasizes dynamic content while down-weighting static scene elements.
```{=latex}
\begin{figure*}[t!]
\includegraphics[width=\linewidth]{fig/fig6.png}
\caption{\textbf{Motion Overlay Visualization.} Comparison of original frames and motion overlays for seven video samples across three time points (early, middle, late). The motion overlay demonstrates the spatial weighting of our motion loss: dynamic regions remain visible, while static backgrounds are attenuated to neutral gray. \emph{Takeaway:} This provides heuristic intuition into what information our motion attribution focuses on: the information in grayer regions, which lack motion, is down-weighted by our method.}
\label{fig:motion_examples}
\end{figure*}
```
