CalibWM: Action-Fidelity Calibrated Uncertainty for Frozen Video World Models

摘要

Action-conditioned video world models are increasingly used as interactive simulators for robot policy evaluation and imagination-based planning, yet they are trained and measured almost entirely on visual fidelity. Recent benchmarks show that this is a poor proxy for action fidelity: a rollout can look photorealistic while silently corrupting the causal consequence of the executed action, and models systematically imagine failures as successes. Uncertainty estimators built for these models inherit the misalignment, because they calibrate the probability that a pixel is visually correct, which is not the quantity a downstream controller needs. We introduce CalibWM, a lightweight probe on a frozen action-conditioned video world model that emits a spatiotemporal uncertainty map and calibrates it against action-fidelity failure: whether a frozen inverse dynamics model can still recover the executed action from the generated frames. A horizon-calibrated quantile objective makes the map grow with the rollout horizon, a spatial term keeps it localized, and an alignment term anchors the deployed readout to that event; a per-horizon, segment-level split conformal layer then supplies finite-sample coverage whose per-unit meaning is fixed by the spatial aggregator. On Bridge and DROID, CalibWM reaches 0.782 AUROC against action-fidelity failure where the strongest baseline that never sees the label reaches 0.694, and it holds a +0.020 margin (clustered 95% CI [0.009,0.031]) over the strongest control that does see it. On a human-annotated optimism-bias subset, where visual quality is uniformly high and failures are rare, it reaches 0.634 AUPRC against a chance floor of 0.09, while an oracle visual proxy handed the ground-truth future reaches 0.106. We prove coverage at a fixed horizon and, under a stated sufficiency condition, AUROC dominance over any visual proxy computable from the generated rollout alone, and we say in both cases what is left uncovered.