Act with Intent

Abstract

Vision-Language-Action (VLA) models can turn multimodal context into robot actions, but their action decoders are still trained largely by behavior cloning. This supervises which motor command was demonstrated while leaving implicit the local objective served by the behavior under the instruction. Future-based supervision enriches action learning with frames, latent observations, trajectories, or motion representations, but these signals capture particular realizations of what may happen rather than the shared semantic objective of the forthcoming behavior. We propose Intention Distillation (INDI), which distills behavior-level intent into the action decoder. During training, a frozen teacher VLM interprets a demonstrated segment from the current observation, instruction, coarse action summary, and corresponding execution video. From its standard inputs, the deployed VLA recovers the resulting multimodal intent representation at an intermediate decoder layer and uses it to organize action prediction together with representations of how the behavior unfolds and what it achieves. On SimplerEnv-Bridge, INDI improves GR00T-N1.7 from 64.3% to 84.7%, and on RoboCasa Kitchen it improves the controlled GR00T-N1.7 baseline from 64.1% to 70.3%, with consistent gains on π_0.5 across both benchmarks. In real-world tasks, INDI improves average success from 62.0% to 68.7%, with gains of up to 12.0 pp on longer-horizon tasks. Further analyses show that the recovered latent is used by the decoder, captures behavior objective and execution progress, and organizes downstream predictions in an objective-dependent manner. These results show that action decoders benefit from explicitly modeling the semantic objective of the behavior they generate.

Method

INDI defines intent as the local objective that the forthcoming behavior should achieve under the instruction, and makes it an explicit supervision target for the action decoder. It has three components:

1. Intent from executed behavior. A frozen teacher VLM reads the current observation, instruction, a coarse action summary, and the execution video of a demonstrated segment, and states the functional purpose the segment serves. Its hidden states from this interpretation become the intent target, complemented by a textual-purpose grounding and an endpoint visual feature.
2. Intent recovery inside the decoder. Learnable intent queries must reconstruct the teacher’s intent representation at an intermediate decoder layer, from standard VLA inputs only. The remaining layers then complete action, visual, and textual grounding predictions on top of the recovered intent.
3. Intent-aware decoding. Asymmetric attention and an intent-mismatch margin make the recovered intent a functional intermediate state that downstream prediction genuinely depends on, rather than an auxiliary readout that could be ignored.

Results

We evaluate on two backbones (GR00T-N1.7 and π_0.5), two simulation benchmarks, and four real-world tasks. Every comparison uses the same demonstrations, optimization budget, and evaluation protocol as its baseline. The only change is behavior-intent supervision.

SimplerEnv-Bridge

Success rate (%), mean over three evaluation runs. INDI improves every task on both backbones. Future-based supervision reaches 68.0% where INDI reaches 84.7%, so the gain comes from behavior-level intent supervision rather than extra signal alone. On EP-Basket, success rises from 36.0% to 96.0%.

RoboCasa Kitchen (24 tasks)

With 100 demonstrations per task, GR00T-N1.7 + INDI reaches 70.3%, within 0.5 pp of the reported GR00T-N1.7 checkpoint trained on 3,000 demonstrations per task (70.8%), and the highest average among reported RoboCasa results.

Real-world tasks

Success rate (%) over 50 trials, GR00T-N1.7 backbone. The gains concentrate on the longer-horizon tasks (Cross-Bin Stacking +7.3 pp, Drawer Storage +10.7 pp averaged across conditions) and persist as the scene shifts from training conditions: +6.0 pp on held-out objects and +9.0 pp under distractors.

BibTeX

@misc{lee2026actintentdistillingbehavior,
      title={Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models},
      author={Sangoh Lee and Sangwoo Mo and Wook-Shin Han},
      year={2026},
      eprint={2608.23478},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2608.23478},
}