Under Review · 2026

SPARK-VLN

Streaming a VLM Reasoner into a Fast Expert Planner for
Dynamic Social Vision-Language Navigation

Tianshuai Hu¹, Yangyi Zhong², Zeying Gong², Lingdong Kong³, XiaoDong Mei¹, Guoyang Zhao², Rong Li², Junwei Liang^1,2*

¹ HKUST · ² HKUST (Guangzhou) · ³ National University of Singapore * Corresponding author

◆ Paper ✦ arXiv ⌥ Code

Comparison of reason-then-act, blocking dual-system, and token-streaming dual-system

Prior reason-then-act policies (a) stall while the VLM thinks, and naive dual systems (b) block on each full inference — both leave the planner acting on stale observations. SPARK-VLN (c) streams the reasoner's evolving hidden states to the fast planner token-by-token, keeping action instant and current.

Abstract

Vision-Language-Navigation in dynamic, human-centric environments exposes a fundamental tension: linguistic reasoning is slow and deliberative, whereas safe, socially compliant planning must be instant and reactive. The resulting observation staleness is safety-critical — a maneuver chosen during inference can already be unsafe by the time it executes. We observe that, long before a VLM finishes its inference, its intermediate hidden states already encode action-relevant intent. We propose SPARK-VLN, a dual-system framework in which a slow VLM reasoner streams its knowledge to a fast flow-matching expert planner token-by-token. We also introduce a human-centric benchmark suite for dynamic social VLN that keeps pedestrians and the robot active throughout inference. Across these settings, SPARK-VLN improves navigation success and social compliance while sustaining inference efficiency.

The Tension

Linguistic reasoning is slow & deliberative; socially compliant planning must be instant & reactive. Reconciling the two is the core challenge of social VLN.

The Insight

Long before a VLM finishes inference, its intermediate hidden states already encode action-relevant intent — usable the moment they appear.

Streaming the reasoner, token by token

Three modules carry the VLM reasoner's evolving knowledge into the flow-matching expert planner without waiting for inference to complete.

SPARK-VLN framework: slow VLM reasoner, sequence-to-slot latent bridge, and fast planner with evolving condition injector

The slow reasoner streams token-wise hidden states into the latent bridge, which projects them into slot guidance for the fast flow-matching planner — producing candidate trajectories and critic scores at reactive speed.

A human-centric benchmark suite

Unlike static protocols, our suite keeps pedestrians and the robot active throughout inference, systematically varying three axes to expose the observation staleness that conventional benchmarks hide.

Benchmark axes: Scene (outdoor, hall, corridor), Encounter (intersection, following, corner), Mode (static, synchronous, asynchronous)

BibTeX

@article{hu2026sparkvln,
  title   = {SPARK-VLN: Dynamic Social Vision-Language Navigation},
  author  = {Hu, Tianshuai and Zhong, Yangyi and Gong, Zeying and Kong, Lingdong
             and Mei, XiaoDong and Zhao, Guoyang and Li, Rong and Liang, Junwei},
  journal = {Under Review},
  year    = {2026}
}