IROS 2026 · Social Navigation

NavThinker

Action-Conditioned World Models for Coupled Prediction
and Planning in Social Navigation

Tianshuai Hu¹, Zeying Gong², Lingdong Kong³, XiaoDong Mei¹, Yiyi Ding², Qi Zeng², Ao Liang⁴, Rong Li², Yangyi Zhong², Junwei Liang^1,2*

¹ HKUST · ² HKUST (Guangzhou) · ³ NUS · ⁴ UCAS * Corresponding author

✦ arXiv ◆ PDF ▶ Video ⌥ Code

NavThinker: future-aware social navigation teaser

Future-aware social navigation. Red regions mark interaction zones. Top: egocentric depth. Bottom: the world model imagines action-conditioned futures — letting the robot anticipate conflicts and choose socially compliant actions.

Abstract

Social navigation requires robots to act safely in dynamic human environments. Effective behavior demands thinking ahead — reasoning about how the scene and pedestrians evolve under different robot actions rather than reacting to current observations alone, a coupled prediction-planning challenge. We propose NavThinker, a future-aware framework that couples an action-conditioned world model with on-policy reinforcement learning. The world model operates in Depth Anything V2 patch-feature space and autoregressively predicts future scene geometry and human motion; multi-head decoders produce future depth maps and human trajectories. We train the policy with DD-PPO while injecting think-ahead signals via action-conditioned future features and social reward shaping. Experiments on Social-HM3D show state-of-the-art success, with zero-shot transfer to Social-MP3D and real-world deployment on a Unitree Go2.

The Challenge

Robot actions and human motion mutually influence each other. Reacting to current observations alone is unsafe — planning must reason about how the scene will evolve.

The Idea

Imagine, then act. A world model rolls out action-conditioned futures, and the policy plans against them — coupling prediction and planning in one loop.

Framework overview

A world model that learns action-conditioned scene dynamics, and an imagination-augmented DD-PPO planner.

World Model Learning

A frozen DA-V2 ViT encodes depth into patch embeddings; a causal Transformer autoregressively predicts future latents. Depth and human-trajectory decoders, with a latent-consistency loss, anchor the representation to geometry and motion.

Policy Learning

The imagination module queries the world model under each candidate action; fused future features feed a DD-PPO actor-critic. Predicted human trajectories provide reward shaping — an imagine-then-act loop.

Results

Single-robot navigation on Social-HM3D and zero-shot transfer to Social-MP3D. SR / SPL ↑, PSC (social compliance) ↑, H-Coll ↓ — best, 2nd-best.

Method	Social-HM3D				Social-MP3D (zero-shot)
Method	SR↑	SPL↑	PSC↑	H-Coll↓	SR↑	SPL↑	PSC↑	H-Coll↓
{{ r.m }}	{{ r.a }}	{{ r.b }}	{{ r.c }}	{{ r.d }}	{{ r.e }}	{{ r.f }}	{{ r.g }}	{{ r.h }}

Multi-Robot Social Navigation

On Social-HM3D — multiple robots reach individual goals without communication. Team-level T-SR / T-SPL ↑.

Method	SR↑	SPL↑	PSC↑	H-Coll↓	T-SR↑	T-SPL↑
{{ r.m }}	{{ r.a }}	{{ r.b }}	{{ r.c }}	{{ r.d }}	{{ r.e }}	{{ r.f }}

59.5

Social-HM3D SR — SOTA

Zero-shot

Transfer to Social-MP3D

Go2

Real-world Unitree deployment

Video

BibTeX

@article{hu2026navthinker,
  title   = {NavThinker: Action-Conditioned World Models for Coupled
             Prediction and Planning in Social Navigation},
  author  = {Hu, Tianshuai and Gong, Zeying and Kong, Lingdong and Mei, XiaoDong
             and Ding, Yiyi and Zeng, Qi and Liang, Ao and Li, Rong
             and Zhong, Yangyi and Liang, Junwei},
  journal = {arXiv preprint arXiv:2603.15359},
  year    = {2026}
}