NaviVLN
TEASER · REAL-WORLD EXECUTION
ICLR 2027 · Under Review

NaviVLN: Task-Centric Hierarchical Vision-Language Navigation

A dual-system policy that aligns vision, language, and trajectories through task-aligned data grounding, cross-modal latent alignment, and staged training.

Anonymous authors

Paper under double-blind review · ICLR 2027 Submission 4022

Under review as a conference paper at ICLR 2027

01 / ABSTRACT

Abstract

Vision-Language Navigation (VLN) integrates spatial perception, instruction understanding, and sequential decision-making. Foundation-model-based policies benefit from multimodal supervision, yet insufficient task relevance and cross-modal correspondence can cause task-agnostic intra-modal redundancy and cross-modal semantic misalignment, limiting sample efficiency and decision consistency.

We present NaviVLN, a dual-system policy built on THREAD (Task-centric Hierarchical Representations via Embodied Alignment for Dual-system), aligning vision, language, and trajectories through data, model, and training. THREAD fuses navigation information into mutually informative visual and textual targets grounding instruction semantics and task progress. Joint Attention integrates implicit visual and textual chain-of-thought during training, shaping shared representations through auxiliary and navigation supervision. Staged training connects these representations to a local trajectory expert; deployment discards auxiliary decoders, using asynchronous intention querying for efficient execution.

On R2R-CE and RxR-CE Val-Unseen, NaviVLN with Joint Attention achieves success rates of 68.3% and 63.7%. Navi-Dataset contains approximately 4.0M effective trajectory samples, of which 0.8M are alignment-annotated decision anchors. On R2R-CE, NaviVLN is 1.4 percentage points above monocular Qwen-RobotNav-4B using about 25.6% as many samples.

68.3%
R2R-CE Val-Unseen SR
63.7%
RxR-CE Val-Unseen SR
4.0M
Effective trajectory samples
25.6%
Samples vs. Qwen-RobotNav-4B
02 / FRAMEWORK

How it works

NaviVLN couples task-aligned data grounding with cross-modal latent alignment and a hierarchical trajectory expert.

01 · OVERVIEW
Start with the big picture: how NaviVLN aligns vision, language, and trajectories in one dual-system policy Figure 1 is the global view, from task-aligned data through shared representations to trajectory generation. The three cards below break out its key ingredients.
FIG 1System overviewdata → shared representations → trajectories
NaviVLN framework: multi-task dataset, model overview, task-aligned embodied reasoning
Figure 1. Overview of NaviVLN and THREAD. Top left: multi-task data examples. Top right: semantic and progress predictors in the model architecture, coupled through Joint Attention, shape shared representations for trajectory generation. Bottom: task-aligned embodied reasoning relates instruction context, scene evidence, task progress, and intended motion.
01 · DATA GROUNDING

Learn task-aligned targets

Progress-aware descriptions and projected expert paths form mutually informative visual and textual targets, grounding instruction semantics and task progress in the scene.

02 · JOINT ATTENTION

Align the latent streams

Implicit visual and textual chain-of-thought are integrated during training; auxiliary decoders shape shared representations that are discarded at deployment.

03 · TRAJECTORY EXPERT

Execute asynchronously

Staged training connects the representations to a local DiT action expert; slow VLM intention queries and fast closed-loop trajectory updates run asynchronously.

02 · DATA
Then the data: where task-aligned supervision comes from Figure 2 shows the composition of Navi-Dataset and its annotation pipeline. The two cards that follow explain how the semantic and progress branches write task relevance into the supervision targets.
FIG 2Navi-Dataset & task-aligned annotation4.0M effective samples · 0.8M decision anchors
Navi-Dataset composition and task-aligned annotation pipeline
Figure 2. Navi-Dataset and its task-aligned annotation pipeline. The top panels show sample composition and scene diversity; the dataset contains approximately 4.0M effective trajectory samples, of which 0.8M are decision anchors with both visual and textual CoT annotations (20\% of all samples), a subset of the same pool. The bottom panels show how trajectories are segmented into subtasks and how progress-aware descriptions guide semantic targets that retain relevant landmarks and the projected expert path.
TSR · TASK-ALIGNED SEMANTIC REASONING

Where can the agent go?

Two-step semantic predictions identify the traversable path region and goal-related scene evidence. Projected expert paths supervise the visual branch, so predicted semantics carry task relevance rather than generic scene structure.

PTR · PROGRESS-AWARE TEXTUAL REASONING

Where is the task now?

Progress descriptions supervise the textual branch with the current stage and next subgoal. Joint Attention couples semantic and progress latents through a stop-gradient navigation reference from the VLM, aligning both branches to the same local decision.

03 · MODEL
Finally, the implementation: architecture and staged training Figure 3 shows the model and training flow: auxiliary predictors shape shared representations via Joint Attention during training, then are removed at deployment, leaving only the asynchronously queried trajectory expert.
FIG 3Architecture & staged trainingjoint attention → DiT expert → async deployment
NaviVLN architecture and training framework
Figure 3. NaviVLN architecture and training framework. The VLM forms semantic, progress, and trajectory-related hidden states; visual and textual auxiliary predictors are coupled through Joint Attention with trajectory hidden states as a shared reference. Staged training learns a DiT action expert conditioned on latent intention queries. At deployment, auxiliary decoders are removed and asynchronous querying links slower VLM reasoning with faster local trajectory generation.
03 / RESULTS

Main results

Tables below follow the current paper: unseen-scene navigation (Table 1), predictor organization (Table 2), training-stage and DiT variants (Table 4), and real-world instruction following (Table 5). NE is in meters; OSR, SR, SPL, and nDTW are on a percentage scale. Training-data counts use each method's own units and are not interchangeable.

Table 1. Navigation results on R2R-CE and RxR-CE Val-Unseen. Bold marks the best value in each metric column. NaviVLN (Joint) uses monocular RGB and ~4.0M effective trajectory samples.
MethodBackboneInputsTrain data R2R-CE Val-UnseenRxR-CE Val-Unseen
NE ↓OSR ↑SR ↑SPL ↑SR ↑SPL ↑nDTW ↑
ETPNavGraph TransformerPano., Odo., Depth—4.7165.057.049.054.844.961.9
ScaleVLNHAMTPano., Odo., Depth4.9M4.80—55.051.0———
NavFoMQwen2-7BFour-view RGB12.7M4.6172.161.755.364.456.265.8
ABot-N0Qwen3-4BPano.16.9M3.7870.866.463.969.360.0—
NaVidLLaMA-VID-7BMono RGB1.28M5.4749.137.435.9———
Uni-NaVidVicuna-7BMono RGB3.58M5.5853.347.042.748.740.9—
NaVILAVILA (Llama 3-8B)Mono RGB~13.1M5.2262.554.049.049.344.058.8
StreamVLNLLaVA-Video-7BMono RGB~26.3M4.9864.256.951.952.946.061.9
InternVLA-N1Qwen2.5-VL-7BDepth, Mono RGB>5M4.8363.358.254.053.546.1—
DualVLNQwen2.5-VL-7BMono RGB~12M4.0570.764.358.561.451.870.0
FutureNav-4BQwen3-VL-4BMono RGB10.46M*4.2470.265.461.360.952.868.0
Qwen-RobotNav-4BQwen3-VL-4BMono RGB15.6M4.2273.666.960.571.361.568.6
NaviVLN (Joint)Qwen3-VL-4BMono RGB~4.0M3.1476.1868.363.5563.755.5365.23

Table 1 notes. Pano. = panoramic observations; Odo. = odometry. *FutureNav reports 10.457M external samples plus approximately 2.43M benchmark training samples. A dash marks an entry not reported by the respective method. For NaviVLN, the 0.8M alignment-annotated decision anchors are contained within the same 4.0M effective trajectory samples.

Under matched monocular-RGB input, NaviVLN (Joint) reaches 68.3% SR on R2R-CE, +1.4 pp over Qwen-RobotNav-4B while using about 25.6% of its training samples.
Table 2. Predictor organization and cross-branch coupling on R2R-CE and RxR-CE Val-Unseen. NE is in meters; OSR, SR, SPL, and nDTW are on a percentage scale. The full model is listed last.
Configuration R2R-CE Val-UnseenRxR-CE Val-Unseen
NE ↓OSR ↑SR ↑SPL ↑SR ↑SPL ↑nDTW ↑
Separate3.2774.6264.258.9261.253.4762.46
Merged3.4273.8562.557.9059.452.2261.78
Joint w/o cross-branch K/V3.5272.1763.258.8161.653.2564.26
Joint w/o navigation reference3.7171.7661.857.6558.250.4260.35
Joint w/ cross-branch gradients3.2175.3267.562.8762.754.5164.17
Joint + alignment loss3.4473.2266.361.6263.454.9564.86
Joint (full model)3.1476.1868.363.5563.755.5365.23

Removing cross-branch K/V excludes the other branch's latent states from attention; the variant without the navigation reference removes the shared trajectory reference from both branches. The alignment-loss variant adds an explicit cross-branch objective but does not improve on the full model.

Table 4. Training-stage and DiT action-expert variants on Val-Unseen navigation. SR and SPL are percentages. Rows are ordered by R2R-CE SR with the full Joint model last.
ConfigurationTraining stages R2R-CE Val-UnseenRxR-CE Val-Unseen
SR ↑SPL ↑SR ↑SPL ↑
Stage 0 + DiT0 → 356.949.652.739.8
Skip Stage 20 → 157.951.2154.641.63
Stage 0 + explicit CoT060.354.8856.144.60
Skip Stage 10 → 262.755.4358.647.75
Joint + DiT0 → 1 → 2 → 367.161.8261.854.17
Full Joint0 → 1 → 268.363.5563.755.53

With explicit CoT, the model generates reasoning text before the navigation answer. The DiT action expert is introduced at Stage 3; adding it to Stage 0 or to the full Joint representation does not improve on the corresponding non-DiT configuration, and the full Joint schedule remains the strongest.

Table 5. Real-world instruction following across five cases. Each configuration is evaluated over 75 trials (15 per case); total successes and success rates are reported.
ConfigurationVisual supervisionTotal trialsTotal successesSR (%) ↑
RGB0.8MFuture RGB752634.7
RGB16MFuture RGB753850.7
Sem-MergedTask-aligned semantics754053.3
Sem-SeparateTask-aligned semantics754661.3
Sem-JointTask-aligned semantics755269.3

A trial succeeds when the robot completes the instruction, including any requested stop or door crossing. Sem-Joint reaches 69.3% success, outperforming the strongest monocular-RGB baseline by 18.6 percentage points.

04 / VIDEOS

Qualitative results

Featured real-world execution followed by five-view synchronized simulation episodes. All clips use the same NaviVLN (Joint) configuration.

Simulation

“Go into the dining room. Go to the China cabinet.”

Simulation

“Walk out onto the patio and take a left into the bedroom. Walk through the bedroom and onto the far patio. Stop on the patio once you are past the door.”

Simulation

“Start in the Entryway and navigate to the Bedroom via a slight left turn and short walk, then proceed toward the Sofa with a right turn and forward movement. From the Sofa, make a slight right turn to reach the Dining Table, then return to the Entryway by turning right twice with slight adjustments. Upon reaching the Entryway again, move back into the Bedroom by turning slightly left and advancing, then traverse through the bedroom into the hallway with a long forward motion. From the hallway, navigate to the Bathroom by turning slightly left, moving forward, and turning left, then head back to the Bedroom with a left turn. Finally, within the Bedroom, execute a series of turns and forward movements to arrive at the chair.”

Simulation

“Walk left passing the fireplace and head out, turn left, passing another fireplace and make a sharp left at the bathroom and stop there.”

05 / CITATION

Citation

If you find this work useful, please cite it. Author information will be added in the camera-ready version.

@article{navivln2027,
  title   = {NaviVLN: Task-Centric Hierarchical Vision-Language Navigation},
  author  = {Anonymous Authors},
  journal = {Under review as a conference paper at ICLR 2027},
  year    = {2027}
}