Lingbot Map

<div align="center"> <img src="assets/teaser.webp" width="100%"> <h1>LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstruction</h1>

Robbyant Team

</div> <div align="center">

</div>

https://github.com/user-attachments/assets/fe39e095-af2c-4ec9-b68d-a8ba97e505ab

---

🗺️ Meet LingBot-Map! We've built a feed-forward 3D foundation model for streaming 3D reconstruction! 🏗️🌍

LingBot-Map has focused on:

• Geometric Context Transformer: Architecturally unifies coordinate grounding, dense geometric cues, and long-range drift correction within a single streaming framework through anchor context, pose-reference window, and trajectory memory.
• High-Efficiency Streaming Inference: A feed-forward architecture with paged KV cache attention, enabling stable inference at ~20 FPS on 518×378 resolution over long sequences exceeding 10,000 frames.
• State-of-the-Art Reconstruction: Superior performance on diverse benchmarks compared to both existing streaming and iterative optimization-based approaches.

---

📑 Table of Contents

<details> <summary>Click to expand</summary>

• 📰 News
• 📋 TODO
• ⚙️ Installation
• 📦 Model Download
• 🚀 Quick Start
• 🎬 Interactive Demo (demo.py)
  • Try the Example Scenes
  • Streaming with Keyframe Interval
  • Windowed Inference (for long sequences, >3000 frames)
  • Sky Masking
  • Visualization Options
  • Performance & Memory
• 🎥 Offline Rendering Pipeline (demo_render/batch_demo.py)
• 📜 License
• 📖 Citation
• ✨ Acknowledgments

</details>

---

📰 News

• 2026-06-28 — Fixed an SDPA KV cache bug. The SDPA backend now performs better on long sequences. We still recommend the FlashInfer backend for the best performance.
• 2026-05-25 — 📊 Evaluation benchmark released. We released the evaluation scripts for KITTI and Oxford Spires — see benchmark/ for the pipeline, and run preprocess/oxford.py to prepare Oxford Spires data before evaluation.
• 2026-04-29 — 📹 Long-video demo released. We released a very-long-video example (~25 000 frames, 13-minute indoor walkthrough) rendered with the offline pipeline — see Worked Example for the command, flag rationale, and rendered output.
• 2026-04-27 — 🚀 LingBot-Map accelerated. Pull the latest main and run python demo.py --compile ... or python gct_profile.py --backend flashinfer --dtype bf16 --compile to verify on your hardware.
• 2026-04-24 — Fixed a FlashInfer KV cache bug where --keyframe_interval > 1 silently cached non-keyframes. You should now see better pose and reconstruction quality when running with more than 320 frames.

---

📋 TODO

• ✅ Release evaluation benchmark
  • ✅ Oxford Spires dataset
  • ✅ KITTI dataset
  • ✅ VBR dataset
  • ✅ Droid-W dataset
  • ✅ TUM-D dataset
  • ✅ 7-scenes dataset
  • ✅ ETH3D dataset
  • ✅ Tanks and Temples dataset
  • ✅ NRGBD dataset
• ✅ Release demo scripts
  • ✅ Indoor long-video demo (Featured indoor walkthrough)
  • ✅ Outdoor long-video demo
  • ✅ LingBot-World demo (Worked example)
  • ✅ Aerial long-video demo

---

⚙️ Installation

1. Create conda environment

conda create -n lingbot-map python=3.10 -y
conda activate lingbot-map

2. Install PyTorch (CUDA 12.8)

pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128

PyTorch 2.8.0 is the recommended version because NVIDIA Kaolin (required by the batch rendering pipeline) has prebuilt wheels for torch-2.8.0_cu128. If you only need demo.py you may use a newer PyTorch, but the batch renderer then requires building Kaolin from source. For other CUDA versions, see PyTorch Get Started.

3. Install lingbot-map

pip install -e .

4. Install FlashInfer (recommended)

FlashInfer provides paged KV cache attention for efficient streaming inference. It is a pure-Python package that JIT-compiles CUDA kernels on first use, so a single wheel works across CUDA/PyTorch versions:

pip install --index-url https://pypi.org/simple flashinfer-python

--index-url https://pypi.org/simple is only needed if your default pip index is an internal mirror that doesn't have flashinfer-python. (Optional) For faster first-use, you can additionally install a CUDA-specific JIT cache: pip install flashinfer-jit-cache -f https://flashinfer.ai/whl/cu128/flashinfer-jit-cache/. See FlashInfer installation for details. If FlashInfer is not installed, the model falls back to SDPA (PyTorch native attention) via --use_sdpa.

5. Visualization dependencies (optional)

pip install -e ".[vis]"

📦 Model Download

| Model Name | Huggingface Repository | ModelScope Repository | Description | | :--- | :--- | :--- | :--- | | lingbot-map-long | robbyant/lingbot-map | Robbyant/lingbot-map | Better suited for long sequences and large scale scenes. | | lingbot-map | robbyant/lingbot-map | Robbyant/lingbot-map | Balanced checkpoint (used in paper, benchmark and offline demo) — trade off all-around performance across short and long sequences. | | lingbot-map-stage1 | robbyant/lingbot-map | Robbyant/lingbot-map | Stage-1 training checkpoint of lingbot-map — can be loaded into the VGGT model for bidirectional inference (c2w). |

🚧 Coming soon: we're training an stronger model that supports longer sequences — stay tuned.

🚀 Quick Start

After installation, run your first scene with one command:

python demo.py --model_path /path/to/lingbot-map.pt \
    --image_folder example/courthouse --mask_sky

This launches an interactive viser viewer at http://localhost:8080. See Interactive Demo below for the full set of scenes and flags, or jump to Offline Rendering Pipeline for long-sequence batch rendering.

🎬 Interactive Demo (demo.py)

Run demo.py for interactive 3D visualization via a browser-based viser viewer (default http://localhost:8080).

Try the Example Scenes

We provide three example scenes in example/ that you can run out of the box:

# courthouse scene
python demo.py --model_path /path/to/lingbot-map.pt \
    --image_folder example/courthouse --mask_sky

https://github.com/user-attachments/assets/aa10f7ab-8024-43c7-92f8-d56159ec85c8

# University scene
python demo.py --model_path /path/to/lingbot-map.pt \
    --image_folder example/university --mask_sky

https://github.com/user-attachments/assets/212a1744-6ff5-4ccf-9bd4-728608248b57

# Loop scene (loop closure trajectory)
python demo.py --model_path /path/to/lingbot-map.pt \
    --image_folder example/loop

https://github.com/user-attachments/assets/5ae0a292-b081-40c6-838c-b7c1a0538d75

🎯 Featured: indoor walkthrough (~25 000 frames, 13 minutes)

Sequence is too long for the interactive viser viewer — this clip was rendered with the Offline Rendering Pipeline. See that section for the full command.

We will provide more examples in the follow-up.

Dynamic Demo (From Droid-W)

Dataset: Download the demo sequences from robbyant/lingbot-map-demo on Hugging Face.

Example run on the dynamic sequence from the dataset above (sky masking on, 4 camera optimization iterations, keyframe every 2 frames):

Run the dynamic sequence with sky masking, 4 camera optimization iterations, and an input stride of 2:

python demo.py \
    --image_folder /path/to/dynamic\
    --model_path ../../Lingbot-Map/lingbot-map.pt \
    --camera_num_iterations 4 \
    --mask_sky \
    --stride 2

https://github.com/user-attachments/assets/567b6e9b-1cbf-402a-96be-9bab70715ec3

<img width="1453" height="1195" alt="image" src="https://github.com/user-attachments/assets/27f8c6b7-339e-4e5f-9776-7cb577147401" />

Streaming with Keyframe Interval

Use --keyframe_interval to reduce KV cache memory by only keeping every N-th frame as a keyframe. Non-keyframe frames still p