X VLA

[ICLR 2026] The offical Implementation of "Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model"

Generate Convert Improve

Install / Use

npx skills add 2toinf/X-VLA

Installs into whichever agent you are using.

About this skill

Quality Score

0/100

README

🤖 X-VLA: Soft-Prompted Transformer as a Scalable Cross-Embodiment Vision-Language-Action Model

🏆 Highlights & News

🎉 Exciting News: X-VLA Accepted to ICLR 2026

We are thrilled to announce that X-VLA has been accepted to ICLR 2026.

🚀 Now Supported in LeRobot

X-VLA is now natively integrated into the LeRobot platform.
Give it a try! We sincerely appreciate the support and collaboration from the Hugging Face team.

🥇 Champion Winner at IROS 2025

X-VLA won 1st Place (Champion) at the AgiBot World Challenge, held at IROS 2025.

🧩 Overview

Successful generalist Vision–Language–Action (VLA) models depend on scalable, cross-platform training across diverse robotic embodiments.
To leverage the heterogeneity of large-scale robot datasets, X-VLA introduces a soft prompt mechanism — embodiment-specific learnable embeddings that guide a unified Transformer backbone toward effective multi-domain policy learning.

The resulting architecture — X-VLA-0.9B — achieves state-of-the-art generalization across six simulation platforms and three real-world robots, surpassing prior VLA approaches in dexterity, adaptability, and efficiency.

https://github.com/user-attachments/assets/c047bac4-17c3-4d66-8036-badfab2b8c41

🚀 Quick Start: Installation & Deployment

1️⃣ Installation

# Clone the repository
git clone https://github.com/2toinf/X-VLA.git
cd X-VLA

# Create and activate Conda environment
conda create -n XVLA python=3.10 -y
conda activate XVLA

# Install dependencies
pip install -r requirements.txt

conda env create -f environment.yml
conda activate xvla-stable

2️⃣ Deploying X-VLA for Inference

X-VLA adopts a Server–Client architecture to separate the model environment from simulation or robot-specific dependencies. This design avoids package conflicts and supports distributed inference across GPUs, SLURM clusters, or edge devices.

🧠 Available Pre-trained Models

[ ] We observed a slight performance drop (around 1% across different datasets) after converting our models to the HF format, and we’re actively investigating the cause.

🧠 About Libero Setup and Evluation

[x] For questions about converting relative actions to absolute actions and our implementation, please first refer to issue #2 and #15. We have updated full preprocessing guidance here.

🔥 Update: We have released the LoRA fine-tuning code, along with checkpoints and the associated inference code.

| Model ID | Embodiment | Description | Performance | Evaluation Guidance | | :------------------------------------------------------------------------------------------------- | :---------------- | :---------------------------------------------------------------------------------------------- | :--------------: | :-----------------: | | 2toINF/X-VLA-Pt | Foundation | Pretrained on large-scale heterogeneous robot–vision–language datasets for general transfer. | — | — | | 2toINF/X-VLA-AgiWorld-Challenge | Agibot-G1 | Fine-tuned for AgiWorld Challenge. | Champion🥇 | - | | 2toINF/X-VLA-Calvin-ABC_D | Franka | Fine-tuned on CALVIN benchmark (ABC_D subset) | 4.43 | Calvin Eval | | 2toINF/X-VLA-Google-Robot | Google Robot | Fine-tuned on large-scale Google Robot dataset | 83.5%(VM) 76.4%(VA) | Simpler Eval | | 2toINF/X-VLA-Libero | Franka | Fine-tuned on LIBERO benchmark | 98.1% | LIBERO Eval | | 2toINF/X-VLA-VLABench | Franka | Fine-tuned on VLABench benchmark | 51.1(score) | VLABench Eval | | 2toINF/X-VLA-RoboTwin2 | Agilex | Trained on RoboTwin2 dataset for dual-arm coordinated manipulation(50 demos for each task). | 70% | RoboTwin2.0 Eval | | 2toINF/X-VLA-WidowX | WidowX | Fine-tuned on BridgeDataV2 (Simpler benchmark). | 95.8% | Simpler Eval | | 2toINF/X-VLA-SoftFold | Agilex | Fine-tuned on Soft-Fold Dataset. Specialized in deformable object manipulation (e.g., folding and cloth control). | cloth folding with a 100% success rate in 2 hours. | SoftFold-Agilex | | LoRA Adapters | || | | | 2toINF/X-VLA-libero-spatial-peft | Franka | Fine-tuned on LIBERO benchmark | 96.2% | LIBERO Eval | | 2toINF/X-VLA-libero-object-peft | Franka | Fine-tuned on LIBERO benchmark | 96% | LIBERO Eval | | 2toINF/X-VLA-libero-goal-peft | Franka | Fine-tuned on LIBERO benchmark | 94.4% | LIBERO Eval | | 2toINF/X-VLA-libero-long-peft | Franka | Fine-tuned on LIBERO benchmark | 83.2% | LIBERO Eval | | 2toINF/X-VLA-simpler-widowx-peft | WidowX | Fine-tuned on BridgeDataV2 (Simpler benchmark). | 66.7% | Simpler Eval |

🧩 Notes

All models share a consistent architecture: configuration_xvla.py, modeling_xvla.py, and unified tokenizer (tokenizer.json).
The X-VLA-Pt model is the foundation checkpoint, trained across multiple robot domains.
Each embodiment is fine-tuned for its respective environment while retaining cross-embodiment alignment.
Evaluation scripts (in evaluation/) follow a standardized format for reproducible benchmarking.

📊 Performance metrics follow standard evaluation protocols detailed in the paper.

3️⃣ Launching the Inference Server

from transformers import AutoModel, AutoProcessor
import json_numpy

# Load model and processor
model = AutoModel.from_pretrained("2toINF/X-VLA-WidowX", trust_remote_code=True)
processor = AutoProcessor.from_pretrained("2toINF/X-VLA-WidowX", trust_remote_code=True)

# Start the inference server
print("🚀 Starting X-VLA inference server...")
model.run(processor, host="0.0.0.0", port=8000)

Once launched, the API endpoint is available at:

POST http://<server_ip>:8000/act

4️⃣ Client Interaction & Action Prediction

The client communicates via HTTP POST, sending multimodal data (vision + language + proprioception) as a JSON payload.

Payload Structure

Example Client Code

import requests
import numpy as np
import json_numpy

server_url = "http://localhost:8000/act"
timeout = 5

# Prepare inputs
proprio = np.zeros(7, dtype=np.float32)
image = np.zeros((256, 256, 3), dtype=np.uint8)
instruction = "Move the gripper to the target position"

payload = {
    "proprio": json_numpy.dumps(proprio),