ManiMux

A unified, composable platform for real-robot experiments.

Policy × Runtime × Embodiment

English · 简体中文

Features · Video · Architecture · Quick Start · Documentation · Citation

ManiMux brings data collection, policy deployment and evaluation onto a shared real-robot control foundation. Instead of rebuilding the deployment stack for every model or robot, choose the policy, runtime strategy, executor and embodiment through configuration. Standard interfaces separate model inference from hardware control, making integrations reusable across embodiments rather than tied to one model–robot pair.

One experiment workflow, visible in Robo GUI. Prepare and run trials, inspect live cameras, 3D state, trajectories and chunk handoffs, then review predictions, commands and robot feedback. XPolicyLab provides the model integration boundary; human labels and PRM-as-a-Judge support evaluation of the recorded experiments.

Collect with the control semantics you deploy with. Teleoperation reuses ManiMux's hardware interfaces and shared control profiles to align action timing, joint/gripper conventions and motion limits between demonstrations and policy execution. Optional execution smoothing stays an explicit choice—not a hidden difference in a separate deployment stack.

📖 Connecting your own YAM or Tianji–TacCap? Start with local station setup. Installation and launch commands are in the Guideline; model and method guides are in Documentation.

First step: connect your own robot

For another installation of a supported robot model, bind the existing components to your devices. Users and coding agents should start with the station guide:

1. Copy the matching YAM or Tianji–TacCap template to manimux/configs/local/station.yaml.
2. Fill in the actual CAN interfaces, controller IPs, device serials and service addresses. can_left is a Linux interface name on the development station; your machine may use can0.
3. Inspect the resolved configuration without connecting devices, then follow the selected model/body runbook. Keep experiment timing, action semantics and execution settings separate.

Runtime startup and the camera, Pi05 and UMI_DP --experiment entry points automatically read this private, Git-ignored station file. Use --local <path> to select another station. Viewer network options, collection and other model launchers still have separate entry points; the station guide explains their scope.

News

• [2026-09-13] Initial version in development. We are building a shared foundation for configurable policy deployment, teleoperation collection and GUI-driven real-robot experiments.

✨ Features

Feature	Status	What it provides
Composable deployment	✅	Config-driven policy × runtime strategy × executor × embodiment
Cross-embodiment interfaces	✅	Shared contracts; component-based hardware assemblies
Inference methods	✅	Async, serial, RTC, PAINT and adaptive chunking
Robo GUI	✅	Rollout controls, cameras, 3D state, trajectories and chunk timelines
Teleop collection	✅	Leader policy + YAM GUI, with ManiMux follower control
Collection / deployment alignment	✅	Shared hardware interfaces, action timing and arm / gripper limits
Execution evidence	✅	Configs, observations, actions, commands, feedback, events and video
Evaluation	✅	Human labels + offline PRM / LLM judging
UMI / DAgger collection	—	Collection roadmap

✅ denotes implemented functionality, not validation of every model / hardware combination. Support counts also include model-only paths.

🎬 Demo Video

Dual-arm YAM rollout with live cameras, 3D robot state and action-chunk handoffs.

▶ Open the MP4 recording

🧩 Architecture

Policy inference and hardware execution are separate responsibilities. The inference strategy decides when and how to hand off a chunk; the executor turns its targets into robot commands.

%%{init: {"flowchart": {"wrappingWidth": 180, "curve": "basis", "nodeSpacing": 24, "rankSpacing": 28, "padding": 14}}}%%
flowchart LR
    OBS["<b>OBSERVE</b><br/>Cameras · robot state<br/>Build policy inputs"]:::stage

    subgraph THINK["<b>PREDICT</b>"]
        direction TB
        XPOLICY["<b>XPolicyLab</b><br/>Pi05 · XR-1 · GR00T<br/>LingBot · OpenWAM"]:::xpolicy
        NATIVE["<b>Legacy native</b><br/>MolmoAct2 · ABC"]:::native
        XPOLICY ~~~ NATIVE
    end

    PLAN["<b>ADAPT & SCHEDULE</b><br/>Async · RTC · PAINT<br/>Serial · adaptive<br/><br/>Adapter → Timeline"]:::handoff
    ACT["<b>EXECUTE</b><br/>Direct · Smooth · MPC<br/><br/>Executor + Safety<br/>Control profile"]:::stage
    ROBOT(["<b>ROBOT</b><br/>RobotBase<br/>Hardware"]):::robot
    TELEOP["<b>COLLECT</b><br/>YAM GUI<br/>LeaderPolicy"]:::collection
    REVIEW(["<b>REVIEW</b><br/>Robo GUI · records<br/>Human labels<br/>PRM-as-a-Judge"]):::side

    OBS --> THINK --> PLAN --> ACT --> ROBOT
    TELEOP --> ACT
    ACT -.-> REVIEW

    classDef stage fill:#F6F8FA,stroke:#8C959F,stroke-width:1px,color:#1F2328
    classDef handoff fill:#FFF4E5,stroke:#E36209,stroke-width:2.5px,color:#1F2328
    classDef side fill:#FFFFFF,stroke:#8C959F,stroke-dasharray:4 3,color:#57606A
    classDef robot fill:#1F2328,stroke:#1F2328,color:#FFFFFF
    classDef xpolicy fill:#8957E5,stroke:#6633B8,color:#FFFFFF
    classDef native fill:#2F6FEB,stroke:#1B4DB1,color:#FFFFFF
    classDef collection fill:#1A7F55,stroke:#125C3D,color:#FFFFFF
    style THINK fill:#FFFFFF,stroke:#8C959F,stroke-dasharray:5 4,color:#1F2328

Loading

Model servers never command hardware. Teleoperation bypasses chunk scheduling and reuses the execution interfaces, while retaining its own collection GUI and recording format. New model integrations must follow the XPolicyLab-only route; the native paths shown here remain for compatibility pending migration.

GitHub: ManiMux · XPolicyLab · PRM-as-a-Judge

🚀 Quick Start · Pi05 on YAM

This example runs Pi05 pure-joint, step-30000, put-bottles with RTC. It assumes the YAM and OpenPI environments, checkpoint and local device configuration are already prepared; see the setup guide. For a hardware-free display, use the Viewer preview.

Complete local station setup first. The camera, Pi05 and runtime commands below use the same experiment and automatically read its station bindings. This RTC recipe uses a 30 Hz command loop and model action points spaced 1/30 s apart.

From the repository root, run these in four separate terminals. Reuse matching camera / Viewer services if already running; collection and inference must not control the same robot simultaneously.

# Terminal 1: cameras
envs/yam/.venv/bin/python -m manimux.servers.camera.server \
  --experiment manimux/configs/experiments/put_bottles/yam_pi05_rtc_joint_step30000.yaml

# Terminal 2: Viewer
envs/yam/.venv/bin/python -m manimux.viewer.dashboard --robot yam --host 127.0.0.1 --port 8086

# Terminal 3: pure-joint 30k model server
XPolicyLab/policy/Pi_05/openpi/.venv/bin/python \
  -m manimux.servers.pi05 \
  --experiment manimux/configs/experiments/put_bottles/yam_pi05_rtc_joint_step30000.yaml

# Terminal 4: matching RTC runtime
envs/yam/.venv/bin/python -m manimux serve \
  --config manimux/configs/experiments/put_bottles/yam_pi05_rtc_joint_step30000.yaml

Open http://127.0.0.1:8086, then Prepare → Start rollout → Finish & Home. Normal rollouts need no label; experiment rollouts require a human label before the next trial. Keep the server and runtime configs paired: this example uses joint, not joint+EE.

📚 Guides

• Run: Guideline · Configuration.
• Integrate: Components and policy runbooks · Inference methods.
• Collect / evaluate: YAM collection · Experiment workflow · PRM guide.
• Extend: Architecture contracts.

📝 Citation

If ManiMux supports your experiments, please cite the repository. For experiments using its XPolicyLab integration, please also cite the XPolicyLab paper.

@misc{manimux2026,
  title = {{ManiMux}: A Composable Platform for Real-Robot Experiments},
  year = {2026},
  howpublished = {GitHub repository},
  url = {https://github.com/SII-LiuLab/manimux}
}

@article{community2026xpolicylab,
  title = {{XPolicyLab}: A Unified Standard and Open Ecosystem for Robot Policy Evaluation and Deployment},
  author = {{XPolicyLab Community} and Chen, Tianxing and Chen, Yue and Nian, Tian and others},
  journal = {arXiv preprint arXiv:2608.09892},
  year = {2026},
  doi = {10.48550/arXiv.2608.09892},
  url = {https://arxiv.org/abs/2608.09892}
}

---

Physical robots require matching model contracts and hardware safety measures; software checks do not certify safety or task success. Upstream attribution: notices · licenses.