epilectrik
/
teleop
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Browse Source

[update] README

main
silencht 4 months ago
parent
62e197c1e4
commit
3528690046
2 changed files with 182 additions and 107 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 205
      README.md
  2. 66
      README_zh-CN.md

205
README.md
View File

@ -40,13 +40,16 @@
- The **image server** has been changed to [teleimager](https://github.com/silencht/teleimager). Please refer to the repository README for details.
- [televuer](https://github.com/silencht/televuer) has been upgraded. Please see the repository README for details.
- Upgraded [televuer](https://github.com/silencht/televuer). Please see the repository README for details.
> The new versions of [teleimager](https://github.com/silencht/teleimager/commit/ab5018691943433c24af4c9a7f3ea0c9a6fbaf3c) + [televuer](https://github.com/silencht/televuer/releases/tag/v3.0) support transmitting head camera images via **WebRTC**.
> The new versions of [teleimager](https://github.com/silencht/teleimager/commit/ab5018691943433c24af4c9a7f3ea0c9a6fbaf3c) + [televuer](https://github.com/silencht/televuer/releases/tag/v3.0) support transmitting **head camera images via WebRTC**.
> Supports **pass-through**, **ego**, and **immersive** modes.
- Improved the system’s **state machine** information and IPC mode.
- Added **pass-through mode**, allowing direct viewing of the external environment through a VR device camera (without using the robot’s head camera).
- ...
- Added support for **Inspire_FTP dexterous hand**.
- …
# 0. 📖 Introduction
@ -128,19 +131,55 @@ For more information, you can refer to [Official Documentation ](https://support
(tv) unitree@Host:~$ cd xr_teleoperate
# Shallow clone submodule
(tv) unitree@Host:~/xr_teleoperate$ git submodule update --init --depth 1
```
```bash
# Install teleimager submodule
(tv) unitree@Host:~/xr_teleoperate$ cd teleop/teleimager
(tv) unitree@Host:~/xr_teleoperate/teleop/teleimager$ pip install -e . --no-deps
```
```bash
# Install televuer submodule
(tv) unitree@Host:~/xr_teleoperate$ cd teleop/televuer
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ pip install -e .
# Generate the certificate files required for televuer submodule
# Configure SSL certificates for the televuer module so that XR devices (e.g., Pico / Quest / Apple Vision Pro) can securely connect via HTTPS / WebRTC
# 1. Generate certificate files
# 1.1 For Pico / Quest XR devices
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout key.pem -out cert.pem
# Install dex-retargeting submodule
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ cd ../robot_control/dex-retargeting/
(tv) unitree@Host:~/xr_teleoperate/teleop/robot_control/dex-retargeting$ pip install -e .
# Install additional dependencies required by this repo
(tv) unitree@Host:~/xr_teleoperate/teleop/robot_control/dex-retargeting$ cd ../../../
(tv) unitree@Host:~/xr_teleoperate$ pip install -r requirements.txt
# 1.2 For Apple Vision Pro
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ openssl genrsa -out rootCA.key 2048
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 365 -out rootCA.pem -subj "/CN=xr-teleoperate"
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ openssl genrsa -out key.pem 2048
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ openssl req -new -key key.pem -out server.csr -subj "/CN=localhost"
# Create server_ext.cnf file with the following content (IP.2 should match your host IP, e.g., 192.168.123.2. Use ifconfig or similar to check)
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ vim server_ext.cnf
subjectAltName = @alt_names
[alt_names]
DNS.1 = localhost
IP.1 = 192.168.123.164
IP.2 = 192.168.123.2
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ openssl x509 -req -in server.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out cert.pem -days 365 -sha256 -extfile server_ext.cnf
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ ls
build cert.pem key.pem LICENSE pyproject.toml README.md rootCA.key rootCA.pem rootCA.srl server.csr server_ext.cnf src test
# Copy rootCA.pem to Apple Vision Pro via AirDrop and install it
# Enable firewall
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ sudo ufw allow 8012
# 2. Configure certificate paths, choose one method
# 2.1 Environment variables (optional)
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ echo 'export XR_TELEOP_CERT="$HOME/xr_teleoperate/teleop/televuer/cert.pem"' >> ~/.bashrc
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ echo 'export XR_TELEOP_KEY="$HOME/xr_teleoperate/teleop/televuer/key.pem"' >> ~/.bashrc
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ source ~/.bashrc
# 2.2 User config directory (optional)
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ mkdir -p ~/.config/xr_teleoperate/
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ cp cert.pem key.pem ~/.config/xr_teleoperate/
```
## 1.2 🕹️ unitree_sdk2_python
```bash
@ -167,10 +206,41 @@ For more information, you can refer to [Official Documentation ](https://support
>
> You can refer to [Harley Hahn's Guide to Unix and Linux](https://www.harley.com/unix-book/book/chapters/04.html#H) and [Conda User Guide](https://docs.conda.io/projects/conda/en/latest/user-guide/getting-started.html) to learn more.
## 1.3 🚀 Launch Parameter Description
- **Basic control parameters**
| ⚙️ Parameter | 📜 Description | 🔘 Available Options | 📌 Default |
| :---------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :---------------: |
| `--frequency` | Set the FPS for recording and control | Any reasonable float value | 30.0 |
| `--input-mode` | Choose XR input mode (how to control the robot) | `hand` (**hand tracking**)`controller` (**controller tracking**) | `hand` |
| `--display-mode` | Choose XR display mode (how to view the robot perspective) | `immersive` (immersive)`ego` (pass-through + small first-person window)`pass-through` (pass-through only) | `immersive` |
| `--arm` | Select the robot arm type (see 0. 📖 Introduction) | `G1_29``G1_23``H1_2``H1` | `G1_29` |
| `--ee` | Select the end-effector type of the arm (see 0. 📖 Introduction) | `dex1``dex3``inspire_ftp``inspire_dfx``brainco` | None |
| `--img-server-ip` | Set the image server IP address for receiving image streams and configuring WebRTC signaling | `IPv4` address | `192.168.123.164` |
- **Mode switch parameters**
| ⚙️ Parameter | 📜 Description |
| :----------: | :----------------------------------------------------------: |
| `--motion` | **Enable motion control mode**When enabled, the teleoperation program can run alongside the robot’s motion control program.In **hand tracking** mode, the [R3 controller](https://www.unitree.com/cn/R3) can be used to control normal robot walking; in **controller tracking** mode, joysticks can also control the robot’s movement. |
| `--headless` | **Enable headless mode**For running the program on devices without a display, e.g., the Development Computing Unit (PC2). |
| `--sim` | **Enable [simulation mode](https://github.com/unitreerobotics/unitree_sim_isaaclab)** |
| `--ipc` | **Inter-process communication mode**Allows controlling the xr_teleoperate program’s state via IPC. Suitable for interaction with agent programs. |
| `--affinity` | **CPU affinity mode**Set CPU core affinity. If you are unsure what this is, do not set it. |
| `--record` | **Enable data recording mode**Press **r** to start teleoperation, then **s** to start recording; press **s** again to stop and save the episode. Press **s** repeatedly to repeat the process. |
| `--task-*` | Configure the save path, target, description, and steps of the recorded task. |
# 2. 💻 Simulation Deployment
## 2.1 📥 Environment Setup
> The v1.4 version’s simulation deployment is not yet available. Please use v1.3 for testing temporarily.
First, install [unitree_sim_isaaclab](https://github.com/unitreerobotics/unitree_sim_isaaclab). Follow that repo’s README.
Then launch the simulation with a G1(29 DoF) and Dex3 hand configuration:
@ -195,23 +265,6 @@ Here is the simulation GUI:
This program supports XR control of a physical robot or in simulation. Choose modes with command-line arguments:
- **Basic control parameters**
| ⚙️ Parameter | 📜 Description | 🔘 Options | 📌 Default |
| :---------: | :-------------------------------------------: | :----------------------------------------------------------: | :-------: |
| `--xr-mode` | Choose XR input mode | `hand` (**hand tracking**)<br/>`controller` (**controller tracking**) | `hand` |
| `--arm` | Choose robot arm type (see 0. 📖 Introduction) | `G1_29`<br/>`G1_23`<br/>`H1_2`<br/>`H1` | `G1_29` |
| `--ee` | Choose end-effector (see 0. 📖 Introduction) | `dex1`<br/>`dex3`<br/>`inspire1`<br />`brainco` | none |
- **Mode flags**
| ⚙️ Flag | 📜 Description |
| :----------: | :----------------------------------------------------------: |
| `--record` | Enable **data recording**<br />After pressing **r** to start, press **s** to start/stop saving an episode. Can repeat. |
| `--motion` | Enable **motion mode**<br />After enabling this mode, the teleoperation program can run alongside the robot's motion control.<br />In **hand tracking** mode, you can use the [R3 Controller](https://www.unitree.com/cn/R3) to control the robot's walking behavior; <br />in **controller tracking** mode, you can also use [controllers to control the robot’s movement](https://github.com/unitreerobotics/xr_teleoperate/blob/375cdc27605de377c698e2b89cad0e5885724ca6/teleop/teleop_hand_and_arm.py#L247-L257). |
| `--headless` | Run without GUI (for headless PC2 deployment) |
| `--sim` | Enable **simulation mode** |
Assuming hand tracking with G1(29 DoF) + Dex3 in simulation with recording:
```bash
@ -231,7 +284,7 @@ Next steps:
2. Connect to the corresponding Wi‑Fi
3. Open a browser (e.g. Safari or PICO Browser) and go to: `https://192.168.123.2:8012?ws=wss://192.168.123.2:8012`
3. Open a browser (e.g. Safari or PICO Browser) and go to: `https://192.168.123.2:8012/?ws=wss://192.168.123.2:8012`
> **Note 1**: This IP must match your **Host** IP (check with `ifconfig`).
>
@ -268,6 +321,8 @@ Next steps:
> **Note 1**: Recorded data is stored in `xr_teleoperate/teleop/utils/data` by default, with usage instructions at this repo: [unitree_IL_lerobot](https://github.com/unitreerobotics/unitree_IL_lerobot/tree/main?tab=readme-ov-file#data-collection-and-conversion).
>
> **Note 2**: Please pay attention to your disk space size during data recording.
>
> **Note 3**: In v1.4 and above, the “record image” window has been removed.
## 2.3 🔚 Exit
@ -283,42 +338,58 @@ Physical deployment steps are similar to simulation, with these key differences:
In the simulation environment, the image service is automatically enabled. For physical deployment, you need to manually start the image service based on your specific camera hardware. The steps are as follows:
Copy `image_server.py` in the `xr_teleoperate/teleop/image_server` directory to the **Development Computing Unit PC2** of Unitree Robot (G1/H1/H1_2/etc.),
1. Install the image service program on the **Development Computing Unit PC2** of the Unitree robot (G1/H1/H1_2, etc.)
```bash
# p.s. You can transfer image_server.py to PC2 via the scp command and then use ssh to remotely login to PC2 to execute it.
# Assuming the IP address of the development computing unit PC2 is 192.168.123.164, the transmission process is as follows:
# log in to PC2 via SSH and create the folder for the image server
(tv) unitree@Host:~$ ssh unitree@192.168.123.164 "mkdir -p ~/image_server"
# Copy the local image_server.py to the ~/image_server directory on PC2
(tv) unitree@Host:~$ scp ~/xr_teleoperate/teleop/image_server/image_server.py unitree@192.168.123.164:~/image_server/
# SSH into PC2 and download the image service repository
(base) unitree@PC2:~$ cd ~
(base) unitree@PC2:~$ git clone https://github.com/silencht/teleimager
# Configure the environment according to the instructions in the teleimager repository README: https://github.com/silencht/teleimager/blob/main/README.md
```
and execute the following command **in the PC2**:
2. On the **local host**, execute the following commands:
```bash
# p.s. Currently, this image transmission program supports two methods for reading images: OpenCV and Realsense SDK. Please refer to the comments in the `ImageServer` class within `image_server.py` to configure your image transmission service according to your camera hardware.
# Now located in Unitree Robot PC2 terminal
unitree@PC2:~/image_server$ python image_server.py
# You can see the terminal output as follows:
# {'fps': 30, 'head_camera_type': 'opencv', 'head_camera_image_shape': [480, 1280], 'head_camera_id_numbers': [0]}
# [Image Server] Head camera 0 resolution: 480.0 x 1280.0
# [Image Server] Image server has started, waiting for client connections...
# Copy the `key.pem` and `cert.pem` files configured in Section 1.1 from the **local host** `xr_teleoperate/teleop/televuer` directory to the corresponding path on PC2
# These two files are required by teleimager to start the WebRTC service
(tv) unitree@Host:~$ scp ~/xr_teleoperate/teleop/televuer/key.pem ~/xr_teleoperate/teleop/televuer/cert.pem unitree@192.168.123.164:~/teleimager
# On PC2, configure the certificate path according to the teleimager repository README, for example:
(teleimager) unitree@PC2:~$ cd teleimager
(teleimager) unitree@PC2:~$ mkdir -p ~/.config/xr_teleoperate/
(teleimager) unitree@PC2:~/teleimager$ cp cert.pem key.pem ~/.config/xr_teleoperate/
```
After image service is started, you can use `image_client.py` **in the Host** terminal to test whether the communication is successful:
3. On the **development computing unit PC2**, configure `cam_config_server.yaml` according to the teleimager documentation and start the image service.
```bash
(tv) unitree@Host:~/xr_teleoperate/teleop/image_server$ python image_client.py
(teleimager) unitree@PC2:~/image_server$ python -m teleimager.image_server
# The following command works the same way
(teleimager) unitree@PC2:~/image_server$ teleimager-server
```
4. On the **local host**, execute the following command to subscribe to the images
```bash
(tv) unitree@Host:~$ cd ~/xr_teleoperate/teleop/teleimager/src
(tv) unitree@Host:~/xr_teleoperate/teleop/teleimager/src$ python -m teleimager.image_client --host 192.168.123.164
# If the WebRTC image stream is set up, you can also open the URL [https://192.168.123.164:60001](https://192.168.123.164:60001) in a browser and click the Start button to test.
```
## 3.2 ✋ Inspire Hand Service (optional)
> **Note 1**: Skip this if your config does not use the Inspire hand.
>
> **Note 2**: For G1 robot with [Inspire DFX hand](https://support.unitree.com/home/zh/G1_developer/inspire_dfx_dexterous_hand), related issue [#46](https://github.com/unitreerobotics/xr_teleoperate/issues/46).
>
> **Note 3**: For [Inspire FTP hand]((https://support.unitree.com/home/zh/G1_developer/inspire_ftp_dexterity_hand)), related issue [#48](https://github.com/unitreerobotics/xr_teleoperate/issues/48).
> **Note 3**: For [Inspire FTP hand]((https://support.unitree.com/home/zh/G1_developer/inspire_ftp_dexterity_hand)), related issue [#48](https://github.com/unitreerobotics/xr_teleoperate/issues/48). FTP dexterous hand is now supported. Please refer to the `--ee` parameter for configuration.
First, use [this URL: DFX_inspire_service](https://github.com/unitreerobotics/DFX_inspire_service) to clone the dexterous hand control interface program. And Copy it to **PC2** of Unitree robots.
@ -342,6 +413,8 @@ unitree@PC2:~/DFX_inspire_service/build$ ./hand_example
If two hands open and close continuously, it indicates success. Once successful, close the `./hand_example` program in Terminal 2.
## 3.3 ✋ BrainCo Hand Service (Optional)
Please refer to the [official documentation](https://support.unitree.com/home/en/G1_developer/brainco_hand) for setup instructions.
@ -363,8 +436,7 @@ sudo ./brainco_hand --id 127 --serial /dev/ttyUSB2
>
> 1. Everyone must keep a safe distance from the robot to prevent any potential danger!
> 2. Please make sure to read the [Official Documentation](https://support.unitree.com/home/zh/Teleoperation) at least once before running this program.
> 3. Without `--motion`, always make sure that the robot has entered [debug mode (L2+R2)](https://support.unitree.com/home/zh/H1_developer/Remote_control) to stop the motion control program, this will avoid potential command conflict problems.
> 4. To use motion mode (with `--motion`), ensure the robot is in control mode (via [R3 remote](https://www.unitree.com/R3)).
> 3. To use motion mode (with `--motion`), ensure the robot is in control mode (via [R3 remote](https://www.unitree.com/R3)).
> 5. In motion mode:
> - Right controller **A** = Exit teleop
> - Both joysticks pressed = soft emergency stop (switch to damping mode)
@ -393,37 +465,37 @@ Same as simulation but follow the safety warnings above.
```
xr_teleoperate/
│
├── assets [Storage of robot URDF-related files]
│
├── hardware [3D‑printed hardware modules]
├── assets [Stores robot URDF-related files]
│
├── teleop
│ ├── image_server
│ │ ├── image_client.py [Used to receive image data from the robot image server]
│ │ ├── image_server.py [Capture images from cameras and send via network (Running on robot's Development Computing Unit PC2)]
│ ├── teleimager [New image service library, supporting multiple features]
│ │
│ ├── televuer
│ │ ├── src/televuer
│ │ ├── television.py [captures XR devices's head, wrist, hand/controller data]
│ │ ├── television.py [Captures head, wrist, and hand/controller data from XR devices using Vuer]
│ │ ├── tv_wrapper.py [Post-processing of captured data]
│ │ ├── test
│ │ ├── _test_television.py [test for television.py]
│ │ ├── _test_tv_wrapper.py [test for tv_wrapper.py]
│ │ ├── _test_television.py [Test program for television.py]
│ │ ├── _test_tv_wrapper.py [Test program for tv_wrapper.py]
│ │
│ ├── robot_control
│ │ ├── src/dex-retargeting [Dexterous hand retargeting algorithm library]
│ │ ├── robot_arm_ik.py [Inverse kinematics of the arm]
│ │ ├── robot_arm.py [Control dual arm joints and lock the others]
│ │ ├── hand_retargeting.py [Dexterous hand retargeting algorithm library Wrapper]
│ │ ├── robot_hand_inspire.py [Control inspire hand joints]
│ │ ├── robot_hand_unitree.py [Control unitree hand joints]
│ │ ├── robot_arm_ik.py [Inverse kinematics for the arm]
│ │ ├── robot_arm.py [Controls dual-arm joints and locks other parts]
│ │ ├── hand_retargeting.py [Wrapper for the dexterous hand retargeting library]
│ │ ├── robot_hand_inspire.py [Controls Inspire dexterous hand]
│ │ ├── robot_hand_unitree.py [Controls Unitree dexterous hand]
│ │
│ ├── utils
│ │ ├── episode_writer.py [Used to record data for imitation learning]
│ │ ├── weighted_moving_filter.py [For filtering joint data]
│ │ ├── rerun_visualizer.py [For visualizing data during recording]
│ │ ├── weighted_moving_filter.py [Filter for joint data]
│ │ ├── rerun_visualizer.py [Visualizes recorded data]
│ │ ├── ipc.py [Handles inter-process communication with proxy programs]
│ │ ├── motion_switcher.py [Switches motion control states]
│ │ ├── sim_state_topic.py [For simulation deployment]
│ │
│ └── teleop_hand_and_arm.py [Startup execution code for teleoperation]
│ └── teleop_hand_and_arm.py [Startup script for teleoperation]
```
# 5. 🛠️ Hardware
@ -445,3 +517,4 @@ This code builds upon following open-source code-bases. Please visit the URLs to
7. https://github.com/zeromq/pyzmq
8. https://github.com/Dingry/BunnyVisionPro
9. https://github.com/unitreerobotics/unitree_sdk2_python
10. https://github.com/ARCLab-MIT/beavr-bot

66
README_zh-CN.md
View File

@ -137,6 +137,7 @@
```bash
# 安装 teleimager 模块
(tv) unitree@Host:~/xr_teleoperate$ cd teleop/teleimager
(tv) unitree@Host:~/xr_teleoperate/teleop/teleimager$ pip install -e . --no-deps
```
@ -163,9 +164,11 @@ IP.2 = 192.168.123.2
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ openssl x509 -req -in server.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out cert.pem -days 365 -sha256 -extfile server_ext.cnf
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ ls
build cert.pem key.pem LICENSE pyproject.toml README.md rootCA.key rootCA.pem rootCA.srl server.csr server_ext.cnf src test
# 通过 AirDrop 将 rootCA.pem 复制到 Apple Vision Pro 并安装它
# 开启防火墙
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ sudo ufw allow 8012
# 通过 AirDrop 将 rootCA.pem 复制到 Apple Vision Pro 并安装它
# 2. 配置证书路径，以下方式任选其一
# 2.1 环境变量配置（可选）
(tv) unitree@Host:~/xr_teleoperate/teleop/televuer$ echo 'export XR_TELEOP_CERT="$HOME/xr_teleoperate/teleop/televuer/cert.pem"' >> ~/.bashrc
@ -217,7 +220,35 @@ build cert.pem key.pem LICENSE pyproject.toml README.md rootCA.key rootCA
>
> 您可以参考 [Harley Hahn's Guide to Unix and Linux](https://www.harley.com/unix-book/book/chapters/04.html#H) 和 [Conda User Guide](https://docs.conda.io/projects/conda/en/latest/user-guide/getting-started.html) 来深入了解这些知识。
## 1.3 🚀 启动参数说明
- 基础控制参数
| ⚙️ 参数 | 📜 说明 | 🔘 目前可选值 | 📌 默认值 |
| :---------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :---------------: |
| `--frequency` | 设置录制和控制的 FPS | 任意正常范围内的浮点数 | 30.0 |
| `--input-mode` | 选择 XR 输入模式（通过什么方式控制机器人） | `hand`（**手势跟踪**）<br />`controller`（**手柄跟踪**） | `hand` |
| `--display-mode` | 选择 XR 显示模式（通过什么方式查看机器人视角） | `immersive`（沉浸式）<br />`ego`（通透+第一人称小窗）<br />`pass-through`（通透） | `immersive` |
| `--arm` | 选择机器人设备类型（可参考 0. 📖 介绍） | `G1_29`<br />`G1_23`<br />`H1_2`<br />`H1` | `G1_29` |
| `--ee` | 选择手臂的末端执行器设备类型（可参考 0. 📖 介绍） | `dex1`<br />`dex3`<br />`inspire_ftp`<br />`inspire_dfx`<br />`brainco` | 无默认值 |
| `--img-server-ip` | 设置图像服务器的 IP 地址，用于接收图像服务流、配置 WebRTC 信令服务地址 | `IPv4` 地址 | `192.168.123.164` |
- 模式开关参数
| ⚙️ 参数 | 📜 说明 |
| :----------: | :----------------------------------------------------------: |
| `--motion` | 【启用**运动控制**模式】<br />开启本模式后，可在机器人运控程序运行下进行遥操作程序。<br />**手势跟踪**模式下，可使用 [R3遥控器](https://www.unitree.com/cn/R3) 控制机器人正常行走；**手柄跟踪**模式下，也可使用[手柄摇杆控制机器人行走](https://github.com/unitreerobotics/xr_teleoperate/blob/375cdc27605de377c698e2b89cad0e5885724ca6/teleop/teleop_hand_and_arm.py#L247-L257)。 |
| `--headless` | 【启用**无图形界面**模式】<br />适用于本程序部署在开发计算单元（PC2）等无显示器情况 |
| `--sim` | 【启用[**仿真模式**](https://github.com/unitreerobotics/unitree_sim_isaaclab)】 |
| `--ipc` | 【进程间通信模式】<br />可通过进程间通信来控制 xr_teleoperate 程序的状态切换，此模式适合与代理程序进行交互 |
| `--affinity` | 【CPU亲和模式】<br />设置 CPU 核心亲和性。如果你不知道这是什么，那么请不要设置它。 |
| `--record` | 【启用**数据录制**模式】<br />按 **r** 键进入遥操后，按 **s** 键可开启数据录制，再次按 **s** 键可结束录制并保存本次 episode 数据。<br />继续按下 **s** 键可重复前述过程。 |
| `--task-*` | 此类参数可配置录制的文件保存路径，任务目标、描述、步骤等信息 |
------
# 2. 💻 仿真部署
@ -255,36 +286,7 @@ build cert.pem key.pem LICENSE pyproject.toml README.md rootCA.key rootCA
本程序支持通过 XR 设备（比如手势或手柄）来控制实际机器人动作，也支持在虚拟仿真中运行。你可以根据需要，通过命令行参数来配置运行方式。
以下是本程序的启动参数说明：
- 基础控制参数
| ⚙️ 参数 | 📜 说明 | 🔘 目前可选值 | 📌 默认值 |
| :---------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :---------------: |
| `--frequency` | 设置录制和控制的 FPS | 任意正常范围内的浮点数 | 30.0 |
| `--input-mode` | 选择 XR 输入模式（通过什么方式控制机器人） | `hand`（**手势跟踪**）<br />`controller`（**手柄跟踪**） | `hand` |
| `--display-mode` | 选择 XR 显示模式（通过什么方式查看机器人视角） | `immersive`（沉浸式）<br />`ego`（通透+第一人称小窗）<br />`pass-through`（通透） | `immersive` |
| `--arm` | 选择机器人设备类型（可参考 0. 📖 介绍） | `G1_29`<br />`G1_23`<br />`H1_2`<br />`H1` | `G1_29` |
| `--ee` | 选择手臂的末端执行器设备类型（可参考 0. 📖 介绍） | `dex1`<br />`dex3`<br />`inspire_ftp`<br />`inspire_dfx`<br />`brainco` | 无默认值 |
| `--img-server-ip` | 设置图像服务器的 IP 地址，用于接收图像服务流、配置 WebRTC 信令服务地址 | `IPv4` 地址 | `192.168.123.164` |
- 模式开关参数
| ⚙️ 参数 | 📜 说明 |
| :----------: | :----------------------------------------------------------: |
| `--motion` | 【启用**运动控制**模式】<br />开启本模式后，可在机器人运控程序运行下进行遥操作程序。<br />**手势跟踪**模式下，可使用 [R3遥控器](https://www.unitree.com/cn/R3) 控制机器人正常行走；**手柄跟踪**模式下，也可使用[手柄摇杆控制机器人行走](https://github.com/unitreerobotics/xr_teleoperate/blob/375cdc27605de377c698e2b89cad0e5885724ca6/teleop/teleop_hand_and_arm.py#L247-L257)。 |
| `--headless` | 【启用**无图形界面**模式】<br />适用于本程序部署在开发计算单元（PC2）等无显示器情况 |
| `--sim` | 【启用[**仿真模式**](https://github.com/unitreerobotics/unitree_sim_isaaclab)】 |
| `--ipc` | 【进程间通信模式】<br />可通过进程间通信来控制 xr_teleoperate 程序的状态切换，此模式适合与代理程序进行交互 |
| `--affinity` | 【CPU亲和模式】<br />设置 CPU 核心亲和性。如果你不知道这是什么，那么请不要设置它。 |
| `--record` | 【启用**数据录制**模式】<br />按 **r** 键进入遥操后，按 **s** 键可开启数据录制，再次按 **s** 键可结束录制并保存本次 episode 数据。<br />继续按下 **s** 键可重复前述过程。 |
| `--task-*` | 此类参数可配置录制的文件保存路径，任务目标、描述、步骤等信息 |
------
根据上述参数说明以及仿真环境配置，我们假设选择**手势跟踪**来控制 G1(29 DoF) + Dex3 灵巧手设备，同时开启仿真模式和数据录制模式。
根据 1.3 节参数说明以及仿真环境配置，我们假设选择**手势跟踪**来控制 G1(29 DoF) + Dex3 灵巧手设备，同时开启仿真模式和数据录制模式。
则启动命令如下所示：
@ -309,7 +311,7 @@ build cert.pem key.pem LICENSE pyproject.toml README.md rootCA.key rootCA
2. 连接对应的 WiFi 热点
3. 打开浏览器应用（比如 Safari 或 PICO Browser），输入并访问网址：https://192.168.123.2:8012?ws=wss://192.168.123.2:8012
3. 打开浏览器应用（比如 Safari 或 PICO Browser），输入并访问网址：https://192.168.123.2:8012/?ws=wss://192.168.123.2:8012
> 注意1：此 IP 地址应与您的 **主机** IP 地址匹配。该地址可以使用 `ifconfig` 等类似命令查询。

Write Preview
Loading…
Cancel
Save