diff --git a/README.md b/README.md index ed612c7..4adcc97 100644 --- a/README.md +++ b/README.md @@ -29,18 +29,37 @@

- # 🔖 Release Note -1. **Upgraded the Vuer library** to support more XR device modes. The project has been renamed from **`avp_teleoperate`** to **`xr_teleoperate`** to better reflect its broader scope — now supporting not only Apple Vision Pro but also Meta Quest 3 (with controllers) and PICO 4 Ultra Enterprise (with controllers). +## 🏷️ v1.1 + +1. Added support for a new end-effector type: **`brainco`**, which refers to the [Brain Hand](https://www.brainco-hz.com/docs/revolimb-hand/) developed by [BrainCo](https://www.brainco.cn/#/product/dexterous). +2. Changed the **DDS domain ID** to `1` in **simulation mode** to prevent conflicts during physical deployment. +3. Fixed an issue where the default frequency was set too high. + +## 🏷️ v1.0 (newvuer) + +1. Upgraded the [Vuer](https://github.com/vuer-ai/vuer) library to version **v0.0.60**, expanding XR device support to two modes: **hand tracking** and **controller tracking**. The project has been renamed from **`avp_teleoperate`** to **`xr_teleoperate`** to better reflect its broader capabilities. + + Devices tested include: Apple Vision Pro, Meta Quest 3 (with controllers), and PICO 4 Ultra Enterprise (with controllers). -2. **Modularized** parts of the codebase and introduced Git submodules (`git submodule`) for better structure and maintainability. +2. Modularized parts of the codebase and integrated **Git submodules** (`git submodule`) to improve code clarity and maintainability. -3. Added **headless**, **motion**, and **simulation** modes. The startup parameter configuration has been improved for ease of use (see Section 2.2). The **simulation** mode enables environment validation and hardware failure diagnostics. +3. Introduced **headless**, **motion control**, and **simulation** modes. Startup parameter configuration has been streamlined for ease of use (see Section 2.2). + The **simulation** mode enables environment validation and hardware failure diagnostics. -4. Changed the default hand retarget algorithm from Vector to **DexPilot**, improving the precision and intuitiveness of fingertip pinching. +4. Changed the default hand retargeting algorithm from *Vector* to **DexPilot**, enhancing the precision and intuitiveness of fingertip pinching interactions. -5. Various other improvements and refinements. +5. Various other improvements and optimizations. + +## 🏷️ v0.5 (oldvuer) + +1. The repository was named **`avp_teleoperate`** in this version. +2. Supported robot included: `G1_29`, `G1_23`, `H1_2`, and `H1`. +3. Supported end-effectors included: `dex3`, `dex1(gripper)`, and `inspire1`. +4. Only supported **hand tracking mode** for XR devices (using [Vuer](https://github.com/vuer-ai/vuer) version **v0.0.32RC7**). + **Controller tracking mode** was **not** supported. +5. Data recording mode was available. @@ -92,6 +111,10 @@ The currently supported devices in this repository: Inspire dexterous hand ✅ Complete + + BrainCo dexterous hand + ✅ Complete + ··· ··· @@ -99,6 +122,7 @@ The currently supported devices in this repository: + # 1. 📦 Installation We tested our code on Ubuntu 20.04 and Ubuntu 22.04, other operating systems may be configured differently. This document primarily describes the **default mode**. @@ -138,9 +162,11 @@ For more information, you can refer to [Official Documentation ](https://support (tv) unitree@Host:~/unitree_sdk2_python$ pip install -e . ``` -> **Note 1**: The [unitree_dds_wrapper](https://github.com/unitreerobotics/unitree_dds_wrapper) in the original h1_2 branch was a temporary version. It has now been fully migrated to the official Python-based control and communication library: [unitree_sdk2_python](https://github.com/unitreerobotics/unitree_sdk2_python). +> **Note 1:** For `xr_teleoperate` versions **v1.1 and above**, please ensure that the `unitree_sdk2_python` repository is checked out to a commit **equal to or newer than** [404fe44d76f705c002c97e773276f2a8fefb57e4](https://github.com/unitreerobotics/unitree_sdk2_python/commit/404fe44d76f705c002c97e773276f2a8fefb57e4). > -> **Note 2**: All identifiers in front of the command are meant for prompting: **Which device and directory the command should be executed on**. +> **Note 2**: The [unitree_dds_wrapper](https://github.com/unitreerobotics/unitree_dds_wrapper) in the original h1_2 branch was a temporary version. It has now been fully migrated to the official Python-based control and communication library: [unitree_sdk2_python](https://github.com/unitreerobotics/unitree_sdk2_python). +> +> **Note 3**: All identifiers in front of the command are meant for prompting: **Which device and directory the command should be executed on**. > > In the Ubuntu system's `~/.bashrc` file, the default configuration is: `PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '` > @@ -183,7 +209,7 @@ This program supports XR control of a physical robot or in simulation. Choose mo | :---------: | :-------------------------------------------: | :----------------------------------------------------------: | :-------: | | `--xr-mode` | Choose XR input mode | `hand` (**hand tracking**)
`controller` (**controller tracking**) | `hand` | | `--arm` | Choose robot arm type (see 0. 📖 Introduction) | `G1_29`
`G1_23`
`H1_2`
`H1` | `G1_29` | -| `--ee` | Choose end-effector (see 0. 📖 Introduction) | `dex1`
`dex3`
`inspire1` | none | +| `--ee` | Choose end-effector (see 0. 📖 Introduction) | `dex1`
`dex3`
`inspire1`
`brainco` | none | - **Mode flags** @@ -316,7 +342,22 @@ unitree@PC2:~/h1_inspire_service/build$ ./h1_hand_example If two hands open and close continuously, it indicates success. Once successful, close the `./h1_hand_example` program in Terminal 2. -## 3.3 🚀 Launch +## 3.3 ✋ BrainCo Hand Service (Optional) + +Please refer to the [official documentation](https://support.unitree.com/home/en/G1_developer/brainco_hand) for setup instructions. + +After installation, you need to manually start the services for both dexterous hands. Example commands are shown below (note: the serial port names may vary depending on your system): + +```bash +# Terminal 1. +sudo ./brainco_hand --id 126 --serial /dev/ttyUSB1 +# Terminal 2. +sudo ./brainco_hand --id 127 --serial /dev/ttyUSB2 +``` + + + +## 3.4 🚀 Launch > ![Warning](https://img.shields.io/badge/Warning-Important-red) > @@ -333,7 +374,7 @@ If two hands open and close continuously, it indicates success. Once successful, Same as simulation but follow the safety warnings above. -## 3.4 🔚 Exit +## 3.5 🔚 Exit > ![Warning](https://img.shields.io/badge/Warning-Important-red) > diff --git a/README_zh-CN.md b/README_zh-CN.md index 36cc857..cca278b 100644 --- a/README_zh-CN.md +++ b/README_zh-CN.md @@ -29,15 +29,36 @@

+# 🔖 版本说明 -# 🔖 发布说明 +## 🏷️ v1.1 + +1. 末端执行器类型新增'brainco',这是[强脑科技第二代灵巧手](https://www.brainco-hz.com/docs/revolimb-hand/) +2. 为避免与实机部署时发生冲突,将仿真模式下的 dds 通道的domain id修改为1 +3. 修复默认频率过高的问题 + +## 🏷️ v1.0 (newvuer) + +1. 升级 [Vuer](https://github.com/vuer-ai/vuer) 库至 v0.0.60 版本,XR设备支持模式扩展为**手部跟踪**和**控制器跟踪**两种。为更准确反映功能范围,项目由 **avp_teleoperate** 更名为 **xr_teleoperate**。 + + 测试设备包括: Apple Vision Pro,Meta Quest 3(含手柄) 与 PICO 4 Ultra Enterprise(含手柄)。 -1. 升级 [Vuer](https://github.com/vuer-ai/vuer) 库,扩展了设备支持模式。为更准确反映功能范围,项目由 **avp_teleoperate** 更名为 **xr_teleoperate**,从最初仅支持 Apple Vision Pro,扩展至兼容 Meta Quest 3(含手柄) 与 PICO 4 Ultra Enterprise(含手柄) 等多款 XR 设备。 2. 对部分功能进行了**模块化**拆分,并通过 Git 子模块(git submodule)方式进行管理和加载,提升代码结构的清晰度与维护性。 + 3. 新增**无头**、**运控**及**仿真**模式,优化启动参数配置(详见第2.2节),提升使用便捷性。**仿真**模式的加入,方便了环境验证和硬件故障排查。 + 4. 将默认手部映射算法从 Vector 切换为 **DexPilot**,优化了指尖捏合的精度与交互体验。 + 5. 其他一些优化 +## 🏷️ v0.5 (oldvuer) + +1. 该版本曾经命名为 `avp_teleoperate` +2. 支持 'G1_29', 'G1_23', 'H1_2', 'H1' 机器人类型 +3. 支持 'dex3', 'gripper', 'inspire1' 末端执行器类型 +4. 仅支持 XR 设备的手部跟踪模式( [Vuer](https://github.com/vuer-ai/vuer) 版本为 v0.0.32RC7),不支持控制器模式 +5. 支持数据录制模式 + # 0. 📖 介绍 @@ -87,6 +108,10 @@
因时灵巧手 ✅ 完成 + + 强脑灵巧手 + ✅ 完成 + ··· ··· @@ -94,6 +119,7 @@ + # 1. 📦 安装 我们在 Ubuntu 20.04 和 Ubuntu 22.04 上测试了我们的代码,其他操作系统可能需要不同的配置。本文档主要介绍常规模式。 @@ -133,9 +159,11 @@ (tv) unitree@Host:~/unitree_sdk2_python$ pip install -e . ``` -> 注意1:原 h1_2 分支中的 [unitree_dds_wrapper](https://github.com/unitreerobotics/unitree_dds_wrapper) 为临时版本,现已全面转换到上述正式的 Python 版控制通信库:[unitree_sdk2_python](https://github.com/unitreerobotics/unitree_sdk2_python) +> 注意1:在 `xr_teleoperate >= v1.1` 版本中,`unitree_sdk2_python` 仓库的 commit **必须是等于或高于** [404fe44d76f705c002c97e773276f2a8fefb57e4](https://github.com/unitreerobotics/unitree_sdk2_python/commit/404fe44d76f705c002c97e773276f2a8fefb57e4) 版本 + +> 注意2:原 h1_2 分支中的 [unitree_dds_wrapper](https://github.com/unitreerobotics/unitree_dds_wrapper) 为临时版本,现已全面转换到上述正式的 Python 版控制通信库:[unitree_sdk2_python](https://github.com/unitreerobotics/unitree_sdk2_python) -> 注意2:命令前面的所有标识符是为了提示:该命令应该在哪个设备和目录下执行。 +> 注意3:命令前面的所有标识符是为了提示:该命令应该在哪个设备和目录下执行。 > > p.s. 在 Ubuntu 系统 `~/.bashrc` 文件中,默认配置: `PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '` > @@ -191,7 +219,7 @@ | :---------: | :----------------------------------------------: | :------------------------------------------------------: | :------: | | `--xr-mode` | 选择 XR 输入模式(通过什么方式控制机器人) | `hand`(**手势跟踪**)
`controller`(**手柄跟踪**) | `hand` | | `--arm` | 选择机器人设备类型(可参考 0. 📖 介绍) | `G1_29`
`G1_23`
`H1_2`
`H1` | `G1_29` | -| `--ee` | 选择手臂的末端执行器设备类型(可参考 0. 📖 介绍) | `dex1`
`dex3`
`inspire1` | 无默认值 | +| `--ee` | 选择手臂的末端执行器设备类型(可参考 0. 📖 介绍) | `dex1`
`dex3`
`inspire1`
`brainco` | 无默认值 | - 模式开关参数 @@ -352,7 +380,20 @@ unitree@PC2:~/h1_inspire_service/build$ ./h1_hand_example 如果两只手连续打开和关闭,则表示成功。一旦成功,即可关闭终端 2 中的 `./h1_hand_example` 程序。 -## 3.3 🚀 启动遥操 +## 3.3 ✋BrainCo 手部服务(可选) + +请参考[官方文档](https://support.unitree.com/home/zh/G1_developer/brainco_hand)。安装完毕后,请手动启动两个灵巧手的服务,命令示例如下(串口名称可能与实际有所差别): + +```bash +# Terminal 1. +sudo ./brainco_hand --id 126 --serial /dev/ttyUSB1 +# Terminal 2. +sudo ./brainco_hand --id 127 --serial /dev/ttyUSB2 +``` + + + +## 3.4 🚀 启动遥操 > ![Warning](https://img.shields.io/badge/Warning-Important-red) > @@ -368,7 +409,7 @@ unitree@PC2:~/h1_inspire_service/build$ ./h1_hand_example 与仿真部署基本一致,但要注意上述警告事项。 -## 3.4 🔚 退出 +## 3.5 🔚 退出 > ![Warning](https://img.shields.io/badge/Warning-Important-red) > diff --git a/assets/brainco_hand/brainco.yml b/assets/brainco_hand/brainco.yml new file mode 100644 index 0000000..50d557e --- /dev/null +++ b/assets/brainco_hand/brainco.yml @@ -0,0 +1,61 @@ +left: + type: DexPilot # or vector + urdf_path: brainco_hand/brainco_left.urdf + + # Target refers to the retargeting target, which is the robot hand + target_joint_names: + [ + "left_thumb_metacarpal_joint", + "left_thumb_proximal_joint", + "left_index_proximal_joint", + "left_middle_proximal_joint", + "left_ring_proximal_joint", + "left_pinky_proximal_joint", + ] + + # for DexPilot type + wrist_link_name: "base_link" + finger_tip_link_names: [ "left_thumb_tip", "left_index_tip", "left_middle_tip", "left_ring_tip", "left_pinky_tip" ] + # If you do not know exactly how it is used, please leave it to None for default. + target_link_human_indices_dexpilot: [[ 9, 14, 19, 24, 14, 19, 24, 19, 24, 24, 0, 0, 0, 0, 0], [ 4, 4, 4, 4, 9, 9, 9, 14, 14, 19, 4, 9, 14, 19, 24]] + + # for vector type + target_origin_link_names: ["base_link", "base_link", "base_link", "base_link", "base_link"] + target_task_link_names: [ "left_thumb_tip", "left_index_tip", "left_middle_tip", "left_ring_tip", "left_pinky_tip" ] + target_link_human_indices_vector: [ [ 0, 0, 0, 0, 0 ], [ 4, 9, 14, 19, 24 ] ] + + # Scaling factor for vector retargeting only + # For example, Allegro is 1.6 times larger than normal human hand, then this scaling factor should be 1.6 + scaling_factor: 1.00 + # A smaller alpha means stronger filtering, i.e. more smooth but also larger latency + low_pass_alpha: 0.5 + +right: + type: DexPilot # or vector + urdf_path: brainco_hand/brainco_right.urdf + + # Target refers to the retargeting target, which is the robot hand + target_joint_names: + [ + "right_thumb_metacarpal_joint", + "right_thumb_proximal_joint", + "right_index_proximal_joint", + "right_middle_proximal_joint", + "right_ring_proximal_joint", + "right_pinky_proximal_joint", + ] + # for DexPilot type + wrist_link_name: "base_link" + finger_tip_link_names: [ "right_thumb_tip", "right_index_tip", "right_middle_tip", "right_ring_tip", "right_pinky_tip" ] + target_link_human_indices_dexpilot: [[ 9, 14, 19, 24, 14, 19, 24, 19, 24, 24, 0, 0, 0, 0, 0], [ 4, 4, 4, 4, 9, 9, 9, 14, 14, 19, 4, 9, 14, 19, 24]] + + # for vector type + target_origin_link_names: ["base_link", "base_link", "base_link", "base_link", "base_link"] + target_task_link_names: [ "right_thumb_tip", "right_index_tip", "right_middle_tip", "right_ring_tip", "right_pinky_tip" ] + target_link_human_indices_vector: [ [ 0, 0, 0, 0, 0 ], [ 4, 9, 14, 19, 24 ] ] + + # Scaling factor for vector retargeting only + # For example, Allegro is 1.6 times larger than normal human hand, then this scaling factor should be 1.6 + scaling_factor: 1.00 + # A smaller alpha means stronger filtering, i.e. more smooth but also larger latency + low_pass_alpha: 0.5 diff --git a/assets/brainco_hand/brainco_left.urdf b/assets/brainco_hand/brainco_left.urdf new file mode 100644 index 0000000..e6a8b14 --- /dev/null +++ b/assets/brainco_hand/brainco_left.urdf @@ -0,0 +1,618 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/assets/brainco_hand/brainco_right.urdf b/assets/brainco_hand/brainco_right.urdf new file mode 100644 index 0000000..8551841 --- /dev/null +++ b/assets/brainco_hand/brainco_right.urdf @@ -0,0 +1,618 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/assets/brainco_hand/meshes/left_base_link.STL b/assets/brainco_hand/meshes/left_base_link.STL new file mode 100644 index 0000000..f8aee10 Binary files /dev/null and b/assets/brainco_hand/meshes/left_base_link.STL differ diff --git a/assets/brainco_hand/meshes/left_index_distal_Link.STL b/assets/brainco_hand/meshes/left_index_distal_Link.STL new file mode 100644 index 0000000..89d3a1b Binary files /dev/null and b/assets/brainco_hand/meshes/left_index_distal_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_index_proximal_Link.STL b/assets/brainco_hand/meshes/left_index_proximal_Link.STL new file mode 100644 index 0000000..2ef94bf Binary files /dev/null and b/assets/brainco_hand/meshes/left_index_proximal_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_index_tip_Link.STL b/assets/brainco_hand/meshes/left_index_tip_Link.STL new file mode 100644 index 0000000..f8720d6 Binary files /dev/null and b/assets/brainco_hand/meshes/left_index_tip_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_middle_distal_Link.STL b/assets/brainco_hand/meshes/left_middle_distal_Link.STL new file mode 100644 index 0000000..7028ace Binary files /dev/null and b/assets/brainco_hand/meshes/left_middle_distal_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_middle_proximal_Link.STL b/assets/brainco_hand/meshes/left_middle_proximal_Link.STL new file mode 100644 index 0000000..59b09b9 Binary files /dev/null and b/assets/brainco_hand/meshes/left_middle_proximal_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_middle_tip_Link.STL b/assets/brainco_hand/meshes/left_middle_tip_Link.STL new file mode 100644 index 0000000..e624cb3 Binary files /dev/null and b/assets/brainco_hand/meshes/left_middle_tip_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_pinky_distal_Link.STL b/assets/brainco_hand/meshes/left_pinky_distal_Link.STL new file mode 100644 index 0000000..bb88313 Binary files /dev/null and b/assets/brainco_hand/meshes/left_pinky_distal_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_pinky_proximal_Link.STL b/assets/brainco_hand/meshes/left_pinky_proximal_Link.STL new file mode 100644 index 0000000..33ff5b7 Binary files /dev/null and b/assets/brainco_hand/meshes/left_pinky_proximal_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_pinky_tip_Link.STL b/assets/brainco_hand/meshes/left_pinky_tip_Link.STL new file mode 100644 index 0000000..1620243 Binary files /dev/null and b/assets/brainco_hand/meshes/left_pinky_tip_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_ring_distal_Link.STL b/assets/brainco_hand/meshes/left_ring_distal_Link.STL new file mode 100644 index 0000000..1ffa718 Binary files /dev/null and b/assets/brainco_hand/meshes/left_ring_distal_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_ring_proximal_Link.STL b/assets/brainco_hand/meshes/left_ring_proximal_Link.STL new file mode 100644 index 0000000..26189dc Binary files /dev/null and b/assets/brainco_hand/meshes/left_ring_proximal_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_ring_tip_Link.STL b/assets/brainco_hand/meshes/left_ring_tip_Link.STL new file mode 100644 index 0000000..a15a561 Binary files /dev/null and b/assets/brainco_hand/meshes/left_ring_tip_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_thumb_distal_Link.STL b/assets/brainco_hand/meshes/left_thumb_distal_Link.STL new file mode 100644 index 0000000..b676fc1 Binary files /dev/null and b/assets/brainco_hand/meshes/left_thumb_distal_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_thumb_metacarpal_Link.STL b/assets/brainco_hand/meshes/left_thumb_metacarpal_Link.STL new file mode 100644 index 0000000..4797f18 Binary files /dev/null and b/assets/brainco_hand/meshes/left_thumb_metacarpal_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_thumb_proximal_Link.STL b/assets/brainco_hand/meshes/left_thumb_proximal_Link.STL new file mode 100644 index 0000000..667b69c Binary files /dev/null and b/assets/brainco_hand/meshes/left_thumb_proximal_Link.STL differ diff --git a/assets/brainco_hand/meshes/left_thumb_tip_Link.STL b/assets/brainco_hand/meshes/left_thumb_tip_Link.STL new file mode 100644 index 0000000..a43f9e2 Binary files /dev/null and b/assets/brainco_hand/meshes/left_thumb_tip_Link.STL differ diff --git a/assets/brainco_hand/meshes/right_base_link.STL b/assets/brainco_hand/meshes/right_base_link.STL new file mode 100644 index 0000000..62aeba8 Binary files /dev/null and b/assets/brainco_hand/meshes/right_base_link.STL differ diff --git a/assets/brainco_hand/meshes/right_index_distal_link.STL b/assets/brainco_hand/meshes/right_index_distal_link.STL new file mode 100644 index 0000000..9e1f56f Binary files /dev/null and b/assets/brainco_hand/meshes/right_index_distal_link.STL differ diff --git a/assets/brainco_hand/meshes/right_index_proximal_link.STL b/assets/brainco_hand/meshes/right_index_proximal_link.STL new file mode 100644 index 0000000..81d4896 Binary files /dev/null and b/assets/brainco_hand/meshes/right_index_proximal_link.STL differ diff --git a/assets/brainco_hand/meshes/right_index_tip.STL b/assets/brainco_hand/meshes/right_index_tip.STL new file mode 100644 index 0000000..b0008f8 Binary files /dev/null and b/assets/brainco_hand/meshes/right_index_tip.STL differ diff --git a/assets/brainco_hand/meshes/right_middle_distal_link.STL b/assets/brainco_hand/meshes/right_middle_distal_link.STL new file mode 100644 index 0000000..440e0d7 Binary files /dev/null and b/assets/brainco_hand/meshes/right_middle_distal_link.STL differ diff --git a/assets/brainco_hand/meshes/right_middle_proximal_link.STL b/assets/brainco_hand/meshes/right_middle_proximal_link.STL new file mode 100644 index 0000000..b13e2fc Binary files /dev/null and b/assets/brainco_hand/meshes/right_middle_proximal_link.STL differ diff --git a/assets/brainco_hand/meshes/right_middle_tip.STL b/assets/brainco_hand/meshes/right_middle_tip.STL new file mode 100644 index 0000000..80adde3 Binary files /dev/null and b/assets/brainco_hand/meshes/right_middle_tip.STL differ diff --git a/assets/brainco_hand/meshes/right_pinky_distal_link.STL b/assets/brainco_hand/meshes/right_pinky_distal_link.STL new file mode 100644 index 0000000..158a367 Binary files /dev/null and b/assets/brainco_hand/meshes/right_pinky_distal_link.STL differ diff --git a/assets/brainco_hand/meshes/right_pinky_proximal_link.STL b/assets/brainco_hand/meshes/right_pinky_proximal_link.STL new file mode 100644 index 0000000..b9eea28 Binary files /dev/null and b/assets/brainco_hand/meshes/right_pinky_proximal_link.STL differ diff --git a/assets/brainco_hand/meshes/right_pinky_tip.STL b/assets/brainco_hand/meshes/right_pinky_tip.STL new file mode 100644 index 0000000..05f82c3 Binary files /dev/null and b/assets/brainco_hand/meshes/right_pinky_tip.STL differ diff --git a/assets/brainco_hand/meshes/right_ring_distal_link.STL b/assets/brainco_hand/meshes/right_ring_distal_link.STL new file mode 100644 index 0000000..267045f Binary files /dev/null and b/assets/brainco_hand/meshes/right_ring_distal_link.STL differ diff --git a/assets/brainco_hand/meshes/right_ring_proximal_link.STL b/assets/brainco_hand/meshes/right_ring_proximal_link.STL new file mode 100644 index 0000000..e849524 Binary files /dev/null and b/assets/brainco_hand/meshes/right_ring_proximal_link.STL differ diff --git a/assets/brainco_hand/meshes/right_ring_tip.STL b/assets/brainco_hand/meshes/right_ring_tip.STL new file mode 100644 index 0000000..40724b6 Binary files /dev/null and b/assets/brainco_hand/meshes/right_ring_tip.STL differ diff --git a/assets/brainco_hand/meshes/right_thumb_distal_link.STL b/assets/brainco_hand/meshes/right_thumb_distal_link.STL new file mode 100644 index 0000000..aaae3a4 Binary files /dev/null and b/assets/brainco_hand/meshes/right_thumb_distal_link.STL differ diff --git a/assets/brainco_hand/meshes/right_thumb_metacarpal_link.STL b/assets/brainco_hand/meshes/right_thumb_metacarpal_link.STL new file mode 100644 index 0000000..b7c6379 Binary files /dev/null and b/assets/brainco_hand/meshes/right_thumb_metacarpal_link.STL differ diff --git a/assets/brainco_hand/meshes/right_thumb_proximal_link.STL b/assets/brainco_hand/meshes/right_thumb_proximal_link.STL new file mode 100644 index 0000000..a8054f6 Binary files /dev/null and b/assets/brainco_hand/meshes/right_thumb_proximal_link.STL differ diff --git a/assets/brainco_hand/meshes/right_thumb_tip.STL b/assets/brainco_hand/meshes/right_thumb_tip.STL new file mode 100644 index 0000000..6715b1e Binary files /dev/null and b/assets/brainco_hand/meshes/right_thumb_tip.STL differ diff --git a/teleop/robot_control/hand_retargeting.py b/teleop/robot_control/hand_retargeting.py index 27f1d40..80872b9 100644 --- a/teleop/robot_control/hand_retargeting.py +++ b/teleop/robot_control/hand_retargeting.py @@ -10,6 +10,8 @@ class HandType(Enum): INSPIRE_HAND_Unit_Test = "../../assets/inspire_hand/inspire_hand.yml" UNITREE_DEX3 = "../assets/unitree_hand/unitree_dex3.yml" UNITREE_DEX3_Unit_Test = "../../assets/unitree_hand/unitree_dex3.yml" + BRAINCO_HAND = "../assets/brainco_hand/brainco.yml" + BRAINCO_HAND_Unit_Test = "../../assets/brainco_hand/brainco.yml" class HandRetargeting: def __init__(self, hand_type: HandType): @@ -21,6 +23,10 @@ class HandRetargeting: RetargetingConfig.set_default_urdf_dir('../assets') elif hand_type == HandType.INSPIRE_HAND_Unit_Test: RetargetingConfig.set_default_urdf_dir('../../assets') + elif hand_type == HandType.BRAINCO_HAND: + RetargetingConfig.set_default_urdf_dir('../assets') + elif hand_type == HandType.BRAINCO_HAND_Unit_Test: + RetargetingConfig.set_default_urdf_dir('../../assets') config_file_path = Path(hand_type.value) @@ -52,22 +58,23 @@ class HandRetargeting: self.left_dex_retargeting_to_hardware = [ self.left_retargeting_joint_names.index(name) for name in self.left_dex3_api_joint_names] self.right_dex_retargeting_to_hardware = [ self.right_retargeting_joint_names.index(name) for name in self.right_dex3_api_joint_names] - # Archive: This is the joint order of the dex-retargeting library version 0.1.1. - # logger_mp.info([joint.get_name() for joint in self.left_retargeting.optimizer.robot.get_active_joints()]) - # ['left_hand_thumb_0_joint', 'left_hand_thumb_1_joint', 'left_hand_thumb_2_joint', - # 'left_hand_middle_0_joint', 'left_hand_middle_1_joint', - # 'left_hand_index_0_joint', 'left_hand_index_1_joint'] - # logger_mp.info([joint.get_name() for joint in self.right_retargeting.optimizer.robot.get_active_joints()]) - # ['right_hand_thumb_0_joint', 'right_hand_thumb_1_joint', 'right_hand_thumb_2_joint', - # 'right_hand_middle_0_joint', 'right_hand_middle_1_joint', - # 'right_hand_index_0_joint', 'right_hand_index_1_joint'] elif hand_type == HandType.INSPIRE_HAND or hand_type == HandType.INSPIRE_HAND_Unit_Test: + # "Joint Motor Sequence" of https://support.unitree.com/home/en/G1_developer/inspire_dfx_dexterous_hand self.left_inspire_api_joint_names = [ 'L_pinky_proximal_joint', 'L_ring_proximal_joint', 'L_middle_proximal_joint', 'L_index_proximal_joint', 'L_thumb_proximal_pitch_joint', 'L_thumb_proximal_yaw_joint' ] self.right_inspire_api_joint_names = [ 'R_pinky_proximal_joint', 'R_ring_proximal_joint', 'R_middle_proximal_joint', 'R_index_proximal_joint', 'R_thumb_proximal_pitch_joint', 'R_thumb_proximal_yaw_joint' ] self.left_dex_retargeting_to_hardware = [ self.left_retargeting_joint_names.index(name) for name in self.left_inspire_api_joint_names] self.right_dex_retargeting_to_hardware = [ self.right_retargeting_joint_names.index(name) for name in self.right_inspire_api_joint_names] + + elif hand_type == HandType.BRAINCO_HAND or hand_type == HandType.BRAINCO_HAND_Unit_Test: + # "Driver Motor ID" of https://www.brainco-hz.com/docs/revolimb-hand/product/parameters.html + self.left_brainco_api_joint_names = [ 'left_thumb_metacarpal_joint', 'left_thumb_proximal_joint', 'left_index_proximal_joint', + 'left_middle_proximal_joint', 'left_ring_proximal_joint', 'left_pinky_proximal_joint' ] + self.right_brainco_api_joint_names = [ 'right_thumb_metacarpal_joint', 'right_thumb_proximal_joint', 'right_index_proximal_joint', + 'right_middle_proximal_joint', 'right_ring_proximal_joint', 'right_pinky_proximal_joint' ] + self.left_dex_retargeting_to_hardware = [ self.left_retargeting_joint_names.index(name) for name in self.left_brainco_api_joint_names] + self.right_dex_retargeting_to_hardware = [ self.right_retargeting_joint_names.index(name) for name in self.right_brainco_api_joint_names] except FileNotFoundError: logger_mp.warning(f"Configuration file not found: {config_file_path}") diff --git a/teleop/robot_control/robot_arm.py b/teleop/robot_control/robot_arm.py index bcecc0c..f8d1f8d 100644 --- a/teleop/robot_control/robot_arm.py +++ b/teleop/robot_control/robot_arm.py @@ -81,7 +81,11 @@ class G1_29_ArmController: self._gradual_time = None # initialize lowcmd publisher and lowstate subscriber - ChannelFactoryInitialize(0) + if self.simulation_mode: + ChannelFactoryInitialize(1) + else: + ChannelFactoryInitialize(0) + if self.motion_mode: self.lowcmd_publisher = ChannelPublisher(kTopicLowCommand_Motion, hg_LowCmd) else: @@ -97,7 +101,7 @@ class G1_29_ArmController: self.subscribe_thread.start() while not self.lowstate_buffer.GetData(): - time.sleep(0.01) + time.sleep(0.1) logger_mp.warning("[G1_29_ArmController] Waiting to subscribe dds...") logger_mp.info("[G1_29_ArmController] Subscribe dds ok.") @@ -364,7 +368,10 @@ class G1_23_ArmController: self._gradual_time = None # initialize lowcmd publisher and lowstate subscriber - ChannelFactoryInitialize(0) + if self.simulation_mode: + ChannelFactoryInitialize(1) + else: + ChannelFactoryInitialize(0) self.lowcmd_publisher = ChannelPublisher(kTopicLowCommand_Debug, hg_LowCmd) self.lowcmd_publisher.Init() self.lowstate_subscriber = ChannelSubscriber(kTopicLowState, hg_LowState) @@ -377,7 +384,7 @@ class G1_23_ArmController: self.subscribe_thread.start() while not self.lowstate_buffer.GetData(): - time.sleep(0.01) + time.sleep(0.1) logger_mp.warning("[G1_23_ArmController] Waiting to subscribe dds...") logger_mp.info("[G1_23_ArmController] Subscribe dds ok.") @@ -629,7 +636,10 @@ class H1_2_ArmController: self._gradual_time = None # initialize lowcmd publisher and lowstate subscriber - ChannelFactoryInitialize(0) + if self.simulation_mode: + ChannelFactoryInitialize(1) + else: + ChannelFactoryInitialize(0) self.lowcmd_publisher = ChannelPublisher(kTopicLowCommand_Debug, hg_LowCmd) self.lowcmd_publisher.Init() self.lowstate_subscriber = ChannelSubscriber(kTopicLowState, hg_LowState) @@ -642,7 +652,7 @@ class H1_2_ArmController: self.subscribe_thread.start() while not self.lowstate_buffer.GetData(): - time.sleep(0.01) + time.sleep(0.1) logger_mp.warning("[H1_2_ArmController] Waiting to subscribe dds...") logger_mp.info("[H1_2_ArmController] Subscribe dds ok.") @@ -899,7 +909,10 @@ class H1_ArmController: self._gradual_time = None # initialize lowcmd publisher and lowstate subscriber - ChannelFactoryInitialize(0) + if self.simulation_mode: + ChannelFactoryInitialize(1) + else: + ChannelFactoryInitialize(0) self.lowcmd_publisher = ChannelPublisher(kTopicLowCommand_Debug, go_LowCmd) self.lowcmd_publisher.Init() self.lowstate_subscriber = ChannelSubscriber(kTopicLowState, go_LowState) @@ -912,7 +925,7 @@ class H1_ArmController: self.subscribe_thread.start() while not self.lowstate_buffer.GetData(): - time.sleep(0.01) + time.sleep(0.1) logger_mp.warning("[H1_ArmController] Waiting to subscribe dds...") logger_mp.info("[H1_ArmController] Subscribe dds ok.") diff --git a/teleop/robot_control/robot_hand_brainco.py b/teleop/robot_control/robot_hand_brainco.py new file mode 100644 index 0000000..a37b554 --- /dev/null +++ b/teleop/robot_control/robot_hand_brainco.py @@ -0,0 +1,194 @@ +from unitree_sdk2py.core.channel import ChannelPublisher, ChannelSubscriber, ChannelFactoryInitialize # dds +from unitree_sdk2py.idl.unitree_go.msg.dds_ import MotorCmds_, MotorStates_ # idl +from unitree_sdk2py.idl.default import unitree_go_msg_dds__MotorCmd_ + +from teleop.robot_control.hand_retargeting import HandRetargeting, HandType +import numpy as np +from enum import IntEnum +import threading +import time +from multiprocessing import Process, Array + +import logging_mp +logger_mp = logging_mp.get_logger(__name__) + +brainco_Num_Motors = 6 +kTopicbraincoLeftCommand = "rt/brainco/left/cmd" +kTopicbraincoLeftState = "rt/brainco/left/state" +kTopicbraincoRightCommand = "rt/brainco/right/cmd" +kTopicbraincoRightState = "rt/brainco/right/state" + +class Brainco_Controller: + def __init__(self, left_hand_array, right_hand_array, dual_hand_data_lock = None, dual_hand_state_array = None, + dual_hand_action_array = None, fps = 100.0, Unit_Test = False, simulation_mode = False): + logger_mp.info("Initialize Brainco_Controller...") + self.fps = fps + self.hand_sub_ready = False + self.Unit_Test = Unit_Test + self.simulation_mode = simulation_mode + + if not self.Unit_Test: + self.hand_retargeting = HandRetargeting(HandType.BRAINCO_HAND) + else: + self.hand_retargeting = HandRetargeting(HandType.BRAINCO_HAND_Unit_Test) + + if self.simulation_mode: + ChannelFactoryInitialize(1) + else: + ChannelFactoryInitialize(0) + + # initialize handcmd publisher and handstate subscriber + self.LeftHandCmb_publisher = ChannelPublisher(kTopicbraincoLeftCommand, MotorCmds_) + self.LeftHandCmb_publisher.Init() + self.RightHandCmb_publisher = ChannelPublisher(kTopicbraincoRightCommand, MotorCmds_) + self.RightHandCmb_publisher.Init() + + self.LeftHandState_subscriber = ChannelSubscriber(kTopicbraincoLeftState, MotorStates_) + self.LeftHandState_subscriber.Init() + self.RightHandState_subscriber = ChannelSubscriber(kTopicbraincoRightState, MotorStates_) + self.RightHandState_subscriber.Init() + + # Shared Arrays for hand states + self.left_hand_state_array = Array('d', brainco_Num_Motors, lock=True) + self.right_hand_state_array = Array('d', brainco_Num_Motors, lock=True) + + # initialize subscribe thread + self.subscribe_state_thread = threading.Thread(target=self._subscribe_hand_state) + self.subscribe_state_thread.daemon = True + self.subscribe_state_thread.start() + + while not self.hand_sub_ready: + time.sleep(0.1) + logger_mp.warning("[brainco_Controller] Waiting to subscribe dds...") + logger_mp.info("[brainco_Controller] Subscribe dds ok.") + + hand_control_process = Process(target=self.control_process, args=(left_hand_array, right_hand_array, self.left_hand_state_array, self.right_hand_state_array, + dual_hand_data_lock, dual_hand_state_array, dual_hand_action_array)) + hand_control_process.daemon = True + hand_control_process.start() + + logger_mp.info("Initialize brainco_Controller OK!\n") + + def _subscribe_hand_state(self): + while True: + left_hand_msg = self.LeftHandState_subscriber.Read() + right_hand_msg = self.RightHandState_subscriber.Read() + self.hand_sub_ready = True + if left_hand_msg is not None and right_hand_msg is not None: + # Update left hand state + for idx, id in enumerate(Brainco_Left_Hand_JointIndex): + self.left_hand_state_array[idx] = left_hand_msg.states[id].q + # Update right hand state + for idx, id in enumerate(Brainco_Right_Hand_JointIndex): + self.right_hand_state_array[idx] = right_hand_msg.states[id].q + time.sleep(0.002) + + def ctrl_dual_hand(self, left_q_target, right_q_target): + """ + Set current left, right hand motor state target q + """ + for idx, id in enumerate(Brainco_Left_Hand_JointIndex): + self.left_hand_msg.cmds[id].q = left_q_target[idx] + for idx, id in enumerate(Brainco_Right_Hand_JointIndex): + self.right_hand_msg.cmds[id].q = right_q_target[idx] + + self.LeftHandCmb_publisher.Write(self.left_hand_msg) + self.RightHandCmb_publisher.Write(self.right_hand_msg) + # logger_mp.debug("hand ctrl publish ok.") + + def control_process(self, left_hand_array, right_hand_array, left_hand_state_array, right_hand_state_array, + dual_hand_data_lock = None, dual_hand_state_array = None, dual_hand_action_array = None): + self.running = True + + left_q_target = np.full(brainco_Num_Motors, 0) + right_q_target = np.full(brainco_Num_Motors, 0) + + # initialize brainco hand's cmd msg + self.left_hand_msg = MotorCmds_() + self.left_hand_msg.cmds = [unitree_go_msg_dds__MotorCmd_() for _ in range(len(Brainco_Left_Hand_JointIndex))] + self.right_hand_msg = MotorCmds_() + self.right_hand_msg.cmds = [unitree_go_msg_dds__MotorCmd_() for _ in range(len(Brainco_Right_Hand_JointIndex))] + + for idx, id in enumerate(Brainco_Left_Hand_JointIndex): + self.left_hand_msg.cmds[id].q = 0.0 + self.left_hand_msg.cmds[id].dq = 1.0 + for idx, id in enumerate(Brainco_Right_Hand_JointIndex): + self.right_hand_msg.cmds[id].q = 0.0 + self.right_hand_msg.cmds[id].dq = 1.0 + + try: + while self.running: + start_time = time.time() + # get dual hand state + with left_hand_array.get_lock(): + left_hand_data = np.array(left_hand_array[:]).reshape(25, 3).copy() + with right_hand_array.get_lock(): + right_hand_data = np.array(right_hand_array[:]).reshape(25, 3).copy() + + # Read left and right q_state from shared arrays + state_data = np.concatenate((np.array(left_hand_state_array[:]), np.array(right_hand_state_array[:]))) + + if not np.all(right_hand_data == 0.0) and not np.all(left_hand_data[4] == np.array([-1.13, 0.3, 0.15])): # if hand data has been initialized. + ref_left_value = left_hand_data[self.hand_retargeting.left_indices[1,:]] - left_hand_data[self.hand_retargeting.left_indices[0,:]] + ref_right_value = right_hand_data[self.hand_retargeting.right_indices[1,:]] - right_hand_data[self.hand_retargeting.right_indices[0,:]] + + left_q_target = self.hand_retargeting.left_retargeting.retarget(ref_left_value)[self.hand_retargeting.left_dex_retargeting_to_hardware] + right_q_target = self.hand_retargeting.right_retargeting.retarget(ref_right_value)[self.hand_retargeting.right_dex_retargeting_to_hardware] + + # In the official document, the angles are in the range [0, 1] ==> 0.0: fully open 1.0: fully closed + # The q_target now is in radians, ranges: + # - idx 0: 0~1.52 + # - idx 1: 0~1.05 + # - idx 2~5: 0~1.47 + # We normalize them using (max - value) / range + def normalize(val, min_val, max_val): + return 1.0 - np.clip((max_val - val) / (max_val - min_val), 0.0, 1.0) + + for idx in range(brainco_Num_Motors): + if idx == 0: + left_q_target[idx] = normalize(left_q_target[idx], 0.0, 1.52) + right_q_target[idx] = normalize(right_q_target[idx], 0.0, 1.52) + elif idx == 1: + left_q_target[idx] = normalize(left_q_target[idx], 0.0, 1.05) + right_q_target[idx] = normalize(right_q_target[idx], 0.0, 1.05) + elif idx >= 2: + left_q_target[idx] = normalize(left_q_target[idx], 0.0, 1.47) + right_q_target[idx] = normalize(right_q_target[idx], 0.0, 1.47) + + # get dual hand action + action_data = np.concatenate((left_q_target, right_q_target)) + if dual_hand_state_array and dual_hand_action_array: + with dual_hand_data_lock: + dual_hand_state_array[:] = state_data + dual_hand_action_array[:] = action_data + # logger_mp.info(f"left_q_target:{left_q_target}") + self.ctrl_dual_hand(left_q_target, right_q_target) + current_time = time.time() + time_elapsed = current_time - start_time + sleep_time = max(0, (1 / self.fps) - time_elapsed) + time.sleep(sleep_time) + finally: + logger_mp.info("brainco_Controller has been closed.") + +# according to the official documentation, https://www.brainco-hz.com/docs/revolimb-hand/product/parameters.html +# the motor sequence is as shown in the table below +# ┌──────┬───────┬────────────┬────────┬────────┬────────┬────────┐ +# │ Id │ 0 │ 1 │ 2 │ 3 │ 4 │ 5 │ +# ├──────┼───────┼────────────┼────────┼────────┼────────┼────────┤ +# │Joint │ thumb │ thumb-aux | index │ middle │ ring │ pinky │ +# └──────┴───────┴────────────┴────────┴────────┴────────┴────────┘ +class Brainco_Right_Hand_JointIndex(IntEnum): + kRightHandThumb = 0 + kRightHandThumbAux = 1 + kRightHandIndex = 2 + kRightHandMiddle = 3 + kRightHandRing = 4 + kRightHandPinky = 5 + +class Brainco_Left_Hand_JointIndex(IntEnum): + kLeftHandThumb = 0 + kLeftHandThumbAux = 1 + kLeftHandIndex = 2 + kLeftHandMiddle = 3 + kLeftHandRing = 4 + kLeftHandPinky = 5 \ No newline at end of file diff --git a/teleop/robot_control/robot_hand_inspire.py b/teleop/robot_control/robot_hand_inspire.py index bec0a63..d884a69 100644 --- a/teleop/robot_control/robot_hand_inspire.py +++ b/teleop/robot_control/robot_hand_inspire.py @@ -1,4 +1,3 @@ -# this file is legacy, need to fix. from unitree_sdk2py.core.channel import ChannelPublisher, ChannelSubscriber, ChannelFactoryInitialize # dds from unitree_sdk2py.idl.unitree_go.msg.dds_ import MotorCmds_, MotorStates_ # idl from unitree_sdk2py.idl.default import unitree_go_msg_dds__MotorCmd_ @@ -19,14 +18,19 @@ kTopicInspireState = "rt/inspire/state" class Inspire_Controller: def __init__(self, left_hand_array, right_hand_array, dual_hand_data_lock = None, dual_hand_state_array = None, - dual_hand_action_array = None, fps = 100.0, Unit_Test = False): + dual_hand_action_array = None, fps = 100.0, Unit_Test = False, simulation_mode = False): logger_mp.info("Initialize Inspire_Controller...") self.fps = fps self.Unit_Test = Unit_Test + self.simulation_mode = simulation_mode if not self.Unit_Test: self.hand_retargeting = HandRetargeting(HandType.INSPIRE_HAND) else: self.hand_retargeting = HandRetargeting(HandType.INSPIRE_HAND_Unit_Test) + + if self.simulation_mode: + ChannelFactoryInitialize(1) + else: ChannelFactoryInitialize(0) # initialize handcmd publisher and handstate subscriber diff --git a/teleop/robot_control/robot_hand_unitree.py b/teleop/robot_control/robot_hand_unitree.py index 3061851..347d618 100644 --- a/teleop/robot_control/robot_hand_unitree.py +++ b/teleop/robot_control/robot_hand_unitree.py @@ -33,7 +33,7 @@ kTopicDex3RightState = "rt/dex3/right/state" class Dex3_1_Controller: def __init__(self, left_hand_array_in, right_hand_array_in, dual_hand_data_lock = None, dual_hand_state_array_out = None, - dual_hand_action_array_out = None, fps = 100.0, Unit_Test = False): + dual_hand_action_array_out = None, fps = 100.0, Unit_Test = False, simulation_mode = False): """ [note] A *_array type parameter requires using a multiprocessing Array, because it needs to be passed to the internal child process @@ -55,10 +55,15 @@ class Dex3_1_Controller: self.fps = fps self.Unit_Test = Unit_Test + self.simulation_mode = simulation_mode if not self.Unit_Test: self.hand_retargeting = HandRetargeting(HandType.UNITREE_DEX3) else: self.hand_retargeting = HandRetargeting(HandType.UNITREE_DEX3_Unit_Test) + + if self.simulation_mode: + ChannelFactoryInitialize(1) + else: ChannelFactoryInitialize(0) # initialize handcmd publisher and handstate subscriber @@ -259,7 +264,9 @@ class Gripper_Controller: else: self.smooth_filter = None - if self.Unit_Test: + if self.simulation_mode: + ChannelFactoryInitialize(1) + else: ChannelFactoryInitialize(0) # initialize handcmd publisher and handstate subscriber diff --git a/teleop/teleop_hand_and_arm.py b/teleop/teleop_hand_and_arm.py index 411ba11..4f04899 100644 --- a/teleop/teleop_hand_and_arm.py +++ b/teleop/teleop_hand_and_arm.py @@ -19,6 +19,7 @@ from teleop.robot_control.robot_arm import G1_29_ArmController, G1_23_ArmControl from teleop.robot_control.robot_arm_ik import G1_29_ArmIK, G1_23_ArmIK, H1_2_ArmIK, H1_ArmIK from teleop.robot_control.robot_hand_unitree import Dex3_1_Controller, Gripper_Controller from teleop.robot_control.robot_hand_inspire import Inspire_Controller +from teleop.robot_control.robot_hand_brainco import Brainco_Controller from teleop.image_server.image_client import ImageClient from teleop.utils.episode_writer import EpisodeWriter from sshkeyboard import listen_keyboard, stop_listening @@ -54,12 +55,13 @@ listen_keyboard_thread.start() if __name__ == '__main__': parser = argparse.ArgumentParser() parser.add_argument('--task_dir', type = str, default = './utils/data', help = 'path to save data') - parser.add_argument('--frequency', type = float, default = 90.0, help = 'save data\'s frequency') + parser.add_argument('--frequency', type = float, default = 60.0, help = 'save data\'s frequency') + # basic control parameters parser.add_argument('--xr-mode', type=str, choices=['hand', 'controller'], default='hand', help='Select XR device tracking source') parser.add_argument('--arm', type=str, choices=['G1_29', 'G1_23', 'H1_2', 'H1'], default='G1_29', help='Select arm controller') - parser.add_argument('--ee', type=str, choices=['dex3', 'gripper', 'inspire1'], help='Select end effector controller') - + parser.add_argument('--ee', type=str, choices=['dex3', 'gripper', 'inspire1', 'brainco'], help='Select end effector controller') + # mode flags parser.add_argument('--record', action = 'store_true', help = 'Enable data recording') parser.add_argument('--motion', action = 'store_true', help = 'Enable motion control mode') parser.add_argument('--headless', action='store_true', help='Enable headless mode (no display)') @@ -168,6 +170,13 @@ if __name__ == '__main__': dual_hand_state_array = Array('d', 12, lock = False) # [output] current left, right hand state(12) data. dual_hand_action_array = Array('d', 12, lock = False) # [output] current left, right hand action(12) data. hand_ctrl = Inspire_Controller(left_hand_pos_array, right_hand_pos_array, dual_hand_data_lock, dual_hand_state_array, dual_hand_action_array) + elif args.ee == "brainco": + left_hand_pos_array = Array('d', 75, lock = True) # [input] + right_hand_pos_array = Array('d', 75, lock = True) # [input] + dual_hand_data_lock = Lock() + dual_hand_state_array = Array('d', 12, lock = False) # [output] current left, right hand state(12) data. + dual_hand_action_array = Array('d', 12, lock = False) # [output] current left, right hand action(12) data. + hand_ctrl = Brainco_Controller(left_hand_pos_array, right_hand_pos_array, dual_hand_data_lock, dual_hand_state_array, dual_hand_action_array) else: pass @@ -228,7 +237,7 @@ if __name__ == '__main__': publish_reset_category(1, reset_pose_publisher) # get input data tele_data = tv_wrapper.get_motion_state_data() - if (args.ee == 'dex3' or args.ee == 'inspire1') and args.xr_mode == 'hand': + if (args.ee == 'dex3' or args.ee == 'inspire1' or args.ee == 'brainco') and args.xr_mode == 'hand': with left_hand_pos_array.get_lock(): left_hand_pos_array[:] = tele_data.left_hand_pos.flatten() with right_hand_pos_array.get_lock(): @@ -299,7 +308,7 @@ if __name__ == '__main__': current_body_action = [-tele_data.tele_state.left_thumbstick_value[1] * 0.3, -tele_data.tele_state.left_thumbstick_value[0] * 0.3, -tele_data.tele_state.right_thumbstick_value[0] * 0.3] - elif args.ee == "inspire1" and args.xr_mode == 'hand': + elif (args.ee == "inspire1" or args.ee == 'brainco') and args.xr_mode == 'hand': with dual_hand_data_lock: left_ee_state = dual_hand_state_array[:6] right_ee_state = dual_hand_state_array[-6:]