# Phase 3: GR00T-WBC Deployment on GB10

**Date:** 2026-02-14
**Goal:** Deploy and validate GR00T-WBC whole-body control framework on Dell Pro Max GB10 for G1 humanoid simulation

---

## Summary

Successfully deployed NVIDIA's GR00T-WholeBodyControl on the GB10 (aarch64, Ubuntu 24.04, Blackwell GPU). Identified and fixed 4 critical blockers. Achieved working simulation with walking robot controlled via keyboard. Established remote visualization via NoMachine and web-based MJPEG streaming.

## Key Achievements

1. **GR00T-WBC running in simulation** — Both Balance and Walk ONNX policies loaded and executing at 50 Hz
2. **Walking robot** — WASD keyboard control verified: forward, backward, strafe, rotate
3. **Performance validated** — 3.5 ms/iteration (17.5% of 20ms budget) in sync mode
4. **Remote visualization** — NoMachine virtual desktop + web MJPEG viewer
5. **4 critical bugs identified and fixed** on aarch64 platform

## Critical Fixes

### Fix 1: CycloneDDS aarch64 Buffer Overflow (ROOT CAUSE)
- **File:** `unitree_sdk2py/core/channel_config.py`
- **Problem:** `<Tracing>` XML section triggers glibc FORTIFY_SOURCE buffer overflow specifically on ARM64
- **Discovery:** Isolated via `Domain(0)` (works) vs `Domain(0, config_with_tracing)` (crashes) vs `Domain(0, config_without_tracing)` (works)
- **Fix:** Remove `<Tracing>` section from both `ChannelConfigHasInterface` and `ChannelConfigAutoDetermine`
- **Impact:** Without this fix, nothing in the Unitree SDK works on aarch64

### Fix 2: ROS2 Python Package Path
- **File:** `.venv/lib/python3.12/site-packages/ros2.pth` (created)
- **Problem:** ROS2 packages at `/opt/ros/jazzy/lib/python3.12/site-packages/` not visible in venv
- **Fix:** Create `.pth` file with that path
- **Impact:** sensor_msgs, std_msgs, rclpy all accessible

### Fix 3: ROS2 Shared Library Path
- **Problem:** rclpy native libraries (librcl_action.so etc.) not found
- **Fix:** `export LD_LIBRARY_PATH=/opt/ros/jazzy/lib:$LD_LIBRARY_PATH`

### Fix 4: Sync Mode Sim Thread Check
- **File:** `run_g1_control_loop.py` line 211
- **Problem:** Code checks `env.sim.sim_thread.is_alive()` but in sync mode no thread exists
- **Fix:** Add `not config.sim_sync_mode` guard to the check

## Architecture Findings

### Pre-trained ONNX Policies
- **Balance:** 516-dim obs → 15-dim action (lower body joint targets)
- **Walk:** Same dimensions, velocity-conditioned locomotion
- **Training:** PPO in Isaac Lab, domain randomization, zero-shot sim-to-real (exact recipe not published)
- **Not mocap-based:** Command-conditioned (velocity tracking), not motion imitation

### Keyboard Control Architecture
- `KeyboardDispatcher` (raw) uses `sshkeyboard` library — fails in remote terminals
- `ROSKeyboardDispatcher` subscribes to `/keyboard_input` topic — works via separate publisher
- Dispatchers call `handle_keyboard_button()` on registered listeners (policy, env, e-stop)
- Walk policy: `]`=enable, `w/s`=fwd/back (+/-0.2 m/s per press), `a/d`=strafe, `q/e`=rotate, `z`=zero

### Visualization Challenges (Headless GB10)
Attempted 5 approaches for remote visualization:

| Approach | Result | Issue |
|----------|--------|-------|
| x11vnc on physical display :0 | Black screen | No monitor connected, GPU output disabled |
| x11vnc on Xvfb :99 | Static frame | OpenGL renders to own framebuffer, x11vnc reads X11 pixmap |
| NoMachine physical shadow | Black screen | Shadowing black physical display |
| NoMachine virtual desktop | Works but GLFW freezes | GLFW passive viewer thread stalls after seconds |
| Web MJPEG streaming | Best solution | Offscreen renderer → JPEG → HTTP, no X11 dependency |

### DDS/ROS2 Coexistence
- CycloneDDS system (0.10.4) and ROS2 (0.10.5) are ABI-incompatible
- FastRTPS is the default RMW in ROS2 Jazzy — avoids CycloneDDS version conflict
- Second ChannelFactory init from BaseSimulator fails gracefully (try/except)
- `CYCLONEDDS_URI` environment variable controls Unitree SDK's DDS config

## Software Installed on GB10

| Package | Purpose |
|---------|---------|
| GR00T-WBC | Whole-body control framework |
| NoMachine 9.3.7 | Remote desktop (NX protocol) |
| Sunshine 2025.924 | Game streaming (NVENC) — installed but Moonlight client unstable |
| x11vnc | VNC server — limited use (can't capture OpenGL) |
| Xvfb | Virtual framebuffer for headless rendering |
| openbox | Lightweight window manager for Xvfb |

## Files Created/Modified on GB10

| File | Action | Purpose |
|------|--------|---------|
| `channel_config.py` | Modified | Removed Tracing XML (buffer overflow fix) |
| `.venv/.../ros2.pth` | Created | ROS2 Python path |
| `run_g1_control_loop.py` | Modified | Sync mode sim thread guard |
| `launch_sim.sh` | Created | Convenience launcher |
| `launch_with_web_viewer.py` | Created | Launcher with web MJPEG streaming |
| `web_viewer.py` | Created | MuJoCo offscreen → MJPEG HTTP server |
| `/tmp/keysender.py` | Created | ROS keyboard publisher for remote control |
| `/etc/gdm3/custom.conf` | Modified | Auto-login enabled |
| `/etc/x11vnc.pass` | Modified | VNC password set |
| Firewall (ufw) | Modified | Opened ports 5900, 4000, 47984-47990, 8080 |

## Next Steps

1. **Get web viewer working** — test `launch_with_web_viewer.py` for smooth browser-based visualization
2. **Connect mocap** — webcam → pose estimation → upper body teleop commands
3. **Network bridge to G1** — connect GB10 to robot's 192.168.123.0/24 subnet
4. **Deploy to real robot** — switch `--interface sim` to `--interface real`
5. **Push recovery testing** — benchmark stock vs. GR00T-WBC push robustness
6. **Custom RL training** — use Isaac Lab to train custom policies with perturbation curriculum