4.8 KiB

Raw Blame History

id	title	status	source_sections	related_topics	key_equations	key_terms	images	examples	open_questions
sensors-perception	Sensors & Perception	established	reference/sources/official-product-page.md, reference/sources/official-developer-guide.md	[hardware-specs locomotion-control sdk-programming safety-limits]	[]	[imu state_estimation realsense_d435i livox_mid360 tactile_sensor dual_encoder]	[]	[]	[IMU model and noise characteristics D435i resolution/FPS as configured on G1 specifically Force/torque sensor availability per variant]

Sensors & Perception

Sensor suite, specifications, and perception capabilities of the G1.

1. Sensor Suite Overview

Sensor Type	Model/Spec	Location	Data Type	Network/Interface	Tier
Depth Camera	Intel RealSense D435i	Head	RGB-D	USB, SDK access	T0
3D LiDAR	Livox MID360	Head	360° point cloud	Ethernet (192.168.123.20)	T0
IMU	Onboard IMU	Body (torso)	6-axis accel+gyro	Internal, via LowState	T0
Microphone Array	4-element array	Head	Audio (spatial)	Internal	T0
Speaker	5W stereo	Head	Audio output	Internal	T0
Joint Encoders	Dual encoders per joint	Each joint	Position + velocity	Internal, via LowState	T0
Temperature Sensors	Per-motor	Each actuator	Temperature	Internal, via LowState	T0
Tactile Sensors	33 per hand (Dex3-1)	Fingertips	Force/touch	Via Dex3-1 DDS topic	T0

[T0 — Official product page and developer guide]

2. Depth Camera — Intel RealSense D435i

The primary visual sensor for RGB-D perception, obstacle detection, and environment mapping.

Standard D435i specs: 1280x720 @ 30fps (depth), 1920x1080 @ 30fps (RGB), 87° x 58° FOV (depth)
Note: G1-specific configuration (resolution, FPS) may differ from standard D435i defaults — see open questions
Mounting: Head-mounted, forward-facing
Access: Via RealSense SDK on development computer, or via SDK2 camera API
Research: Used for end-to-end vision-based locomotion (arXiv:2602.06382) with depth augmentation [T1]

3. 3D LiDAR — Livox MID360

360-degree environment scanning for navigation, SLAM, and obstacle avoidance.

Horizontal FOV: 360°
Vertical angle: 59°
Network: Ethernet at 192.168.123.20
Access: Requires separate Livox driver setup (unilidar_sdk2)
Use cases: SLAM, navigation, environment mapping, point cloud generation

4. IMU (Inertial Measurement Unit)

Critical for balance and state estimation during locomotion.

Type: 6-axis (3-axis accelerometer + 3-axis gyroscope)
Access: Available in rt/lowstate → imu_state field
Used by: Locomotion computer for real-time balance control, state estimation
User access: Via SDK2 LowState subscription

5. Joint Proprioception

Dual encoders per joint provide high-accuracy feedback:

Position feedback: Absolute joint angle (rad)
Velocity feedback: Joint angular velocity (rad/s)
Access: rt/lowstate → motor_state array
Rate: Updated at 500 Hz (control loop frequency)

6. Perception Pipeline

Sensors → Locomotion Computer → DDS → Development Computer → User Applications
   │              │
   │              └── IMU + encoders → state estimation → balance control (internal)
   │
   ├── D435i → RealSense SDK → RGBD frames → perception/planning
   ├── MID360 → Livox driver → point cloud → SLAM/navigation
   └── Microphones → audio processing → voice interaction

The locomotion computer handles proprioceptive sensing (IMU, encoders) internally for real-time control. Exteroceptive sensors (cameras, LiDAR) are processed on the development computer. [T1 — Inferred from architecture]

7. State Estimation

The locomotion computer fuses IMU and joint encoder data for real-time state estimation: [T2 — Observed from RL papers]

Body orientation (roll, pitch, yaw) from IMU
Angular velocity from IMU gyroscope
Joint positions and velocities from encoders
Contact state inference for foot contact detection

This state estimate feeds directly into the RL-based locomotion policy at 500 Hz.

Key Relationships

Mounted on: hardware-specs
Feeds into: locomotion-control (balance, terrain adaptation)
Accessed via: sdk-programming (sensor data API)
Network config: networking-comms

4.8 KiB Raw Blame History