Perception & Navigation
Combine learned walking, online lidar mapping, and planning to navigate unfamiliar mazes.
Result & current limits
The 22-DoF humanoid reached 12/12 fresh 6×6 maze goals with no falls or wall contacts using one qualified gait. A separate biped study reached 24/24 goals across 5×5 and 6×6 mazes. Both use scripted planning and oracle pose/goal in simulation; they do not establish hardware navigation.
Problem
A walking policy needs a route, a map, and commands it can execute. Navigation tests must distinguish reaching a goal from avoiding falls and wall contacts.
Solution
Built online occupancy mapping from simulated lidar, scripted A* planning, turn-then-walk commands, and a depth-based speed brake above a learned gait. Evaluated fresh maze layouts, sensor faults, timing, and a learned navigation alternative. The featured maze system uses the simulator's true pose and goal; its stereo, optical-flow, and IMU panels are display-only.
What I built
Built the mapping and planning stack, sensor instrumentation, command filters, and matched evaluation protocols. Separated learned control, scripted planning, and oracle inputs so each result states what the system actually uses.
Technologies
More demos, engineering decisions, and evaluation notes
Mapping and planning on the same control interface
The robot integrates lidar returns into a log-odds occupancy grid and replans with A*. A reactive depth brake slows translation near obstacles or stale packets. The learned gait receives forward-velocity and yaw-rate commands. The simulator supplies localization and the goal coordinate. System and maze protocol.
Compare outcomes on matched runs
A separate October 8 command-filter study recorded 259/288 goals versus 247/288 for the baseline, with 0 versus 9 falls, across three actors. Wall contacts remain, and shared layouts are clustered observations. This is a different study from the 12-layout humanoid demonstration. Confirmation report.
Sensor accuracy and navigation success are separate tests
Sensor studies cover packet loss, pose errors, IMU delay, native estimators, and processing schedules. In the October 12 scheduling study, LiDAR–IMU clean goals changed from 0/12 to 11/12, while stereo remained 0/12 under both schedules. The report retains timing, execution-order, and validation limits. Schedule results.
The matched IMU-history and stereo-pose studies also retain negative findings. Better sensor measurements alone do not establish better closed-loop navigation.
A 20 ms IMU delay halves success
In an Isaac Lab route test, four policies with different sensor inputs completed 124 of 128 first episodes. Delaying the IMU by one 20 ms control step cut that to 61/128, and 40 ms cut it to 3/128. Fine-tuning one policy for 2,000 iterations with randomized sensor dropout, IMU bias and up to 20 ms of IMU delay let it complete 29/32 at 40 ms, against 0/32 for the untouched policy. The route is fixed and a blind policy also completes it, so this measures tolerance to a delayed IMU, not perception. Delay probe · Fine-tune and control.
A learned navigator in place of the planner
A PPO navigation policy replaced A* and sent velocity and turn-rate commands to the same frozen biped gait. Three trained navigators reached 35 of 36 goals on 12 unseen mazes with no falls; scripted A* reached 12/12. A predeclared 48-maze confirmation then failed its zero-fall target. The navigators reached 44, 46 and 44 of 48 goals (A* 44/48), but 6 falls occurred across the nominal and 35% lidar-dropout runs. Pose and goal come from the simulator, and a scripted speed brake stays in the loop. First transfer · 48-maze confirmation.
Newer recovery audit: more goals, fewer clean completions
The October 14 publication audits 1,152 episodes across four controllers, three fixed navigation actors, two sensing conditions, and 48 shared layouts. Budgeted recovery reaches 288/288 goals versus 286/288 for the existing recovery reference, with zero falls in both. Clean goals decrease from 282 to 276, and contact episodes increase from 5 to 12.
This is a tradeoff from scripted recovery around a learned navigation policy, with oracle pose/goal and no measured world-model influence. It is separate from the 22-joint maze demonstration and native estimated-pose studies. The overall campaign remains incomplete because terrain confirmation failed its asset-matching checks. Dated audit and reproducible outcome counts.
Related work: Locomotion & Control and Language-Guided Manipulation.