Fig 1: A UR5 on a differential-drive base moves through a cluttered environment while tracking a monitor with its wrist-mounted camera. Top: the robot's trajectory (fading right to left) as it maintains a high detection rate. Bottom: camera views for the configurations shown above.

Fig 2: The camera model: line of sight (orange) and camera frustum (purple), shown against a cup intersecting each. These two visibility constraints define the manifold within the robot's free configuration space on which PS-PRM samples.

Fig 3: The PS-PRM pipeline. Bottom: environment with candidate configurations. Middle: PRM generation, neural surrogate scoring, and occlusion checking run in parallel to select the optimal path. Top: camera views confirming differing perception scores.

Fig 4: Perception-aware evaluation via neural surrogate model and occlusion-aware ray tracing. Left to right: environment with target (human) and occluder (orange box); surrogate-predicted perception score field; ray-traced occlusion refinement; camera views at points A and B, showing lower scores under occlusion.

Fig 5: Parallel perception-score evaluation for a batch of robot configurations. Camera poses are computed via forward kinematics; occlusion checking and neural surrogate scoring run concurrently; outputs combine into a final perception score (0–1) per configuration. Batching this pipeline on the GPU is what enables online replanning in dynamic scenes.

Fig 6: Example PS-PRM trajectory in the simulated home environment. Robot traces and camera images are sampled along the path, fading in as the robot nears the goal. Snapshots show YOLO confidence scores and ground-truth bounding boxes. The planner selects a trajectory that stays slightly farther from the monitored object, trading a marginally longer path for a better viewing angle.

Fig 7: Qualitative comparison in the simulated home environment. Each row shows one method's trajectory (left) with the camera views sampled along it (right): Post keeps the TV in view only intermittently, Manifold-frustum improves coverage but at low confidence, and PS-PRM-line maintains detection with consistently higher confidence scores throughout the motion.

Fig 8: Simulated nursing environment: the robot plans start-to-goal while tracking both masked nurses. Manifold-line takes a shorter path (orange) that passes behind the nurses and yields poor visibility; Neural-PS-PRM-line takes a longer, more cluttered path (blue) that keeps both faces in view.

Fig 9: Distance-based priority under conflicting perception objectives: with the two nurses spatially separated, high-quality perception of both at once is infeasible, so the Stretch robot shifts attention from Nurse 1 (green) to Nurse 2 (blue) as it moves left to right.

Fig 10: Planning through a narrow passage: the distance-based priority metric shifts which object the camera focuses on as the robot crosses, while sampling-based planning solves the narrow-passage motion.

Fig 11: Dynamic planning for human face detection: as the human rotates, the Stretch 2 robot replans from the shelf (Goal 1, orange) to the cabinet (Goal 2, blue) to preserve visibility. Neural-PS-PRM-line's online replanning achieves much higher detection rates than static planning.

Fig 12: Real-robot cup-detection experiment: motion paths for Manifold-line (orange) and PS-PRM-line (blue), with camera views for each shown left and right. The left path is shorter but the cup is occluded by the monitor; PS-PRM-line takes the longer route around the table and keeps the cup detected in significantly more frames.

Fig 13: Real-robot face-detection experiment in a nurse training scenario: as the nurse rotates, the Stretch 2 robot replans from the left cabinet (Goal 1, orange) to the right cabinet (Goal 2, blue) to preserve visibility, using Neural-PS-PRM-line's online replanning.

Fig 14: Dynamic human, obstacle, and goal movement: the Stretch 2 robot (blue) switches paths as the person rotates (B), replans around a moved obstacle (C), and replans again as the goal cabinet moves (D). The planner reaches execution rates of up to 2 Hz and generates an average of 8 trajectories per run, more than the 3 environment changes.

Fig 15: Perception-constrained planning for human activity recognition on a 6-DoF UR5. Without perception costs (left, short path), the person is largely occluded by the wooden board and the drinking action is missed; the Neural-PS-PRM trajectory (right) detours above the board to maintain visibility and recovers the full activity timeline. Trajectories are dotted, red when occluded and green when visible.