Planning, Perception, and Control for a Personal Robot

HERB, the Home Exploring Robot Butler, is a bi-manual mobile manipulator designed and built by the Personal Robotics Lab at Carnegie Mellon University to complete household manipulation tasks. As a member of the Personal Robotics Lab, I developed many of the planning, perception, and control algorithms used by HERB to autonomously manipulate his environment.

I am the lead developer of HerbPy and PrPy, the core Python libraries used to control HERB. This software, developed in collaboration with my colleagues in the Personal Robotics Lab, spans the spectrum of robotics, including:

We have used this software to make HERB complete a variety of tasks, including: stocking a bookshelf, clearing a table, starring in a play, and completing a number of YCB tasks. Videos of HERB completing these tasks has been widely featured by the press, including National Geographic's Robots movie shown in IMAX theaters, an Oreo cookie advertisement, and an episode of Katie Couric's Cities Rising: Rebuilding America series. Several highlights are included below


HERB has used the software I developed to complete a variety of YCB tasks, including this block sorting task. This task highlights HERB's motion planning and post-processing pipelines. This task tasks place in a relatively open environment, so most motion planning queries are solved by HERB's heuristic motion planning algorithms.

Tight integration between HERB's high-level software and his controller allowed HERB to star in a play. I developed a controller plugin that allowed HERB to natively execute F-Curves, a type of Bezier curve produced by Blender's animation tools. This allowed HERB to faithfully reproduce the animator's intent.

HERB has completed a variety of multi-stage manipulation tasks, such as stocking a library, use all of HERB's capabilities. This task integrates all of the techniques that I have described above.