Autonomous Detection and Control of Human Tools for Robot Manipulation
Robots that can manipulate everyday objects in unstructured, human settings could more easily work with people and perform tasks that are important to people. Ideally, a robot would be able to detect and control task-relevant visual features of an object it has not previously encountered, since the set of objects found within human environments is large and diverse. For a significant set of tasks and tool-like objects, detection and control of the distal end of the object is sufficient for its use. The tips of these objects, such as the end of a screwdriver or the mouth of a bottle, have an approximately convex projection into the visual scene. In this paper we address the problem of visually detecting and controlling the tip of an unknown tool-like object that is rigidly grasped by a robot. We present a multi-scale motionbased feature detector that visually detects a tool tip and we show results in using a series of these 2D detections to produce a 3D estimate of the tool tip’s position in the hand’s frame. We also describe and evaluate a method for controlling the tool tip’s position and orientation in the image. 1