Highlights

Title: Interaction and Learning for Autonomous Assembly Robots

Proposal being submitted by: Andrea L. Thomaz and Henrik Christensen School of Interactive Computing, Georgia Institute of Technology 85 5th Street NW, TSRB Atlanta, GA 30308

Date of submission of Proposal: September 9, 2008 Proposed duration of project: 24 months

Body of Proposal

1. Title: Interaction and Learning for Autonomous Assembly Robots

2. Proposal being submitted by: Andrea L. Thomaz and Henrik Christensen School of Interactive Computing, Georgia Institute of Technology

3. Abstract

The purpose of this project is to investigate the potential for feasible human cooperation with autonomous assembly robots that can perform useful, real work for assembly tasks. Human interaction, robot learning capability, flexible perception and cognitive behavior are to be integrated in a system that can be demonstrated in the role of “Co-Worker” for assembly tasks.

4. Introduction

A new next generation of assembly robot(s) is anticipated that significantly reduce the reliance on fixturing, mechanical structure and conventional sense-plan-act programming. This capability enables assembly automation with a set of little or no more infrastructure requirements than would a completely manual process. These new assembly robots would exploit the existence of a flexible robot perception system as an integral component of a three part strategy that includes highly flexible robots/dexterous end effectors and integration/harmony with people also performing related tasks in the assembly process. The cognitive component of the perception system would facilitate the “assignment” of the automation to a set of assembly tasks and/or assistance to others performing a task not yet appropriate for automation. This capability will also enable the rapid “teaching” and “reassignment” of the robot(s) to other tasks as required by production mix and business needs.

One of the objectives of this project is to leverage the existing work in the Socially Intelligent Machines research lab at Georgia Tech that can be applied toward human-robot interaction and collaboration at semantic level. Dr. Thomaz’s research on Socially Guided Machine Learning (previously with the MIT Leonardo robot, and now with the GT Simon robot), it’s behavioral architecture and related development are all regarded as an excellent starting point for this work. We anticipate that a new generation of highly capable assembly robot mechanical designs will evolve in the future. Given this capability, the feasibility of their use for autonomous assembly hinges on the availability of new solutions for behavior, learning and interaction.

5. Work plan / Tasks

One of the objectives of this project is to leverage the considerable existing work in the Socially Intelligent Machines research lab at Georgia Tech that can be applied toward human-robot interaction and collaboration at a semantic level. The existing visual and behavioral architecture are regarded as an excellent starting point for this activity.

This project seeks to adapt the current work and extend it into the direction of manufacturing relevant assembly tasks in a learning and interactive “Co-Worker” environment.

A capability demonstration is envisioned for showcasing the “co-worker” concept and challenging researchers in taking the existing state of development to the next level.

Some aspects of the capability demonstration will be reverse engineered from the existing physical performance capabilities of the robots at Georgia Tech that will be used for this research. We plan to use two platforms for this research. One is a mobile manipulator (segway RMP + Kuka lightweight arm + Schunk hand). The other is an upper-torso humanoid robot (two arms, two hands, socially expressive head).

Speed, payload, force levels, grasp and perception performance would all be considered in engineering the assembly tasks and components within a 3σ capability range. The “parts”, “tools” and tasks all should be recognizable as useful in an automotive context, but could be adapted for use by each robot platform. The domain of door-assembly has been chosen has the initial domain for task analysis and development of basic competencies. An array of tasks will be used to build a non trivial set of assembly sequences anywhere within the work envelope of the robot system.

The demonstration would include interaction and teaching multiple sequences of operations and having the system re-execute the tasks with variability in initial conditions, part/task locations, and interaction. The following is a set of ideas and potential requirements to be considered in the formulation of a formal project proposal:

• Non-trivial assembly: 5-10 parts, presented different locations/orientations or by hand, combined by insertion, snap fit, twist-lock, slots, Velcro, other forms of fasteners and using 3 different tools to set the orientation of part features or set fasteners with a total of at least 50 different combinations of possible outcomes (not counting all of these occurring at various positions). Parts could be a combination of prismatic, planar, rigid or flexible/compressible. An important aspect of the co-worker robot is that some assembly tasks may not be able to be done completely autonomously, it may be the case that for some tasks there are actions the robot can do, and other actions that it relies on the human partner to do.

• Behavior and learned task build: Teaching or learning a task should build a dynamic list of interpreted commands and behaviors that could represent the tasks. This list (or other structure) could possibly be edited by the teacher or used to reverse the task sequence and indicate future intention. Another possible use of this behavior structure is export for use by another robot that is working similar tasks. It’s envisioned that there should be a smooth transition from task element to element and the system should reason through unexpected contacts and retry operations.

• Interaction: The learning process will be situated within a social interaction with a human partner. Thus the system should be able to recognize and respond to human gestures, behavior, spoken commands and other cues related to the teaching/learning interaction. Teleoperation and/or physical lead-through are possibilities for teaching or “coaching” the system into performing the tasks. Interaction could be human to robot, hand to hand, robot to robot, and robot to supporting automation. A dynamic, icon based display or other creative audio-visual-haptic method (ranging from mild to wild) could be used to indicate the current state of learning and intention to proceed with future states or tasks. Various types of help could be requested to facilitate learning or executing the tasks. One goal would be to include task familiarization and optimization such that the system improves its performance with experience in a way that is understandable and trusted by the human partner. In the execution phase of a learned task, and important part of the interaction will be the negotiation of roles. For example, in cases where the robot does part of the task and then needs the human’s assistance to finish the task.

• Sensing and Perception: Should be flexible enough to accommodate point of assy. and tool use with relative positioning for parts and tools in the grasp, location of intersection points for insertion and tool tip guidance. Force and tactile data and responses should be integrated into the various behaviors.

• Safety: The system should exhibit behavior and situational awareness of human intention for safe movement response and task execution.

• Architecture: It is anticipated that the enhancements to the existing algorithms and behavior subsystems could extend beyond the demonstration work and into future work with next generation robot hardware.