What's New

• [10/10/2024] We open-sourced the LABOR implementation on the NICOL robot in Coppeliasim.
• [9/09/2024] Our paper has been accepted by Humanoids 2024 .
• [3/28/2024] Initial release of LABOR Agent.

Abstract

Although there has been rapid progress in endowing robots with the ability to solve complex manipulation tasks, generating control policies for bimanual robots to solve tasks involving two hands is still challenging because of the difficulties in effective temporal and spatial coordination. With emergent abilities in terms of step-by-step reasoning and in-context learning, Large Language Models (LLMs) have demonstrated promising potential in a variety of robotic tasks. However, the nature of language communication via a single sequence of discrete symbols makes LLM-based coordination in continuous space a particular challenge for bimanual tasks. To tackle this challenge, we present LAnguage-model-based Bimanual OR (LABOR), an agent utilizing an LLM to analyze task configurations and devise coordination control policies for addressing long-horizon bimanual tasks. We evaluate our method through simulated experiments involving two classes of long-horizon tasks using the NICOL humanoid robot. Our results demonstrate that our method outperforms the baseline in terms of success rate. Additionally, we thoroughly analyze failure cases, offering insights into LLM-based approaches in bimanual robotic control and revealing future research trends.

Acknowledgements

The authors would like to thank OpenAI and their Researcher Access Program for generously providing GPT-4o API tokens support, and Jan-Gerrit Habekost and Lennart Clasmeier for their work and support for the NICOL robot in the simulation.

The website template was borrowed from Jon Barron.