ChatGPT Robot Manipulation Prompts: Human-Robot Task Execution
This repository provides a set of prompts that can be used with OpenAI's ChatGPT to enable natural language communication between humans and robots for executing tasks. The prompts are designed to allow ChatGPT to convert natural language instructions into a sequence of executable robot actions, with a focus on robot manipulation tasks. The prompts are easy to customize and integrate with existing robot control and visual recognition systems. For more information, please see our blog post and our paper, ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application.
How to use
🚀 New Feature Alert: We've updated the prompts to support the OpenAI's official API. Additionally, we've updated the prompts to support the latest version of the Azure OpenAI's API (as of September 2023).
We provide sample codes for using ChatGPT through Azure OpenAI and OpenAI API. Fill in the secrets.json with your credential information. Even if you do not have a subscription, you can try it out by copying and pasting the prompts into the OpenAI's interface.
If you have a subscription of Azure OpenAI or OpenAI, install the required python packages by running the following command in a terminal session (note: we have confirmed that the sample codes work with python 3.9.16):
Then, go to a subfolder in examples/ (for example, examples/task_decomposition), run the following command to run the sample code:
Replace <scenario_name> with the name of the scenario you want to run. Specific scenario names can be found in the aimodel.py.
Bibliography
@article{10235949,
author={Wake, Naoki and Kanehira, Atsushi and Sasabuchi, Kazuhiro and Takamatsu, Jun and Ikeuchi, Katsushi},
journal={IEEE Access},
title={ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application},
year={2023},
volume={},
number={},
pages={1-1},
doi={10.1109/ACCESS.2023.3310935}}Trademarks
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.
python aimodel.py --scenarios <scenario_name>
> pip install -r requirements.txt
Download Details:
Author: Microsoft
Official Github: https://github.com/microsoft/ChatGPT-Robot-Manipulation-Prompts
License: MIT
