Everyone has experienced looking up a recipe online then having to scroll through the recipe’s back story to reach the end of the page just to find out how to poach chicken. Being an avid cook I began thinking about how a chatbot could take a user request about how to cook a specific food and respond with some quick cooking advice without all the filler.
One challenge of doing so is generating the advice itself. There are a lot of potential cooking questions users could ask and building that knowledge base is an enormous task. When OpenAI released GPT-3 earlier this summer I began to think how GPT-3 could be leveraged to provide cooking advice for any food users ask about. The result is a chatbot that takes user cooking queries and then relies on OpenAI’s GPT-3 language model to generate the content!
In this post I will walk through how Twilio Autopilot and OpenAI together can help provide all kinds of cooking advice. Users will be able to ask their cooking related questions and Autopilot will collect the input and pass it to OpenAI, which will generate a response like the examples in the image below.
Follow along and let’s start cooking!
Tutorial requirements
- You will need a Twilio account (this link gets you a free $10 to get started!)
- An OpenAI API key
- Install the Twilio CLI tool and the Autopilot CLI plug-in, which will be used to deploy our Autopilot bot.
Setup the Python virtual environment
The first thing we’re going to do is create a virtual environment to work in. For this tutorial we will create a directory called twilio-openai-cooking for our code and other artifacts. To create the project directory and virtual environment for the tutorial open a terminal window and enter the following commands:
mkdir twilio-openai-cooking
cd twilio-openai-cooking
python3 -m venv venv
source venv/bin/activate
pip install openai flask python-dotenv pyngrok
If you are using Windows you can use the following:
md twilio-openai-cooking
cd twilio-openai-cooking
python -m venv venv
venv\Scripts\activate
pip install openai flask python-dotenv pyngrok
Building the Autopilot bot
To interface with our bot we’re going to use Twilio Autopilot. Autopilot helps manage the user conversation by interpreting the user’s request and associating the intent with a task that will fulfill the intent. For our bot the main task is going to listen for cooking related questions. When cooking related questions are identified, Autopilot will route those requests to a Flask endpoint we will create that will make a call to the GPT-3 API to get the cooking instructions. GPT-3 works best when there are prompts to seed the interaction, so having Autopilot manage the conversation so that only cooking related questions trigger the GPT-3 task helps ensure the user input roughly matches the prompts we send to GPT-3 to generate reasonable outputs.
The behavior for a Twilio Autopilot bot is defined in a JSON schema file. The schema file defines the tasks your bot will accomplish, where each task can perform a number of different actions. You may want to check out the Twilio Autopilot Templates GitHub to see a collection of schema templates to get you started on a variety of different use cases!
Within the project directory create a new file called schema.json and populate the file with the following:
{
"friendlyName": "cooking_with_gpt3",
"logQueries": true,
"uniqueName": "cooking_with_gpt3",
"defaults": {
"defaults": {
"assistant_initiation": "task://welcome_message",
"fallback": "task://fallback"
}
},
"styleSheet" : {
"style_sheet" : {
"voice": { "say_voice": "Polly.Matthew" }
}
},
"fieldTypes": [
{
"uniqueName": "Custom.METHOD", "values": [
{ "language": "en-US", "value": "cook", "synonymOf": null },
{ "language": "en-US", "value": "roast", "synonymOf": null },
{ "language": "en-US", "value": "bake", "synonymOf": null },
{ "language": "en-US", "value": "grill", "synonymOf": null },
{ "language": "en-US", "value": "poach", "synonymOf": null }
]
},
{
"uniqueName": "Custom.FOOD", "values": [
{ "language": "en-US", "value": "chicken", "synonymOf": null },
{ "language": "en-US", "value": "eggs", "synonymOf": null },
{ "language": "en-US", "value": "salmon", "synonymOf": null },
{ "language": "en-US", "value": "broccoli", "synonymOf": null },
{ "language": "en-US", "value": "chicken breast", "synonymOf": null },
{ "language": "en-US", "value": "asparagus", "synonymOf": null },
{ "language": "en-US", "value": "zucchini", "synonymOf": null },
{ "language": "en-US", "value": "squash", "synonymOf": null },
{ "language": "en-US", "value": "steak", "synonymOf": null }
]
}
],
"tasks": [
{
"uniqueName": "fallback",
"actions": {
"actions": [
{ "say": "Hmmmm it doesn't look like you have asked about anything cooking related. Please try again." },
{ "listen": true }
]
},
"fields": [],
"samples": []
},
{
"uniqueName": "cooking_tips",
"actions": {
"actions": [
{ "say": "Your cooking advice will be here once we setup our Flask app!" }
]
},
"fields": [
{ "uniqueName": "food", "fieldType": "Custom.FOOD" },
{ "uniqueName": "method", "fieldType": "Custom.METHOD" }
],
"samples": [
{ "language": "en-US", "taggedText": "{method} {food}" },
{ "language": "en-US", "taggedText": "{food}" },
{ "language": "en-US", "taggedText": "how do i {method} {food}" },
{ "language": "en-US", "taggedText": "tell me how to {method} {food}" },
{ "language": "en-US", "taggedText": "how can i {method} {food}" },
{ "language": "en-US", "taggedText": "how long does {food} have to {method}" }
]
},
{
"uniqueName" : "welcome_message",
"actions" : {
"actions" : [
{ "say" : "Hi! Need help cooking? I'm here to help! \n\nYou can ask me things like ' roast asparagus' or 'grill hamburgers'.\n\nTry it out with your favorite foods!" },
{ "listen" : true }
]
},
"fields": [],
"samples": [
{ "language": "en-US", "taggedText": "Hello there" },
{ "language": "en-US", "taggedText": "Whats up" },
{ "language": "en-US", "taggedText": "Heyo" },
{ "language": "en-US", "taggedText": "Hi there" },
{ "language": "en-US", "taggedText": "Yo" },
{ "language": "en-US", "taggedText": "Hey" },
{ "language": "en-US", "taggedText": "How's it going" },
{ "language": "en-US", "taggedText": "Hi" },
{ "language": "en-US", "taggedText": "Hello" }
]
}
],
"modelBuild": { "uniqueName": "V2" }
#code #tutorials and hacks #python
