A tutorial showing how to deploy SpaCy on a large scale.

Background:

SpaCy is one of my favourite NLP libraries. And I have been using spaCy to perform a lot of Named Entity Recognition (NER) tasks. Generally, we first need to load a spaCy pre-trained model of a specific language and fine-tune the model with our training dataset. The training process can be done offline with a local computer and we can even test the fine-tuned model performance by hosting it locally through Flask / Streamlit.

Although I have found many great tutorials on deploying a spaCy model locally with Flask / Streamlit, there are not many tutorials on how to deploy it on a larger scale, for example, how to deploy a spaCy model with AWS.

It’s a very interesting topic and after a great amount of work I summarized my solution into this article; hopefully it can be useful for people facing the same question.

Introduction:

In this article, I will explain my solution on how to deploy a custom spaCy model with AWS services including:

• AWS ECR (Elastic Container Registry)
• AWS SageMaker
• AWS Lambda
• AWS S3 Bucket (Optional)

Here is my plan🧗🏻:

• First, data input can be sent as an event into the AWS Lambda.
• Then, within the Lambda, we invoke a SageMaker endpoint. The endpoint takes the data input coming from the Lambda event and returns a response containing the results from the spaCy model.
• Finally, this endpoint response will display as an execution result. Hence we are able to check the response on the AWS Lambda result tab or debug on the AWS CloudWatch.

#aws #aws ecr #sagemaker #lambda