_All the code used in this article is _here
Recently, PyTorch has introduced its new production framework to properly serve models, called torchserve.
So, without further due, let’s present today’s roadmap:
To showcase torchserve, we will serve a fully trained ResNet34 to perform image classification.
_Official doc _here
The best way to install torchserve is with docker. You just need to pull the image.
You can use the following command to save the latest image.
docker pull pytorch/torchserve:latest
All the tags are available here
More about docker and torchserve here
_Official doc _here
Handlers are the ones responsible to make a prediction using your model from one or more HTTP requests.
Default handlers
Torchserve supports the following default handlers
image_classifier
object_detector
text_classifier
image_segmenter
But keep in mind that none of them supports batching requests!
Custom handlers
torchserve exposes a rich interface to do almost everything you want. An Handler
is just a class that must have three functions
You can create your own class or just subclassBaseHandler
. The main advantage of subclasssing BaseHandler
is to have the model loaded accessible at self.model
. The following snippet shows how to subclass BaseHandler
Subclassing BaseHandler to create your own handler
Going back to our image classification example. We need to
#pytorch #data-science #deep-learning #data analysis