MLOps: Building Continuous Training and Delivery Pipelines

MLOps: Building Continuous Training and Delivery Pipelines

MLOps: Building Continuous Training and Delivery Pipelines. Learn how to get started with building robust, automated ML pipelines for automatically retraining, tracking and redeploying your models.

What you’ll learn

This post aims to help you get started with building robust, automated ML pipelines (on a budget!) for automatically retraining, tracking and redeploying your models. It covers:

  • An overview of the origins and aims of the MLOps movement;
  • An introduction to a couple of key MLOps concepts;
  • A tutorial for setting up a Continuous Training/Continuous Delivery (CT/CD) ML pipeline with GitHub Actions and Google Cloud Functions.

The tutorial section is designed to make use of free (or _nearly _free) services, so following along should cost you a few pennies at most. If you’re working on an MVP and need some ML infrastructure in place sharpish but want to avoid the price tag and technical overhead of AWS SageMaker or Azure ML deployments, you might find the example useful too. Finally, if you’re interested in understanding how the tutorial fits together to run it end-to-end for yourself, you should check out the previous post in this series on deploying lightweight ML models as serverless functions.

If you know what ML Ops is all about and just want to follow the tutorial, feel free to skip ahead.

What is MLOps?

In the last decade or so, the movement popularly referred to as ‘DevOps’ has gained a significant professional following within the world of software engineering, with a large number of dedicated DevOps roles springing up across development teams around the world. The motivation for this movement is to combine aspects of software development (Dev) with elements of Operational (Ops) software activities with the aim of accelerating the delivery of reliable, working software on an ongoing basis.

A major focus for adherents of the DevOps movement is on establishing and maintaining Continuous Integration and Continuous Delivery (CI/CD) pipelines. In practice, well designed and cleanly implemented CI/CD pipelines offer teams utilising them the ability to continuously modify their software system to (in principle) dramatically reduce the time-to-value for new software patches and features, while simultaneously minimising the risk of downside from bugs and outages related to releasing these patches and features. Teams operating mature implementations of this delivery mechanism often release updates on an hourly basis (or faster!) with the ability to quickly and cleanly rollback changes if they introduce a bug (though most of these should be caught somewhere in the pipeline).

In contrast, ‘traditional’ approaches to releasing software essentially stockpile fixes and features for predefined release windows (perhaps on a weekly, monthly or quarterly basis). While this sort of approach is not uniformly a poor approach, it does introduce a lot of pressure around the release window, can create a lot of complexity around the product integration and release process, and ultimately heighten the risk of serious service outages and by extension brand damage.

mlops serverless gcp data-science machine-learning

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Most popular Data Science and Machine Learning courses — July 2020

Most popular Data Science and Machine Learning courses — August 2020. This list was last updated in August 2020 — and will be updated regularly so as to keep it relevant

15 Machine Learning and Data Science Project Ideas with Datasets

Learning is a new fun in the field of Machine Learning and Data Science. In this article, we’ll be discussing 15 machine learning and data science projects.

Data Science Pull Requests — A Method for Data Science Review & Merging

Data Science Pull Requests — A Method for Data Science Review & Merging. A step forward for MLOps and unlocking Open Source Data Science

Best Free Datasets for Data Science and Machine Learning Projects

This post will help you in finding different websites where you can easily get free Datasets to practice and develop projects in Data Science and Machine Learning.

50 Data Science Jobs That Opened Just Last Week

Data Science and Analytics market evolves to adapt to the constantly changing economic and business environments. Our latest survey report suggests that as the overall Data Science and Analytics market evolves to adapt to the constantly changing economic and business environments, data scientists and AI practitioners should be aware of the skills and tools that the broader community is working on. A good grip in these skills will further help data science enthusiasts to get the best jobs that various industries in their data science functions are offering.