A/B Testing Machine Learning Models in Production

A/B Testing Machine Learning Models in Production

In this guide, I want to explain both the how and why of our approach, and hopefully, give you a better way to use A/B testing machine for test your models in production.

There is (rightfully) quite a bit of emphasis on testing and optimizing models pre-deployment in the machine learning ecosystem, with meta machine learning platforms like Comet becoming a standard part of the data science stack.

There has been less of an emphasis, however, on testing and optimizing models post-deployment, at least as far as tooling is concerned. This dearth of tooling has forced many to build extra in-house infrastructure, adding yet another bottleneck to deploying to production.

We’ve spent a lot of time thinking about A/B testing deployed models in Cortex, our open source ML deployment platform. After several iterations, we’ve built a set of features that make it easy to conduct scalable, automated A/B tests of deployed models. In this guide, I want to explain both the how and why of our approach, and hopefully, give you a better way to test your models in production.


What is Geek Coin

What is GeekCash, Geek Token

Best Visual Studio Code Themes of 2021

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

How to Extend your DevOps Strategy For Success in the Cloud?

DevOps and Cloud computing are joined at the hip, now that fact is well appreciated by the organizations that engaged in SaaS cloud and developed applications in the Cloud. During the COVID crisis period, most of the organizations have started using cloud computing services and implementing a cloud-first strategy to establish their remote operations. Similarly, the extended DevOps strategy will make the development process more agile with automated test cases.

What Is DevOps and Is Enterprise DevOps Any Good?

What is DevOps? How are organizations transitioning to DevOps? Is it possible for organizations to shift to enterprise DevOps? Read more to find out!

DevOps Basics: What You Should Know

What is DevOps? What are the goals it helps achieves? What are its benefits? This article has answers!

Why You Should Be Adopting a DevOps Culture in 2020 - DZone DevOps

The year 2020 has arrived, and its arrival brings a lot of innovations and transformations in the Information and Technology (IT) sector to DevOps technologies.

Measuring DevOps Metrics: A How-To Guide

DevOps is supposed to help streamline the process of taking code changes and getting them to production for users to enjoy. But what exactly does it mean for the process to be "streamlined"? One way to answer this is to start measuring metrics.