Cross-validate your machine-learning model with SageMaker and Step Functions

Cross-validate your machine-learning model with SageMaker and Step Functions

Cross-validate your machine-learning model with SageMaker and Step Functions. Cross-validation is a powerful technique to build machine learning models that perform well on unseen data. How to easily cross-validate a machine-learning model using several services of Amazon Web Services (AWS), including SageMaker, Step Functions, and Lambda.

Automatize cross-validated machine-learning training jobs on AWS infrastructure

Cross-validation is a powerful technique to build machine learning models that perform well on unseen data. However, it can also be time-consuming as it includes training multiple models. This post will show you how to easily cross-validate a machine-learning model using several services of Amazon Web Services (AWS), including SageMaker, Step Functions, and Lambda.

Why do you need cross-validation?

If you know the concept of cross-validation, feel free to jump directly to the section introducing SMX-Validator.

The problem of small datasets and sample distribution

Imagine the antelopes of the savanna entrust you to train an image classifier model that helps them recognize jaguars in a picture. They give you 50 photos of a jaguar and 50 photos of the savanna with no jaguars. You divide the dataset into a training set of 80 images and a test set of 20, taking care that there would be an equal number of jaguar and non-jaguar photos in each partition. You train your model with your favorite image classifier algorithm and get an impressive validation accuracy of 100%.

As a visual check, you look at some correctly classified photos in the test set:

Amur Leopard

Image by  Mark Murphy from  Pixabay

Everything looks good.

Sometime later, you retrain your model. You split the same dataset again into 80% train — 20% test sets, use the same hyperparameters that you used for the first model, and get a validation accuracy of 80%, with a couple of false negatives (lethal for the antelopes!). So what has happened?

step-functions aws serverless sagemaker crossvalidation

What is Geek Coin

What is GeekCash, Geek Token

Best Visual Studio Code Themes of 2021

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

ETL Orchestration on AWS with AWS Step Functions

Before being able to allow the figure of the data analyst to explore and visualize the data, a crucial step is needed. This procedure is commonly identified as ETL (extract, transform, and load) and, usually, it’s far from being simple. ETL Orchestration on AWS with AWS Step Functions.

Cutting Step-Functions Costs on Enterprise-Scale Workflows

AWS Step Functions is a great service for orchestrating multi-step workflows with complex logic. It’s fast to implement, relatively easy to use and just works. The problem is its price.

Combine AWS Step Functions with CloudWatch Events using aws-cdk

AWS Step Functions allow one to execute & coordinate long-running processes. Step Functions fall into serverless AWS services, and the…

How To Unite AWS KMS with Serverless Application Model (SAM)

AWS KMS is a Key Management Service that let you create Cryptographic keys that you can use to encrypt and decrypt data and also other keys. You can read more about it here.

AWS Introduces a New Workflow Studio for AWS Step Functions

AWS Introduces a New Workflow Studio for AWS Step Functions. Recently, AWS introduced a new Workflow Studio for its Step Functions offering. Through the AWS Management Console, developers can now access a visual builder to create Step Functions workflows.