Cross-validate your machine-learning model with SageMaker and Step Functions. Cross-validation is a powerful technique to build machine learning models that perform well on unseen data. How to easily cross-validate a machine-learning model using several services of Amazon Web Services (AWS), including SageMaker, Step Functions, and Lambda.
Cross-validation is a powerful technique to build machine learning models that perform well on unseen data. However, it can also be time-consuming as it includes training multiple models. This post will show you how to easily cross-validate a machine-learning model using several services of Amazon Web Services (AWS), including SageMaker, Step Functions, and Lambda.
If you know the concept of cross-validation, feel free to jump directly to the section introducing SMX-Validator.
Imagine the antelopes of the savanna entrust you to train an image classifier model that helps them recognize jaguars in a picture. They give you 50 photos of a jaguar and 50 photos of the savanna with no jaguars. You divide the dataset into a training set of 80 images and a test set of 20, taking care that there would be an equal number of jaguar and non-jaguar photos in each partition. You train your model with your favorite image classifier algorithm and get an impressive validation accuracy of 100%.
As a visual check, you look at some correctly classified photos in the test set:
Image by Mark Murphy from Pixabay
Everything looks good.
Sometime later, you retrain your model. You split the same dataset again into 80% train — 20% test sets, use the same hyperparameters that you used for the first model, and get a validation accuracy of 80%, with a couple of false negatives (lethal for the antelopes!). So what has happened?
Before being able to allow the figure of the data analyst to explore and visualize the data, a crucial step is needed. This procedure is commonly identified as ETL (extract, transform, and load) and, usually, it’s far from being simple. ETL Orchestration on AWS with AWS Step Functions.
AWS Step Functions is a great service for orchestrating multi-step workflows with complex logic. It’s fast to implement, relatively easy to use and just works. The problem is its price.
AWS Step Functions allow one to execute & coordinate long-running processes. Step Functions fall into serverless AWS services, and the…
AWS KMS is a Key Management Service that let you create Cryptographic keys that you can use to encrypt and decrypt data and also other keys. You can read more about it here.
AWS Introduces a New Workflow Studio for AWS Step Functions. Recently, AWS introduced a new Workflow Studio for its Step Functions offering. Through the AWS Management Console, developers can now access a visual builder to create Step Functions workflows.