To understand how machine learning models are performing, we need to go deeper than just reporting accuracy. Introducing behavioral tests: an approach to understanding the robustness and reasonableness of ML models before you deploy them!

In this GitHub Actions tutorial, we’ll explore how to write a basic behavioral test to check that an NLP sentiment classifier is robust to typos. We’ll use the GitHub Actions continuous integration system, plus CML and DVC, to automatically compare a new model’s robustness to typos to our current model, and report the comparison in a pull request.

Resources in this video
Project GitHub repository: https://github.com/elleobrien/typo_test
Jeremy Jordan blog: https://www.jeremyjordan.me/testing-ml/
Continuous Machine Learning (CML) project: https://github.com/iterative/cml

#github #mlops #machine-learning #data-science