MLOps Tutorial #3: Track ML Models with Git & GitHub Actions

In this tutorial, we’ll compare ML models across two different Git branches of a project- and we’ll do it in a continuous integration system (GitHub Actions) for automation superpowers! We’ll cover:

• Why comparing model metrics takes more than a git diff
• How pipelines, a method for making model training more reproducible, help you standardize model comparisons across Git branches
• How to display a table comparing model performance to the main branch in a GitHub Pull Request

Helpful links:
Dataset: https://www.sciencedirect.com/science/article/pii/S2352340920303048
Code: https://github.com/elleobrien/farmer
DVC pipelines & metrics documentation: https://dvc.org/doc/start/data-pipelines#data-pipelines
CML project repo: https://github.com/iterative/cml

#mlops #github #machine-learning

What is GEEK

Buddha Community

MLOps Tutorial #3: Track ML Models with Git & GitHub Actions

In this tutorial, we’ll compare ML models across two different Git branches of a project- and we’ll do it in a continuous integration system (GitHub Actions) for automation superpowers! We’ll cover:

• Why comparing model metrics takes more than a git diff
• How pipelines, a method for making model training more reproducible, help you standardize model comparisons across Git branches
• How to display a table comparing model performance to the main branch in a GitHub Pull Request

Helpful links:
Dataset: https://www.sciencedirect.com/science/article/pii/S2352340920303048
Code: https://github.com/elleobrien/farmer
DVC pipelines & metrics documentation: https://dvc.org/doc/start/data-pipelines#data-pipelines
CML project repo: https://github.com/iterative/cml

#mlops #github #machine-learning

7 Best Practices in GIT for Your Code Quality

There is no doubt that Git plays a significant role in software development. It allows developers to work on the same code base at the same time. Still, developers struggle for code quality. Why? They fail to follow git best practices. In this post, I will explain seven core best practices of Git and a Bonus Section.

1. Atomic Commit

Committing something to Git means that you have changed your code and want to save these changes as a new trusted version.

Version control systems will not limit you in how you commit your code.

• You can commit 1000 changes in one single commit.
• Commit all the dll and other dependencies
• Or you can check in broken code to your repository.

But is it good? Not quite.

Because you are compromising code quality, and it will take more time to review code. So overall, team productivity will be reduced. The best practice is to make an atomic commit.

When you do an atomic commit, you’re committing only one change. It might be across multiple files, but it’s one single change.

2. Clarity About What You Can (& Can’t) Commit

Many developers make some changes, then commit, then push. And I have seen many repositories with unwanted files like dll, pdf, etc.

You can ask two questions to yourself, before check-in your code into the repository

1. Are you suppose to check-in all these files?
2. Are they part of your source code?

You can simply use the .gitignore file to avoid unwanted files in the repository. If you are working on more then one repo, it’s easy to use a global .gitignore file (without adding or pushing). And .gitignore file adds clarity and helps you to keep your code clean. What you can commit, and it will automatically ignore the unwanted files like autogenerated files like .dll and .class, etc.

#git basics #git command #git ignore #git best practices #git tutorial for beginners #git tutorials

Best Practices for Using Git

Git has become ubiquitous as the preferred version control system (VCS) used by developers. Using Git adds immense value especially for engineering teams where several developers work together since it becomes critical to have a system of integrating everyone’s code reliably.

But with every powerful tool, especially one that involves collaboration with others, it is better to establish conventions to follow lest we shoot ourselves in the foot.

At DeepSource, we’ve put together some guiding principles for our own team that make working with a VCS like Git easier. Here are 5 simple rules you can follow:

1. Make Clean, Single-Purpose Commits

Oftentimes programmers working on something get sidetracked into doing too many things when working on one particular thing — like when you are trying to fix one particular bug and you spot another one, and you can’t resist the urge to fix that as well. And another one. Soon, it snowballs and you end up with so many changes all going together in one commit.

This is problematic, and it is better to keep commits as small and focused as possible for many reasons, including:

• It makes it easier for other people in the team to look at your change, making code reviews more efficient.
• If the commit has to be rolled back completely, it’s far easier to do so.
• It’s straightforward to track these changes with your ticketing system.

Additionally, it helps you mentally parse changes you’ve made using git log.

#open source #git #git basics #git tools #git best practices #git tutorials #git commit

How to Create a Custom GitHub Actions Using JavaScript — Beginner Level

In this blog, we are going to learn how to create our own custom GitHub action using javaScript.

Prerequisite

• Basic JavaScript Knowledge
• Basic Git & GitHub Knowledge

About GitHub Actions

Automate, customize, and execute your software development workflows right in your repository with GitHub Actions. You can discover, create, and share actions to perform any job you’d like, including CI/CD, and combine actions in a completely customized workflow.

Types of Actions

There are three types of actions: Docker container actions, JavaScript actions, and composite run steps actions.

JavaScript Custom Action

Let’s create a Custom GitHub Action using JavaScript by creating a public repo, once the repo is created, we can clone it to our local machine using VS Code or GitPod. You need to have Node.js 12.x or higher and npm installed on your machine to perform the steps described here. You can verify the node and npm versions with the following commands in a VS Code or GitPod terminal.

node --version 
npm --version

#github #github-tutorial #github-actions #github-trend

Track ML Model Metrics across branches with Github Actions and Docker - MLOps Tutorial

In this video tutorial I demonstrate how to track Machine Learning or Deep Learning metrics across different Github branches. For example, if you have a master or main branch in your Github repository, and want to make some changes in your model/Python code without touching the main branch, the solution is provided in this video.

For this, we will use Continuous Machine Learning (CML) technique which is very popular and mostly used in MLOps (Machine Learning Operations) domain, and Github Actions which becomes more and more popular among Data Scientists and Developers (I am Python Developer).

As we have discussed in previous MLOps tutorial (https://morioh.com/p/3f80cb88f48e) CI-CD functionality delivered by DVC (Data Versioning Control) can bring the beneficial into this kind of tasks.

This combination of tools let you build your custom ML Pipeline which will run inside the Docker container within the Github Environment, where you can see the progress of all stages in your ML pipeline in real time.

So this tutorial shows the idea how you can to compare different metrics of your Data Science project across multiple Github branches. You can have as many branches as you want. The output of your ML pipeline should be some kind of metrics.json and any format additional files which you can to integrate into Pull requests comments as markdown Readme file very easily.

The content of the video:
0:00 - Main Idea
2:23 - #1. Machine Learning Pipeline for this project
3:13 - #2. Run ML Pipeline Locally
6:48 - #3. Install DVC Package
7:31 - #4. Set ML Pipeline Stages in terminal and YAML file
18:22 - #5. Push DVC to Github Repository
19:37 - #6. Build a Github Action to run ML Pipeline
25:58 - #7. Make changes in ML project code
27:48 - #8. Track changes in metrics across different Github branches

For this purpose we will be using:

• GitHub Actions: https://github.com/features/actions​

• CML: https://github.com/iterative/cml

• Github Repository: https://github.com/vb100/mlops-track-…

Subscribe: https://www.youtube.com/c/VytautasBielinskas/featured

#mlops #github #docker