Workflow Engine for Kubernetes

Workflow Engine for Kubernetes

Workflow Engine for Kubernetes. Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).

What is Argo Workflows?

Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).

  • Define workflows where each step in the workflow is a container.
  • Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a directed acyclic graph (DAG).
  • Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo Workflows on Kubernetes.

Argo is a Cloud Native Computing Foundation (CNCF) hosted project.

Use Cases

  • Machine Learning pipelines
  • Data and batch processing
  • ETL
  • Infrastructure automation
  • CI/CD

Why Argo Workflows?

  • Argo Workflows is the most popular workflow execution engine for Kubernetes.
  • It can run 1000s of workflows a day, each with 1000s of concurrent tasks.
  • Our users say it is lighter-weight, faster, more powerful, and easier to use
  • Designed from the ground up for containers without the overhead and limitations of legacy VM and server-based environments.
  • Cloud agnostic and can run on any Kubernetes cluster.

Read what people said in our latest survey

Try Argo Workflows

Access the demo environment (login using Github)

Screenshot

Ecosystem

Ecosystem

Argo Events | Argo Workflows Catalog | Couler | OnePanel | Ploomber | SQLFlow | Kubeflow Pipelines

SDKs

Check out our Java, Golang and Python SDKs.

Quickstart

kubectl create namespace argo
kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo-workflows/stable/manifests/install.yaml

Who uses Argo Workflows?

Official Argo Workflows user list

Documentation

Features

  • UI to visualize and manage Workflows
  • Artifact support (S3, Artifactory, Alibaba Cloud OSS, HTTP, Git, GCS, raw)
  • Workflow templating to store commonly used Workflows in the cluster
  • Archiving Workflows after executing for later access
  • Scheduled workflows using cron
  • Server interface with REST API (HTTP and GRPC)
  • DAG or Steps based declaration of workflows
  • Step level input & outputs (artifacts/parameters)
  • Loops
  • Parameterization
  • Conditionals
  • Timeouts (step & workflow level)
  • Retry (step & workflow level)
  • Resubmit (memoized)
  • Suspend & Resume
  • Cancellation
  • K8s resource orchestration
  • Exit Hooks (notifications, cleanup)
  • Garbage collection of completed workflow
  • Scheduling (affinity/tolerations/node selectors)
  • Volumes (ephemeral/existing)
  • Parallelism limits
  • Daemoned steps
  • DinD (docker-in-docker)
  • Script steps
  • Event emission
  • Prometheus metrics
  • Multiple executors
  • Multiple pod and workflow garbage collection strategies
  • Automatically calculated resource usage per step
  • Java/Golang/Python SDKs
  • Pod Disruption Budget support
  • Single-sign on (OAuth2/OIDC)
  • Webhook triggering
  • CLI
  • Out-of-the box and custom Prometheus metrics
  • Windows container support
  • Embedded widgets
  • Multiplex log viewer

Community Meetings

We host monthly community meetings where we and the community showcase demos and discuss the current and future state of the project. Feel free to join us! For Community Meeting information, minutes and recordings please see here.

Participation in the Argo Workflows project is governed by the CNCF Code of Conduct

Community Blogs and Presentations

Project Resources

kubernetes argo devops

What is Geek Coin

What is GeekCash, Geek Token

Best Visual Studio Code Themes of 2021

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

50+ Useful Kubernetes Tools for 2020 - Part 2

Our original Kubernetes tool list was so popular that we've curated another great list of tools to help you improve your functionality with the platform.

How to Extend your DevOps Strategy For Success in the Cloud?

DevOps and Cloud computing are joined at the hip, now that fact is well appreciated by the organizations that engaged in SaaS cloud and developed applications in the Cloud. During the COVID crisis period, most of the organizations have started using cloud computing services and implementing a cloud-first strategy to establish their remote operations. Similarly, the extended DevOps strategy will make the development process more agile with automated test cases.

What Is DevOps and Is Enterprise DevOps Any Good?

What is DevOps? How are organizations transitioning to DevOps? Is it possible for organizations to shift to enterprise DevOps? Read more to find out!

DevOps Basics: What You Should Know

What is DevOps? What are the goals it helps achieves? What are its benefits? This article has answers!

Why You Should Be Adopting a DevOps Culture in 2020 - DZone DevOps

The year 2020 has arrived, and its arrival brings a lot of innovations and transformations in the Information and Technology (IT) sector to DevOps technologies.