Spark on Kubernetes, The Easy Way

How our startup Data Mechanics (YCombinator, S19) builds on top of the open-source project. Let's start with Spark on Kubernetes, The Easy Way.

I’m an ex-Databricks engineer, now co-founder of Data Mechanics, a managed Spark platform deployed on a Kubernetes cluster inside our customers cloud account (AWS, Azure, or GCP). Our mission is to make Spark more developer friendly and more cost-effecvtive for data engineering teams.

In this story, I’m answering a legitimate question we get asked all the time — what’s the point of buying your startup offering versus running Spark on Kubernetes the open-source way. The short answer is:

1. We add specific features that make Spark-on-Kubernetes easy-to-use, cost-effective, secure and stable.
2. We take care of the setup and maintenance of Spark and Kubernetes for you — saving your DevOps team a lot of headaches.
3. While keeping you in control and giving you the flexibility of a home-built solution, as we’re deployed in your cloud account and your private network.

But let’s support these points with facts and illustrations! What do we add on top of Spark-on-Kubernetes open-source?

kubernetes

Our original Kubernetes tool list was so popular that we've curated another great list of tools to help you improve your functionality with the platform.

This article explains how you can leverage Kubernetes to reduce multi cloud complexities and improve stability, scalability, and velocity.

Get Hands-on experience on Kubernetes and the best comparison of Kubernetes over the DevOps at your place at Kubernetes training

Get Hands-on experience on Kubernetes and the best comparison of Kubernetes over the DevOps at your place at Kubernetes training

Microsoft announced the general availability of Bridge to Kubernetes, formerly known as Local Process with Kubernetes.