Earlier this year at Spark + AI Summit, we had the pleasure of presenting our session on the best practices and pitfalls of running Apache Spark on Kubernetes (K8s).

In this post we’d like to expand on that presentation and talk to you about:

1. What is Kubernetes?
2. Why run Spark on Kubernetes?
3. Getting started with Spark on Kubernetes
4. Optimizing performance and cost
5. Monitoring your Spark applications on Kubernetes
6. The future of Spark on Kubernetes

If you’re already familiar with k8s and why Spark on Kubernetes might be a fit for you, feel free to skip the first couple of sections and get straight to the meat of the post!

#apache-spark #spark-on-kubernetes #docker #kubernetes #spark-on-k8s #k8s #good-company #devops