Kubernetes Monitoring and Logging — An Apache Spark Example

Kubernetes Monitoring and Logging — An Apache Spark Example

We'll learn Kubernetes Monitoring and Logging — An Apache Spark Example. I will use Apache Spark on Kubernetes as an example to share what I use as my monitoring and logging stack. I want to give an overview here, I will have another blog to explain the how-to in details.

I have moved almost all my big data and machine learning projects to Kubernetes and Pure Storage. I am very happy with this move so far. With this platform, my life as a data engineer / data scientist becomes easier — much easier to deploy, scale and manage my Spark jobsPresto queriesTensorFlow trainings and so on. It is also easier to share the infrastructure resource among these projects.

However, when I set up the environment at the beginning, I found it is a little difficult to understand what is actually going there. I realised I need an end-to-end monitoring setup for my Kubernetes cluster and its applications. In this blog, I will use Apache Spark on Kubernetes as an example to share what I use as my monitoring and logging stack. I want to give an overview here, I will have another blog to explain the how-to in details.

kubernetes

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

50+ Useful Kubernetes Tools for 2020 - Part 2

Our original Kubernetes tool list was so popular that we've curated another great list of tools to help you improve your functionality with the platform.

Kubernetes in the Cloud: Strategies for Effective Multi Cloud Implementations

This article explains how you can leverage Kubernetes to reduce multi cloud complexities and improve stability, scalability, and velocity.

Kubernetes vs Docker

Get Hands-on experience on Kubernetes and the best comparison of Kubernetes over the DevOps at your place at Kubernetes training

Typical flow for deploying applications to Kubernetes

Get Hands-on experience on Kubernetes and the best comparison of Kubernetes over the DevOps at your place at Kubernetes training

Microsoft Announces General Availability Of Bridge To Kubernetes

Microsoft announced the general availability of Bridge to Kubernetes, formerly known as Local Process with Kubernetes.