Monitoring the Hadoop Application/Job differently with Rust

Monitoring the Hadoop Application/Job differently with Rust

Disclaimer: This is ONLY my weekend research and development topic; the idea behind is to learn more about the Hadoop application/job logs and Rust language. The solution is NOT for production used. Use it at your own risk. Monitoring the Hadoop Application/Job differently with Rust

Disclaimer: This is ONLY my weekend research and development topic; the idea behind is to learn more about the Hadoop application/job logs and Rust language. The solution is NOT for production used. Use it at your own risk.

Motivation

Everything starts with “Why.” I have been in the Big Data area for years. One of the critical pain point over the years, it’s the Hadoop job monitoring. And you will say: “wait, there is a lot of tools over there.” Yes, I agree with you. However, if you put into a company daily operation standpoint, you will have a lot of challenges on:

  1. How to get one single view for the platform/application development or operation team, without overhead monitoring tools for different groups.
  2. How to make sure to store those metrics consistently with the full capabilities for the historical analysis view? And even apply ML on top of it to drive the compatible platform or job level operation improvements.
  3. How to not get lost in all the new fancy tooling around the monitoring?
  4. How to increase the tooling usage rate and the Return on Investment (R0I) rate?

Based on those why and how, I’m leveraging this long weekend to start some exciting R&D (Research and Development).

Keep learning and fail fast

This weekend build and learn, of course, with my current favorable language Rust.

hadoop sqlite rust research-and-development hdfs

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

How long does it take to develop/build an app?

This article covers A-Z about the mobile and web app development process and answers your question on how long does it take to develop/build an app.

Developer Career Path: To Become a Team Lead or Stay a Developer?

For a developer, becoming a team leader can be a trap or open up opportunities for creating software. Two years ago, when I was a developer, ... by Oleg Sklyarov, Fullstack Developer at Skyeng company

Offshore Software Development - Best Practices

To make the most out of the benefits of offshore software development, you should understand the crucial factors that affect offshore development.

Setting up a Rust Development Environment

In this post we'll be discussing one way to set up your machine for Rust development.

Hire Ruby on Rails Developer | Hire RoR Developer

#1 Ruby on Rails development company. Hire Ruby on rails developer or a team to build secure, scalable and complex web solutions with a quick turnaround time.