Weekend build and learn

First of all, I hope everyone is safe and healthy, especially at this challenging time of the COVID-19. Stay Strong, my friends.

When we are in a difficult time, we need to more positive. Take the time with your family, your love one, use the time to improve ourselves.

---

In my previous articles, I demonstrate/share some of my new learning about how to process a large file (between 1 GB to 2 GB) with Rust.

However, I want to go to the next level of testing to understand the capability of processing large files with Rust. Such as 14 GB file.

---

Background & Use Case

Recently I have read an article that is processing an extensive dataset from Wikipedia, which contains all the Wikipedia page information.

My curiosity raised by one sentence in the article:

If we use a single machine to do that, it may take even days to complete since it is a compute-extensive task.

I decide to use Rust to try this statement.

---

In Part 1, I would like to start with the code and process the file with my daily coding machine, which is Huawei Matebook 13 (Linux edition).

#rustlang #rust #apache-spark #wikipedia #big-data