If deep learning is a super power, then turning theories from a paper to usable code is a hyper power

Why should I learn to implement machine learning research papers?

As I’ve said, being able to convert a paper to code is definitely a hyper power, especially in a field like machine learning which is moving faster and faster each day.

Most research papers come from people within giant tech companies or universities who may be PhD holders or the ones who are working on the cutting edge technologies.

What else can be more cool than being able to reproduce the research done by these top notch professionals. Another thing to note is that the ones who can reproduce research papers as code is in huge demand.

Once you get the knack of implementing research papers, you will be in a state on par with these researchers.

These researchers too has acquired these skills through the practice of reading and implementing research papers.

How do I read and implement papers?

You might say, “Hm, I have a general understanding of the deep learning algorithms like fully connected networks, convolutional neural networks, recurrent neural networks, but the problem is that I would like to develop SOTA(state of the art) voice cloning AI but I know nothing about voice cloning :( ”.

Okay, here is your answer(some parts of my method is taken from Andrew Ng’s advice on reading papers).

If you want to learn about a specific topic:

1. Collect 5–6 papers related to the specific topic(you can simply search arxiv or similar websites to get papers related to a topic).
2. Don’t read a single paper completely, instead skim through all of the papers and pick a paper that interests you or if you had a specific paper in mind, go pick it up, no one can stop you.
3. Read the abstract carefully and understand the idea from a high level and see whether your interest still persists, if so continue to skim through the images and see whether you can make assumptions on what the paper might be about.
4. Now read the introduction carefully line by line because most of what the paper contains will be explained here in the most simplest manner with minimal math.
5. If you wish, you can skip the math equations in the first pass, don’t skip the math if the Greek letters are familiar.
6. At any situation, if you get stuck or some words are confusing, never hesitate to google it. No one is born as master of everything ;)
7. After completing the first pass, you will be in a state where you understand the high level view of what the paper is trying prove or improve.
8. In the second pass, try to understand almost everything in the paper and if you encounter any pseudo-code, try to convert it into your python library of choice(PyTorch, TensorFlow…)
9. You can get more papers to read and get a better understanding of the field by going to the references section of each paper(same as connecting the dots).

💡 Some tips for effectively understanding a paper:

• If you are a beginner to reading research papers, it’s good to read some blog posts and videos related to that topic/research paper before reading the paper itself. This makes your job easier and you will not be discouraged by all that Greek letters.
• Always take notes and highlight important points in the research paper for easier reference while implementing the code for the paper.
• If you are new to implementing research papers and get stuck any where, it’s not a bad idea to go through open source implementations and see how others have done this.

#deep-learning #research #unsupervised-learning #machine-learning #deep learning