In this era of computation power greed, we tend to forget to use the power we can utilize on our very computers

The hunger for computation power among programmers, gamers, scientists, software developers, and most humans that know how to use a computer, in general, is immense. We are always looking for applications that are less compute-intensive and more efficient. This allows us to make use of our computer setups more efficiently.

However, many of us do not fully utilize the computation power already available to us on our computers. Utilizing this power when needed can lead to exponentially better performances and usually, you can run the processes 2–3 times faster with some changes in code. How do we do this you ask? Well, let’s dive in.

This blog focuses on parallel programming. i.e, running a program on multiple processors simultaneously. When you run your program it usually uses one of the cores in your computer. However, most computers have multiple cores. Depending on your processor it maybe dual-core, quad-core, octa-core, or may contain more cores. If (let’s say) you have a quad-core processor running a program only on one core, you are essentially letting go of the other three cores and hence three times the computation power you are using. Using all these cores can theoretically speed up your tasks by four.

However, it is not so simple otherwise software companies would all be using all the cores all the time for better performance. If you want to increase the performance of your program you need to make sure that it can be parallelized. i.e, it can be run on different cores at once simultaneously.

Let us take a simple example to understand this point. Let us say you have to build a product and you divide its manufacturing into four stages. If you can go from stage 1 to stage 2 only when stage 1 is already completed, and then from stage 2 to stage 3 only when stage 2 is completed and so on. Then this process is a sequential process since you have to follow a sequence to execute your instructions. However, If you can break the manufacturing into four parts and assign it to different workers, then the process can be parallelized. e.g, building four components for a toy that can be joined once they are finished.

Once you have established that your program can be parallelized, the next step is to write the code for it. I will not write the code in this blog, however, the code for this blog can be found at this Github repo. I choose python to write the code and I used the multiprocessing module to run the program on multiple processors.

In this program, we will see two applications of parallel programming. First is Matrix Multiplication which can be easily parallelized and next we shall see prefix sum scan, which on the first look seems to be a sequential problem but can be parallelized to run on multiple processors.

#programming #computer-science #computers #gpu #parallel-computing