Concurrency in Golang and WorkerPool [Part 1]

Concurrency in Golang and WorkerPool [Part 1]

Project Link: https://github.com/Joker666/goworkerpool

In modern programming languages concurrency has become an unequivocal need. Almost all programming languages today have some method of concurrency. Some have really powerful constructs, that can offload works to different threads in the OS, e.g., Java, some mimic this behavior in the same thread, e.g., Ruby.

Golang has a very powerful concurrency model called CSP (communicating sequential processes), which breaks a problem into smaller sequential processes and then schedules several instances of these processes called Goroutines. The communication between these processes happens by passing immutable messages via Channels.

Here we will explore, how we can take advantage of concurrency in golang and how we can limit its usage with worker pools. In the second part, we will explore how to build a robust solution for concurrency around this concept.

A Simple Case

Let's imagine we have an external API call which takes about 100ms to get done. If we have 1000 of these calls, and we call these synchronously, it would take about 100s to complete

//// model/data.go

package model

type SimpleData struct {
    ID int
}

//// basic/basic.go

package basic

import (
    "fmt"
    "github.com/Joker666/goworkerpool/model"
    "time"
)

func Work(allData []model.SimpleData) {
    start := time.Now()
    for i, _ := range allData {
        Process(allData[i])
    }
    elapsed := time.Since(start)
    fmt.Printf("Took ===============> %s\n", elapsed)
}

func Process(data model.SimpleData) {
    fmt.Printf("Start processing %d\n", data.ID)
    time.Sleep(100 * time.Millisecond)
    fmt.Printf("Finish processing %d\n", data.ID)
}

//// main.go

package main

import (
    "fmt"
    "github.com/Joker666/goworkerpool/basic"
    "github.com/Joker666/goworkerpool/model"
    "github.com/Joker666/goworkerpool/worker"
)

func main() {
    // Prepare the data
    var allData []model.SimpleData
    for i := 0; i < 1000; i++ {
        data := model.SimpleData{ ID: i }
        allData = append(allData, data)
    }
    fmt.Printf("Start processing all work \n")

    // Process
    basic.Work(allData)
}
Start processing all work 
Took ===============> 1m40.226679665s

Here, we have a simple model that holds a data struct with just an integer value. We process an array of this data synchronously. This is obviously not an optimum solution since these tasks can be processed concurrently. Let's turn this into an asynchronous process with goroutines and channels.

golang programming backend optimization

What is Geek Coin

What is GeekCash, Geek Token

Best Visual Studio Code Themes of 2021

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Hire Backend Developers India

Are you looking to hire experienced Backend Developers at a reasonable cost to boost-up your IT business? **[Hire Backend Developers India](https://hourlydeveloper.io/hire-back-end-developer/ "Hire Backend Developers India")** and accomplish...

Hire Dedicated Backend Developers

Want to create a backend for a web or mobile app using PHP & JS frameworks? **[Hire Dedicated Backend Developers](https://hourlydeveloper.io/hire-back-end-developer/ "Hire Dedicated Backend Developers")** who offer end-to-end, robust, scalable...

Golang Web Development:Th Best Programming Language in 2020

Golang Web Development is the new trend globally. Get all the details about the Golang development here. Hire Golang Developers.

Concurrency in Golang And WorkerPool [Part 2]

Project Link: https://github.com/Joker666/goworkerpool

Hire Dedicated Golang Developers | Golang Web Development Company

Hire dedicated GoLang developers from WebClues Infotech to build applications using Go language. Get best GoLang developers for your web development projects.