Concurrency in Golang And WorkerPool [Part 2]

Concurrency in Golang And WorkerPool [Part 2]

Project Link: https://github.com/Joker666/goworkerpool

Goroutines and channels are powerful language structs that make golang a powerful concurrent language. In the first part of the article, we explored, how we can build a workerpool to optimize the performance of the concurrency structs of golang namely limiting the resource utilization. But it was a simple example to demonstrate how we can go about it.

Here, we will build a robust solution according to the learning from the first part so that we can use this solution in any application. There are some solutions on the internet with complex architecture using dispatchers and all. In reality, we do not need it, we can do everything using one shared channel. Let's see how we can build that here

Architecture

Here we make a generic workerpool package that can handle tasks with workers based on the desired concurrency. Let's see the directory structure.

workerpool
├── pool.go
├── task.go
└── worker.go

The 

workerpool directory is in the root folder of the project. Let's go over what Task is. Task is a single unit of work that needs to be processed. Worker is a simple worker function that handles running the task. And Pool actually handles the creation and managing the workers.

Implementation

Let's code out 

Task first.

// workerpool/task.go

package workerpool

import (
    "fmt"
)

type Task struct {
    Err  error
    Data interface{}
    f    func(interface{}) error
}

func NewTask(f func(interface{}) error, data interface{}) *Task {
    return &Task{f: f, Data: data}
}

func process(workerID int, task *Task) {
    fmt.Printf("Worker %d processes task %v\n", workerID, task.Data)
    task.Err = task.f(task.Data)
}

golang programming backend optimization

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Hire Backend Developers India

Are you looking to hire experienced Backend Developers at a reasonable cost to boost-up your IT business? **[Hire Backend Developers India](https://hourlydeveloper.io/hire-back-end-developer/ "Hire Backend Developers India")** and accomplish...

Hire Dedicated Backend Developers

Want to create a backend for a web or mobile app using PHP & JS frameworks? **[Hire Dedicated Backend Developers](https://hourlydeveloper.io/hire-back-end-developer/ "Hire Dedicated Backend Developers")** who offer end-to-end, robust, scalable...

Golang Web Development:Th Best Programming Language in 2020

Golang Web Development is the new trend globally. Get all the details about the Golang development here. Hire Golang Developers.

Concurrency in Golang and WorkerPool [Part 1]

Project Link: https://github.com/Joker666/goworkerpool

Optimizing String Comparisons in Golang

Want your Go programs to run faster? Optimizing string comparisons in Go can improve your application’s response time and help scalability. In this tutorial, you'll see Optimizing String Comparisons in Golang