ClusteredBigCache: Golang Bigcache with Clustering As A Library

clusteredBigCache

This is a library based on bigcache with some modifications to support

  • clustering and
  • individual item expiration

Bigcache is an excellent piece of software but the fact that items could only expire based on a predefined value was not just too appealing. Bigcache had to be modified to support individual expiration of items using a single timer. This happens by you specifying a time value as you add items to the cache. Running two or more instances of an application that would require some level of caching would normally default to memcache or redis which are external applications adding to the mix of services required for your application to run.

With clusteredBigCache there is no requirement to run an external application to provide caching for multiple instances of your application. The library handles caching as well as clustering the caches between multiple instances of your application providing you with simple library APIs (by just calling functions) to store and get your values.

With clusteredBigCache, when you store a value in one instance of your application and every other instance or any other application for that matter that you configure to form/join your "cluster" will see that exact same value.

Installing

Using go get

$ go get github.com/oaStuff/clusteredBigCache

Samples 1

This is the application responsible for storing data into the cache

package main

import (
    "fmt"
    "bufio"
    "os"
    "github.com/oaStuff/clusteredBigCache/Cluster"
    "strings"
    "time"
)

//
//main function
//
func main() {
    fmt.Println("starting...")
    cache := clusteredBigCache.New(clusteredBigCache.DefaultClusterConfig(), nil)
    count := 1
    cache.Start()

    reader := bufio.NewReader(os.Stdin)
    var data string
    for strings.ToLower(data) != "exit" {
        fmt.Print("enter data : ")
        data, _ = reader.ReadString('\n')
        data = strings.TrimSpace(data)
        err := cache.Put(fmt.Sprintf("key_%d", count), []byte(data), time.Minute * 60)
        if err != nil {
            panic(err)
           }
           fmt.Printf("'%s' stored under key 'key_%d'\n", data, count)
           count++
       }
}

Explanation:

The above application captures data from the keyboard and stores them inside clusteredBigCache starting with keys 'key_1', 'key_2'...'key_n'. As the user types and presses the enter key the data is stored in the cache.

cache := clusteredBigCache.New(clusteredBigCache.DefaultClusterConfig(), nil) This statement will create the cache using the default configuration. This configuration has default values for LocalPort = 9911, Join = false amongst others. If you intend to use this library for applications that will run on the same machine, you will have to give unique values for LocalPort

cache.Start() This must be called before using any other method on this cache.

err := cache.Put(fmt.Sprintf("key_%d", count), []byte(data), time.Minute * 60). You set values in the cache giving it a key, the data as a []byte slice and the expiration or time to live (ttl) for that key/value within the cache. When the key/value pair reaches its expiration time, they are removed automatically.

Samples 2

This is the application responsible for reading data of the cache. This can be run on the same or different machine on the network.

package main

import (
    "github.com/oaStuff/clusteredBigCache/Cluster"
    "bufio"
    "os"
    "strings"
    "fmt"
    "time"
)

//
//
func main() {
    config := clusteredBigCache.DefaultClusterConfig()
    config.LocalPort = 8888
    config.Join = true
    config.JoinIp = "127.0.0.1:9911"
    cache := clusteredBigCache.New(config, nil)
    err := cache.Start()
    if err != nil {
        panic(err)
    }
    
    reader := bufio.NewReader(os.Stdin)
    var data string
    for strings.ToLower(data) != "exit" {
        fmt.Print("enter key : ")
        data, _ = reader.ReadString('\n')
        data = strings.TrimSpace(data)
        value, err := cache.Get(data, time.Millisecond * 160)
        if err != nil {
            fmt.Println(err)
            continue
        }
        fmt.Printf("you got '%s' from the cache\n", value)
    }
}

Explanation:

The above application reads a string from the keyboard which should represent a key for a value in clusteredBigCache. If a user enters the corresponding keys shown in sample1 above ('key_1', 'key_2'...'key_n'), the corresponding values will be returned.

    config := clusteredBigCache.DefaultClusterConfig()
    config.LocalPort = 8888
    config.Join = true
    config.JoinIp = "127.0.0.1:9911"
    cache := clusteredBigCache.New(config, nil)
    err := cache.Start()

The above uses the default configuration to create a config and modifies what it actually needs. config.LocalPort = 8888 has to be changed since this application will run on the same machine with the sample1 application. This is to avoid 'port already in use' errors.

config.Join = true. For an application to join another application or applications using clusteredBigCache, it must set config.Join value to true and set config.JoinIP to the IP address of one of the systems using clusteredBigCache eg config.Join = "127.0.0.1:9911. This example says that this application wants to join another application using clusteredBigCache at IP address 127.0.0.1 and port number 9911.

cache := clusteredBigCache.New(config, nil) creates the cache and cache.Start() must be called to start everything running.

NB

After cache.Start() is called the library tries to connect to the specified IP address using the specified port. When successfully connected, it create a cluster of applications using clusteredBigCache as a single cache. ie all applications connected will see every value every application sets in the cache.

Sample way to parse config in an app

    join := flag.String("join", "", "ipAddr:port number of remote server")
    localPort := flag.Int("port", 6060, "local server port to bind to")

    
    flag.Parse()
    
    config := clusteredBigCache.DefaultClusterConfig()
    if *join != "" {
        config.JoinIp = *join
        config.Join = true
    }
    config.LocalPort = *localPort

Your application could pass parameters to it in any form and make use of them in configuring clusteredBigCache. The above sample just only catered for join and localport. If you want network connections between machine to be reconnected in the event of a disconnection, you will have to set config.ReconnectOnDisconnect = true.

Logging within the library

clusteredBigCache takes a second parameter is its New() function for logging. This function expects an interface of

type AppLogger interface {
    Info(msg string)
    Warn(msg string)
    Critical(msg string)
    Error(msg string)
}

You could easily just wrap any logger within a struct and provide this interface method for that struct and simple delegate calls to the underlining logger or better still just wrap a logger function to provide the interface like example bellow

type myLogger func(...interface{})

func (log myLogger) Info(msg string)  {
    log(msg)
}

func (log myLogger) Warn(msg string)  {
    log(msg)
}

func (log myLogger) Error(msg string)  {
    log(msg)
}

func (log myLogger) Critical(msg string)  {
    log(msg)
}


cache := clusteredBigCache.New(config, myLogger(log.Println))

Using Passive client

Passive client are nodes in the clusteredBigCache network that do not store any data locally but functions all the same like every other node. To create a passive client you simply call clusteredBigCache.NewPassiveClient("linux_box_100","localhost:9090", 8885, 0, 0, 0, nil) This will connect to an existing cluster at address localhost:9090 and join the cluster. the linux_box_100 is the node's id. This can be an empty string if you want an auto generated id. Every other function can be performed on the returned object.

credits

Core cache system from bigcache

Data structures from emirpasic

Author: OaStuff
Source Code: https://github.com/oaStuff/clusteredBigCache 
License: MIT License

#go #golang #caching 

What is GEEK

Buddha Community

ClusteredBigCache: Golang Bigcache with Clustering As A Library

Hire Dedicated Golang Developers | Golang Web Development Company

Does your business need a robust system across large-scale network servers then developing your app with a Golang programming language is the way to go. Golang is generally used for the development of highly secured, High Speed and High Modularity apps such as a FinTech Industry.

Want to develop a Highly secured app for your business?

Then hire a dedicated Golang developer from WebClues Infotech that are highly skilled in carrying out the work in a timely and qualitative output. With WebClues Infotech you get the assurance that we know what are the customers’ expectations and how to deliver on them on time.

Get your desired Golang Developer based on your project requirement!!

Share your requirements here https://www.webcluesinfotech.com/contact-us/

Book Free Interview with Golang developer: https://bit.ly/3dDShFg

#hire golang developer #hire go language developer #dedicated golang app developers #golang web development company #hire golang developers india #hire expert golang developers

ClusteredBigCache: Golang Bigcache with Clustering As A Library

clusteredBigCache

This is a library based on bigcache with some modifications to support

  • clustering and
  • individual item expiration

Bigcache is an excellent piece of software but the fact that items could only expire based on a predefined value was not just too appealing. Bigcache had to be modified to support individual expiration of items using a single timer. This happens by you specifying a time value as you add items to the cache. Running two or more instances of an application that would require some level of caching would normally default to memcache or redis which are external applications adding to the mix of services required for your application to run.

With clusteredBigCache there is no requirement to run an external application to provide caching for multiple instances of your application. The library handles caching as well as clustering the caches between multiple instances of your application providing you with simple library APIs (by just calling functions) to store and get your values.

With clusteredBigCache, when you store a value in one instance of your application and every other instance or any other application for that matter that you configure to form/join your "cluster" will see that exact same value.

Installing

Using go get

$ go get github.com/oaStuff/clusteredBigCache

Samples 1

This is the application responsible for storing data into the cache

package main

import (
    "fmt"
    "bufio"
    "os"
    "github.com/oaStuff/clusteredBigCache/Cluster"
    "strings"
    "time"
)

//
//main function
//
func main() {
    fmt.Println("starting...")
    cache := clusteredBigCache.New(clusteredBigCache.DefaultClusterConfig(), nil)
    count := 1
    cache.Start()

    reader := bufio.NewReader(os.Stdin)
    var data string
    for strings.ToLower(data) != "exit" {
        fmt.Print("enter data : ")
        data, _ = reader.ReadString('\n')
        data = strings.TrimSpace(data)
        err := cache.Put(fmt.Sprintf("key_%d", count), []byte(data), time.Minute * 60)
        if err != nil {
            panic(err)
           }
           fmt.Printf("'%s' stored under key 'key_%d'\n", data, count)
           count++
       }
}

Explanation:

The above application captures data from the keyboard and stores them inside clusteredBigCache starting with keys 'key_1', 'key_2'...'key_n'. As the user types and presses the enter key the data is stored in the cache.

cache := clusteredBigCache.New(clusteredBigCache.DefaultClusterConfig(), nil) This statement will create the cache using the default configuration. This configuration has default values for LocalPort = 9911, Join = false amongst others. If you intend to use this library for applications that will run on the same machine, you will have to give unique values for LocalPort

cache.Start() This must be called before using any other method on this cache.

err := cache.Put(fmt.Sprintf("key_%d", count), []byte(data), time.Minute * 60). You set values in the cache giving it a key, the data as a []byte slice and the expiration or time to live (ttl) for that key/value within the cache. When the key/value pair reaches its expiration time, they are removed automatically.

Samples 2

This is the application responsible for reading data of the cache. This can be run on the same or different machine on the network.

package main

import (
    "github.com/oaStuff/clusteredBigCache/Cluster"
    "bufio"
    "os"
    "strings"
    "fmt"
    "time"
)

//
//
func main() {
    config := clusteredBigCache.DefaultClusterConfig()
    config.LocalPort = 8888
    config.Join = true
    config.JoinIp = "127.0.0.1:9911"
    cache := clusteredBigCache.New(config, nil)
    err := cache.Start()
    if err != nil {
        panic(err)
    }
    
    reader := bufio.NewReader(os.Stdin)
    var data string
    for strings.ToLower(data) != "exit" {
        fmt.Print("enter key : ")
        data, _ = reader.ReadString('\n')
        data = strings.TrimSpace(data)
        value, err := cache.Get(data, time.Millisecond * 160)
        if err != nil {
            fmt.Println(err)
            continue
        }
        fmt.Printf("you got '%s' from the cache\n", value)
    }
}

Explanation:

The above application reads a string from the keyboard which should represent a key for a value in clusteredBigCache. If a user enters the corresponding keys shown in sample1 above ('key_1', 'key_2'...'key_n'), the corresponding values will be returned.

    config := clusteredBigCache.DefaultClusterConfig()
    config.LocalPort = 8888
    config.Join = true
    config.JoinIp = "127.0.0.1:9911"
    cache := clusteredBigCache.New(config, nil)
    err := cache.Start()

The above uses the default configuration to create a config and modifies what it actually needs. config.LocalPort = 8888 has to be changed since this application will run on the same machine with the sample1 application. This is to avoid 'port already in use' errors.

config.Join = true. For an application to join another application or applications using clusteredBigCache, it must set config.Join value to true and set config.JoinIP to the IP address of one of the systems using clusteredBigCache eg config.Join = "127.0.0.1:9911. This example says that this application wants to join another application using clusteredBigCache at IP address 127.0.0.1 and port number 9911.

cache := clusteredBigCache.New(config, nil) creates the cache and cache.Start() must be called to start everything running.

NB

After cache.Start() is called the library tries to connect to the specified IP address using the specified port. When successfully connected, it create a cluster of applications using clusteredBigCache as a single cache. ie all applications connected will see every value every application sets in the cache.

Sample way to parse config in an app

    join := flag.String("join", "", "ipAddr:port number of remote server")
    localPort := flag.Int("port", 6060, "local server port to bind to")

    
    flag.Parse()
    
    config := clusteredBigCache.DefaultClusterConfig()
    if *join != "" {
        config.JoinIp = *join
        config.Join = true
    }
    config.LocalPort = *localPort

Your application could pass parameters to it in any form and make use of them in configuring clusteredBigCache. The above sample just only catered for join and localport. If you want network connections between machine to be reconnected in the event of a disconnection, you will have to set config.ReconnectOnDisconnect = true.

Logging within the library

clusteredBigCache takes a second parameter is its New() function for logging. This function expects an interface of

type AppLogger interface {
    Info(msg string)
    Warn(msg string)
    Critical(msg string)
    Error(msg string)
}

You could easily just wrap any logger within a struct and provide this interface method for that struct and simple delegate calls to the underlining logger or better still just wrap a logger function to provide the interface like example bellow

type myLogger func(...interface{})

func (log myLogger) Info(msg string)  {
    log(msg)
}

func (log myLogger) Warn(msg string)  {
    log(msg)
}

func (log myLogger) Error(msg string)  {
    log(msg)
}

func (log myLogger) Critical(msg string)  {
    log(msg)
}


cache := clusteredBigCache.New(config, myLogger(log.Println))

Using Passive client

Passive client are nodes in the clusteredBigCache network that do not store any data locally but functions all the same like every other node. To create a passive client you simply call clusteredBigCache.NewPassiveClient("linux_box_100","localhost:9090", 8885, 0, 0, 0, nil) This will connect to an existing cluster at address localhost:9090 and join the cluster. the linux_box_100 is the node's id. This can be an empty string if you want an auto generated id. Every other function can be performed on the returned object.

credits

Core cache system from bigcache

Data structures from emirpasic

Author: OaStuff
Source Code: https://github.com/oaStuff/clusteredBigCache 
License: MIT License

#go #golang #caching 

Anda Lacacima

Anda Lacacima

1605336288

Exploring Go Libraries: Docopt

In this tutorial, we walk through basic concepts and help messages in Docopt. You’ll learn about a practical use case that you can code right now.

› Shortly about Docopt

Docopt is a command-line interface description language. It has many implementations for different programming languages. In this tutorial, we’ll discuss the Go version: Docopt Go.

This library gives programmers the ability to describe a program interface with a well-structured help message.

› Before You Begin

Install Docopt on your computer with this command:

$ go get -u -v github.com/docopt/docopt.go

The -u flag instructs go get to use the network to update the named packages and their dependencies. By default, get uses the network to check out missing packages but does not use it to look for updates to existing packages.

The -v flag enables verbose progress and debug output.

See this resource to learn more about the go get command.

Import the library in your *.go file as follows:

import docopt “github.com/docopt/docopt-go”

#golang-libraries #golang-tutorial #command-line #golang

Golang Web Development:Th Best Programming Language in 2020

https://www.mobinius.com/blogs/golang-web-development-company

#golang web development #golang-app-development-company #golang-development-solutions #hire-golang-developers #golang-development-services

Ajay Kapoor

1624960485

Golang vs. Node.JS: Who Trumps the Battle of Backend Frameworks?

Full blog here

The backend of your application is truly the essential part of your product. No matter how much you appreciate the design, the application’s success lies in its backend. A scalable backend that effectively implements the required business logic is the primary goal of programmers.

Therefore, it is crucial to choose the most powerful and scalable technology. There are plenty of languages ​​in the market that can form the backend of any application, Node.js and Golang are the two most popular technologies among them.

They are real and developed languages ​​that have recently been used in various outstanding projects. Golang is an open-source programming language, whereas Node.js is an open-source server framework. They both are gaining popularity for various reasons.

According to a development stat, it is observed that almost 50% out of 58,543 respondents use Node.js as their preferred app development tool.

Golang, on the other hand, has overtaken other programming languages in the application development market and has gained huge recognition over the past few years.

But, which backend framework is best for you? In this article, I’ll make a healthy comparison of two of Google’s most popular backend development tools based on several essential features and various other factors.

Golang developers for hire

#best backend frameworks #node or golang #golang or nodejs #nodejs vs golang #golang vs nodejs #top backend frameworks