Go and CPU Caches

How understanding the processor architecture can help us in optimizing performance?

According to Jackie Stewart, a three-time world champion F1 driver, having an understanding of how a car works made him a better pilot.

“You don’t have to be an engineer to be be a racing driver, but you do have to have Mechanical Sympathy”

Martin Thompson (the designer of the LMAX Disruptor) applied the concept mechanical sympathy to programming. In a nutshell, understanding the underlying hardware should make us a better developer when it comes to designing algorithms, data structures, etc.

In this post, we will dig into the processor and see how understanding some of its concepts can help us when it comes to optimizing solutions.

#go #web-development #developer

What is GEEK

Buddha Community

Fannie Zemlak

1599854400

What's new in the go 1.15

Go announced Go 1.15 version on 11 Aug 2020. Highlighted updates and features include Substantial improvements to the Go linker, Improved allocation for small objects at high core counts, X.509 CommonName deprecation, GOPROXY supports skipping proxies that return errors, New embedded tzdata package, Several Core Library improvements and more.

As Go promise for maintaining backward compatibility. After upgrading to the latest Go 1.15 version, almost all existing Golang applications or programs continue to compile and run as older Golang version.

#go #golang #go 1.15 #go features #go improvement #go package #go new features

George Koelpin

1597706160

From Caching to CDN: How To Decide Which Way to Go

For content creators and web developers who are seeking to speed up their web pages, learn more about whether CDNs or caching works for you.

In an attempt to speed up their websites, owners are ready to take various measures. When we talk about the speed of a website, most often we mean the speed of its content loading. There are two effective methods to improve the load time — data caching and using a content delivery network (CDN).

Both methods are good in their own way and are used by a variety of web resources. Our article aims to compare them in terms of speed of data load. Our task is not to point you in the right direction, but to provide enough details so that you make an informed choice.

What Is Caching and How Does It Work?

At its core, data caching is the process of storing information from a website on a computer for a specific period of time. Usually, caching employs the part of RAM that is not used. This process starts automatically after the user loads the website page for the first time. Saving content (images, banners, videos, text, and so on) has a positive effect on the speed of its load. And this, in turn, accelerates the speed of site loading. The user no longer needs to wait until they access the source server and receive a response.

This process makes sense not only in terms of improving the user experience but also from the perspective of improving the website’s rank in search engines. For example, Google ranks fast sites higher. Demand for improved caching has resulted in various widgets hitting the market. They promise to make the caching process faster and better. However, often, this only leads to slower loading.

Of course, cached content isn’t stored forever. Usually, owners of web resources set specific caching options, including how long the data should be kept. This is done to free up the RAM space for more recent data.

What Is CDN and How Does It Work?

The way you see information on a website involves several processes. It all starts with your request for data when entering the site. The request travels to the server on which the website is running. The site receives a response from the server and the information appears in front of your eyes. Fast websites ensure that this process is maintained in a second. However, the speed of content loading is affected not only by how well-optimized the site is but also by the physical distance between the user and the server. For example, if you are located in Warsaw and the website's server is in Tokyo, then the request processing may take a longer time (~ 3-4 seconds). Therefore, using CDN image hosting, you can significantly reduce this time.

At its core, CDN is a network of third-party cache servers distributed around the world. They store cached data from multiple websites. Simply put, using a CDN, the website allows its content to be stored in several places around the globe. Expanding on the case above, the request from Warsaw won’t go to Tokyo and back. Instead, it will be sent to a server in Berlin, for example. The distance is reduced significantly and site loads faster (less than a second).

#performance #caching #cdn #website speed #data caching #go

Josefa Corwin

1659852060

A Template Language That Completely Separates Structure and Logic/Ruby

Curly

Curly is a template language that completely separates structure and logic. Instead of interspersing your HTML with snippets of Ruby, all logic is moved to a presenter class.

Installing

Installing Curly is as simple as running gem install curly-templates. If you're using Bundler to manage your dependencies, add this to your Gemfile

gem 'curly-templates'

Curly can also install an application layout file, replacing the .erb file commonly created by Rails. If you wish to use this, run the curly:install generator.

$ rails generate curly:install

How to use Curly

In order to use Curly for a view or partial, use the suffix .curly instead of .erb, e.g. app/views/posts/_comment.html.curly. Curly will look for a corresponding presenter class named Posts::CommentPresenter. By convention, these are placed in app/presenters/, so in this case the presenter would reside in app/presenters/posts/comment_presenter.rb. Note that presenters for partials are not prepended with an underscore.

Add some HTML to the partial template along with some Curly components:

<!-- app/views/posts/_comment.html.curly -->
<div class="comment">
  <p>
    {{author_link}} posted {{time_ago}} ago.
  </p>

  {{body}}

  {{#author?}}
    <p>{{deletion_link}}</p>
  {{/author?}}
</div>

The presenter will be responsible for providing the data for the components. Add the necessary Ruby code to the presenter:

# app/presenters/posts/comment_presenter.rb
class Posts::CommentPresenter < Curly::Presenter
  presents :comment

  def body
    SafeMarkdown.render(@comment.body)
  end

  def author_link
    link_to @comment.author.name, @comment.author, rel: "author"
  end

  def deletion_link
    link_to "Delete", @comment, method: :delete
  end

  def time_ago
    time_ago_in_words(@comment.created_at)
  end

  def author?
    @comment.author == current_user
  end
end

The partial can now be rendered like any other, e.g. by calling

render 'comment', comment: comment
render comment
render collection: post.comments

Curly components are surrounded by curly brackets, e.g. {{hello}}. They always map to a public method on the presenter class, in this case #hello. Methods ending in a question mark can be used for conditional blocks, e.g. {{#admin?}} ... {{/admin?}}.

Identifiers

Curly components can specify an identifier using the so-called dot notation: {{x.y.z}}. This can be very useful if the data you're accessing is hierarchical in nature. One common example is I18n:

<h1>{{i18n.homepage.header}}</h1>

# In the presenter, the identifier is passed as an argument to the method. The
# argument will always be a String.
def i18n(key)
  translate(key)
end

The identifier is separated from the component name with a dot. If the presenter method has a default value for the argument, the identifier is optional – otherwise it's mandatory.

Attributes

In addition to an identifier, Curly components can be annotated with attributes. These are key-value pairs that affect how a component is rendered.

The syntax is reminiscent of HTML:

<div>{{sidebar rows=3 width=200px title="I'm the sidebar!"}}</div>

The presenter method that implements the component must have a matching keyword argument:

def sidebar(rows: "1", width: "100px", title:); end

All argument values will be strings. A compilation error will be raised if

an attribute is used in a component without a matching keyword argument being present in the method definition; or
a required keyword argument in the method definition is not set as an attribute in the component.

You can define default values using Ruby's own syntax. Additionally, if the presenter method accepts arbitrary keyword arguments using the **doublesplat syntax then all attributes will be valid for the component, e.g.

def greetings(**names)
  names.map {|name, greeting| "#{name}: #{greeting}!" }.join("\n")
end

{{greetings alice=hello bob=hi}}
<!-- The above would be rendered as: -->
alice: hello!
bob: hi!

Note that since keyword arguments in Ruby are represented as Symbol objects, which are not garbage collected in Ruby versions less than 2.2, accepting arbitrary attributes represents a security vulnerability if your application allows untrusted Curly templates to be rendered. Only use this feature with trusted templates if you're not on Ruby 2.2 yet.

Conditional blocks

If there is some content you only want rendered under specific circumstances, you can use conditional blocks. The {{#admin?}}...{{/admin?}} syntax will only render the content of the block if the admin? method on the presenter returns true, while the {{^admin?}}...{{/admin?}} syntax will only render the content if it returns false.

Both forms can have an identifier: {{#locale.en?}}...{{/locale.en?}} will only render the block if the locale? method on the presenter returns true given the argument "en". Here's how to implement that method in the presenter:

class SomePresenter < Curly::Presenter
  # Allows rendering content only if the locale matches a specified identifier.
  def locale?(identifier)
    current_locale == identifier
  end
end

Furthermore, attributes can be set on the block. These only need to be specified when opening the block, not when closing it:

{{#square? width=3 height=3}}
  <p>It's square!</p>
{{/square?}}

Attributes work the same way as they do for normal components.

Collection blocks

Sometimes you want to render one or more items within the current template, and splitting out a separate template and rendering that in the presenter is too much overhead. You can instead define the template that should be used to render the items inline in the current template using the collection block syntax.

Collection blocks are opened using an asterisk:

{{*comments}}
  <li>{{body}} ({{author_name}})</li>
{{/comments}}

The presenter will need to expose the method #comments, which should return a collection of objects:

class Posts::ShowPresenter < Curly::Presenter
  presents :post

  def comments
    @post.comments
  end
end

The template within the collection block will be used to render each item, and it will be backed by a presenter named after the component – in this case, comments. The name will be singularized and Curly will try to find the presenter class in the following order:

Posts::ShowPresenter::CommentPresenter
Posts::CommentPresenter
CommentPresenter

This allows you some flexibility with regards to how you want to organize these nested templates and presenters.

Note that the nested template will only have access to the methods on the nested presenter, but all variables passed to the "parent" presenter will be forwarded to the nested presenter. In addition, the current item in the collection will be passed, as well as that item's index in the collection:

class Posts::CommentPresenter < Curly::Presenter
  presents :post, :comment, :comment_counter

  def number
    # `comment_counter` is automatically set to the item's index in the collection,
    # starting with 1.
    @comment_counter
  end

  def body
    @comment.body
  end

  def author_name
    @comment.author.name
  end
end

Collection blocks are an alternative to splitting out a separate template and rendering that from the presenter – which solution is best depends on your use case.

Context blocks

While collection blocks allow you to define the template that should be used to render items in a collection right within the parent template, context blocks allow you to define the template for an arbitrary context. This is very powerful, and can be used to define widget-style components and helpers, and provide an easy way to work with structured data. Let's say you have a comment form on your page, and you'd rather keep the template inline. A simple template could look like:

<!-- post.html.curly -->
<h1>{{title}}</h1>
{{body}}

{{@comment_form}}
  <b>Name: </b> {{name_field}}<br>
  <b>E-mail: </b> {{email_field}}<br>
  {{comment_field}}

  {{submit_button}}
{{/comment_form}}

Note that an @ character is used to denote a context block. Like with collection blocks, a separate presenter class is used within the block, and a simple convention is used to find it. The name of the context component (in this case, comment_form) will be camel cased, and the current presenter's namespace will be searched:

class PostPresenter < Curly::Presenter
  presents :post
  def title; @post.title; end
  def body; markdown(@post.body); end

  # A context block method *must* take a block argument. The return value
  # of the method will be used when rendering. Calling the block argument will
  # render the nested template. If you pass a value when calling the block
  # argument it will be passed to the presenter.
  def comment_form(&block)
    form_for(Comment.new, &block)
  end

  # The presenter name is automatically deduced.
  class CommentFormPresenter < Curly::Presenter
    # The value passed to the block argument will be passed in a parameter named
    # after the component.
    presents :comment_form

    # Any parameters passed to the parent presenter will be forwarded to this
    # presenter as well.
    presents :post

    def name_field
      @comment_form.text_field :name
    end

    # ...
  end
end

Context blocks were designed to work well with Rails' helper methods such as form_for and content_tag, but you can also work directly with the block. For instance, if you want to directly control the value that is passed to the nested presenter, you can call the call method on the block yourself:

def author(&block)
  content_tag :div, class: "author" do
    # The return value of `call` will be the result of rendering the nested template
    # with the argument. You can post-process the string if you want.
    block.call(@post.author)
  end
end

Context shorthand syntax

If you find yourself opening a context block just in order to use a single component, e.g. {{@author}}{{name}}{{/author}}, you can use the shorthand syntax instead: {{author:name}}. This works for all component types, e.g.

{{#author:admin?}}
  <p>The author is an admin!</p>
{{/author:admin?}}

The syntax works for nested contexts as well, e.g. {{comment:author:name}}. Any identifier and attributes are passed to the target component, which in this example would be {{name}}.

Setting up state

Although most code in Curly presenters should be free of side effects, sometimes side effects are required. One common example is defining content for a content_for block.

If a Curly presenter class defines a setup! method, it will be called before the view is rendered:

class PostPresenter < Curly::Presenter
  presents :post

  def setup!
    content_for :title, post.title

    content_for :sidebar do
      render 'post_sidebar', post: post
    end
  end
end

Escaping Curly syntax

In order to have {{ appear verbatim in the rendered HTML, use the triple Curly escape syntax:

This is {{{escaped}}.

You don't need to escape the closing }}.

Comments

If you want to add comments to your Curly templates that are not visible in the rendered HTML, use the following syntax:

{{! This is some interesting stuff }}

Presenters

Presenters are classes that inherit from Curly::Presenter – they're usually placed in app/presenters/, but you can put them anywhere you'd like. The name of the presenter classes match the virtual path of the view they're part of, so if your controller is rendering posts/show, the Posts::ShowPresenter class will be used. Note that Curly is only used to render a view if a template can be found – in this case, at app/views/posts/show.html.curly.

Presenters can declare a list of accepted variables using the presents method:

class Posts::ShowPresenter < Curly::Presenter
  presents :post
end

A variable can have a default value:

class Posts::ShowPresenter < Curly::Presenter
  presents :post
  presents :comment, default: nil
end

Any public method defined on the presenter is made available to the template as a component:

class Posts::ShowPresenter < Curly::Presenter
  presents :post

  def title
    @post.title
  end

  def author_link
    # You can call any Rails helper from within a presenter instance:
    link_to author.name, profile_path(author), rel: "author"
  end

  private

  # Private methods are not available to the template, so they're safe to
  # use.
  def author
    @post.author
  end
end

Presenter methods can even take an argument. Say your Curly template has the content {{t.welcome_message}}, where welcome_message is an I18n key. The following presenter method would make the lookup work:

def t(key)
  translate(key)
end

That way, simple ``functions'' can be added to the Curly language. Make sure these do not have any side effects, though, as an important part of Curly is the idempotence of the templates.

Layouts and content blocks

Both layouts and content blocks (see content_for) use yield to signal that content can be inserted. Curly works just like ERB, so calling yield with no arguments will make the view usable as a layout, while passing a Symbol will make it try to read a content block with the given name:

# Given you have the following Curly template in
# app/views/layouts/application.html.curly
#
#   <html>
#     <head>
#       <title>{{title}}</title>
#     </head>
#     <body>
#       <div id="sidebar">{{sidebar}}</div>
#       {{body}}
#     </body>
#   </html>
#
class ApplicationLayout < Curly::Presenter
  def title
    "You can use methods just like in any other presenter!"
  end

  def sidebar
    # A view can call `content_for(:sidebar) { "some HTML here" }`
    yield :sidebar
  end

  def body
    # The view will be rendered and inserted here:
    yield
  end
end

Rails helper methods

In order to make a Rails helper method available as a component in your template, use the exposes_helper method:

class Layouts::ApplicationPresenter < Curly::Presenter
  # The components {{sign_in_path}} and {{root_path}} are made available.
  exposes_helper :sign_in_path, :root_path
end

Testing

Presenters can be tested directly, but sometimes it makes sense to integrate with Rails on some levels. Currently, only RSpec is directly supported, but you can easily instantiate a presenter:

SomePresenter.new(context, assigns)

context is a view context, i.e. an object that responds to render, has all the helper methods you expect, etc. You can pass in a test double and see what you need to stub out. assigns is the hash containing the controller and local assigns. You need to pass in a key for each argument the presenter expects.

Testing with RSpec

In order to test presenters with RSpec, make sure you have rspec-rails in your Gemfile. Given the following presenter:

# app/presenters/posts/show_presenter.rb
class Posts::ShowPresenter < Curly::Presenter
  presents :post

  def body
    Markdown.render(@post.body)
  end
end

You can test the presenter methods like this:

# You can put this in your `spec_helper.rb`.
require 'curly/rspec'

# spec/presenters/posts/show_presenter_spec.rb
describe Posts::ShowPresenter, type: :presenter do
  describe "#body" do
    it "renders the post's body as Markdown" do
      assign(:post, double(:post, body: "**hello!**"))
      expect(presenter.body).to eq "<strong>hello!</strong>"
    end
  end
end

Note that your spec must be tagged with type: :presenter.

Examples

Here is a simple Curly template – it will be looked up by Rails automatically.

<!-- app/views/posts/show.html.curly -->
<h1>{{title}}<h1>
<p class="author">{{author}}</p>
<p>{{description}}</p>

{{comment_form}}

<div class="comments">
  {{comments}}
</div>

When rendering the template, a presenter is automatically instantiated with the variables assigned in the controller or the render call. The presenter declares the variables it expects with presents, which takes a list of variables names.

# app/presenters/posts/show_presenter.rb
class Posts::ShowPresenter < Curly::Presenter
  presents :post

  def title
    @post.title
  end

  def author
    link_to(@post.author.name, @post.author, rel: "author")
  end

  def description
    Markdown.new(@post.description).to_html.html_safe
  end

  def comments
    render 'comment', collection: @post.comments
  end

  def comment_form
    if @post.comments_allowed?
      render 'comment_form', post: @post
    else
      content_tag(:p, "Comments are disabled for this post")
    end
  end
end

Caching

Caching is handled at two levels in Curly – statically and dynamically. Static caching concerns changes to your code and templates introduced by deploys. If you do not wish to clear your entire cache every time you deploy, you need a way to indicate that some view, helper, or other piece of logic has changed.

Dynamic caching concerns changes that happen on the fly, usually made by your users in the running system. You wish to cache a view or a partial and have it expire whenever some data is updated – usually whenever a specific record is changed.

Dynamic Caching

Because of the way logic is contained in presenters, caching entire views or partials by the data they present becomes exceedingly straightforward. Simply define a #cache_key method that returns a non-nil object, and the return value will be used to cache the template.

Whereas in ERB you would include the cache call in the template itself:

<% cache([@post, signed_in?]) do %>
  ...
<% end %>

In Curly you would instead declare it in the presenter:

class Posts::ShowPresenter < Curly::Presenter
  presents :post

  def cache_key
    [@post, signed_in?]
  end
end

Likewise, you can add a #cache_duration method if you wish to automatically expire the fragment cache:

class Posts::ShowPresenter < Curly::Presenter
  ...

  def cache_duration
    30.minutes
  end
end

In order to set any cache option, define a #cache_options method that returns a Hash of options:

class Posts::ShowPresenter < Curly::Presenter
  ...

  def cache_options
    { compress: true, namespace: "my-app" }
  end
end

Static Caching

Static caching will only be enabled for presenters that define a non-nil #cache_key method (see Dynamic Caching.)

In order to make a deploy expire the cache for a specific view, set the version of the view to something new, usually by incrementing by one:

class Posts::ShowPresenter < Curly::Presenter
  version 3

  def cache_key
    # Some objects
  end
end

This will change the cache keys for all instances of that view, effectively expiring the old cache entries.

This works well for views, or for partials that are rendered in views that themselves are not cached. If the partial is nested within a view that is cached, however, the outer cache will not be expired. The solution is to register that the inner partial is a dependency of the outer one such that Curly can automatically deduce that the outer partial cache should be expired:

class Posts::ShowPresenter < Curly::Presenter
  version 3
  depends_on 'posts/comment'

  def cache_key
    # Some objects
  end
end

class Posts::CommentPresenter < Curly::Presenter
  version 4

  def cache_key
    # Some objects
  end
end

Now, if the version of Posts::CommentPresenter is bumped, the cache keys for both presenters would change. You can register any number of view paths with depends_on.

Curly integrates well with the caching mechanism in Rails 4 (or Cache Digests in Rails 3), so the dependencies defined with depends_on will be tracked by Rails. This will allow you to deploy changes to your templates and have the relevant caches automatically expire.

Thanks

Thanks to Zendesk for sponsoring the work on Curly.

Contributors

Daniel Schierbeck (@dasch)
Benjamin Quorning (@bquorning)
Jeremy Rodi (@medcat)
Alisson Cavalcante Agiani (@thelinuxlich)
Łukasz Niemier (@hauleth)
Cristian Planas (@Gawyn)
Steven Davidovitz (@steved)

Build Status

Author: zendesk
Source code: https://github.com/zendesk/curly

#ruby #ruby-on-rails

Nigel Uys

1647475980

Go-cache: An in-memory Key:value Store/cache Library for Go

go-cache

go-cache is an in-memory key:value store/cache similar to memcached that is suitable for applications running on a single machine. Its major advantage is that, being essentially a thread-safe map[string]interface{} with expiration times, it doesn't need to serialize or transmit its contents over the network.

Any object can be stored, for a given duration or forever, and the cache can be safely used by multiple goroutines.

Although go-cache isn't meant to be used as a persistent datastore, the entire cache can be saved to and loaded from a file (using c.Items() to retrieve the items map to serialize, and NewFrom() to create a cache from a deserialized one) to recover from downtime quickly. (See the docs for NewFrom() for caveats.)

Installation

go get github.com/patrickmn/go-cache

Usage

import (
    "fmt"
    "github.com/patrickmn/go-cache"
    "time"
)

func main() {
    // Create a cache with a default expiration time of 5 minutes, and which
    // purges expired items every 10 minutes
    c := cache.New(5*time.Minute, 10*time.Minute)

    // Set the value of the key "foo" to "bar", with the default expiration time
    c.Set("foo", "bar", cache.DefaultExpiration)

    // Set the value of the key "baz" to 42, with no expiration time
    // (the item won't be removed until it is re-set, or removed using
    // c.Delete("baz")
    c.Set("baz", 42, cache.NoExpiration)

    // Get the string associated with the key "foo" from the cache
    foo, found := c.Get("foo")
    if found {
        fmt.Println(foo)
    }

    // Since Go is statically typed, and cache values can be anything, type
    // assertion is needed when values are being passed to functions that don't
    // take arbitrary types, (i.e. interface{}). The simplest way to do this for
    // values which will only be used once--e.g. for passing to another
    // function--is:
    foo, found := c.Get("foo")
    if found {
        MyFunction(foo.(string))
    }

    // This gets tedious if the value is used several times in the same function.
    // You might do either of the following instead:
    if x, found := c.Get("foo"); found {
        foo := x.(string)
        // ...
    }
    // or
    var foo string
    if x, found := c.Get("foo"); found {
        foo = x.(string)
    }
    // ...
    // foo can then be passed around freely as a string

    // Want performance? Store pointers!
    c.Set("foo", &MyStruct, cache.DefaultExpiration)
    if x, found := c.Get("foo"); found {
        foo := x.(*MyStruct)
            // ...
    }
}

Reference

godoc or http://godoc.org/github.com/patrickmn/go-cache

Author: Patrickmn
Source Code: https://github.com/patrickmn/go-cache
License: MIT License

#go #golang #cache

Nigel Uys

1647460980

Fastcache: Fast Thread-safe inmemory Cache for Big Number Of Entries

fastcache - fast thread-safe inmemory cache for big number of entries in Go

Features

Fast. Performance scales on multi-core CPUs. See benchmark results below.
Thread-safe. Concurrent goroutines may read and write into a single cache instance.
The fastcache is designed for storing big number of entries without GC overhead.
Fastcache automatically evicts old entries when reaching the maximum cache size set on its creation.
Simple API.
Simple source code.
Cache may be saved to file and loaded from file.
Works on Google AppEngine.

Benchmarks

Fastcache performance is compared with BigCache, standard Go map and sync.Map.

GOMAXPROCS=4 go test github.com/VictoriaMetrics/fastcache -bench='Set|Get' -benchtime=10sgoos: linuxgoarch: amd64pkg: github.com/VictoriaMetrics/fastcacheBenchmarkBigCacheSet-4              2000      10566656 ns/op       6.20 MB/s     4660369 B/op           6 allocs/opBenchmarkBigCacheGet-4              2000       6902694 ns/op       9.49 MB/s      684169 B/op      131076 allocs/opBenchmarkBigCacheSetGet-4           1000      17579118 ns/op       7.46 MB/s     5046744 B/op      131083 allocs/opBenchmarkCacheSet-4                 5000       3808874 ns/op      17.21 MB/s        1142 B/op           2 allocs/opBenchmarkCacheGet-4                 5000       3293849 ns/op      19.90 MB/s        1140 B/op           2 allocs/opBenchmarkCacheSetGet-4              2000       8456061 ns/op      15.50 MB/s        2857 B/op           5 allocs/opBenchmarkStdMapSet-4                2000      10559382 ns/op       6.21 MB/s      268413 B/op       65537 allocs/opBenchmarkStdMapGet-4                5000       2687404 ns/op      24.39 MB/s        2558 B/op          13 allocs/opBenchmarkStdMapSetGet-4              100     154641257 ns/op       0.85 MB/s      387405 B/op       65558 allocs/opBenchmarkSyncMapSet-4                500      24703219 ns/op       2.65 MB/s     3426543 B/op      262411 allocs/opBenchmarkSyncMapGet-4               5000       2265892 ns/op      28.92 MB/s        2545 B/op          79 allocs/opBenchmarkSyncMapSetGet-4            1000      14595535 ns/op       8.98 MB/s     3417190 B/op      262277 allocs/op

MB/s column here actually means millions of operations per second. As you can see, fastcache is faster than the BigCache in all the cases. fastcache is faster than the standard Go map and sync.Map on workloads with inserts.

Limitations

Keys and values must be byte slices. Other types must be marshaled before storing them in the cache.
Big entries with sizes exceeding 64KB must be stored via distinct API.
There is no cache expiration. Entries are evicted from the cache only on cache size overflow. Entry deadline may be stored inside the value in order to implement cache expiration.

Architecture details

The cache uses ideas from BigCache:

The cache consists of many buckets, each with its own lock. This helps scaling the performance on multi-core CPUs, since multiple CPUs may concurrently access distinct buckets.
Each bucket consists of a hash(key) -> (key, value) position map and 64KB-sized byte slices (chunks) holding encoded (key, value) entries. Each bucket contains only O(chunksCount) pointers. For instance, 64GB cache would contain ~1M pointers, while similarly-sized map[string][]byte would contain ~1B pointers for short keys and values. This would lead to huge GC overhead.

64KB-sized chunks reduce memory fragmentation and the total memory usage comparing to a single big chunk per bucket. Chunks are allocated off-heap if possible. This reduces total memory usage because GC collects unused memory more frequently without the need in GOGC tweaking.

Users

Fastcache has been extracted from VictoriaMetrics sources. See this article for more info about VictoriaMetrics.

FAQ

What is the difference between `fastcache` and other similar caches like BigCache or FreeCache?

Fastcache is faster. See benchmark results above.
Fastcache uses less memory due to lower heap fragmentation. This allows saving many GBs of memory on multi-GB caches.
Fastcache API is simpler. The API is designed to be used in zero-allocation mode.

Why `fastcache` doesn't support cache expiration?

Because we don't need cache expiration in VictoriaMetrics. Cached entries inside VictoriaMetrics never expire. They are automatically evicted on cache size overflow.

It is easy to implement cache expiration on top of fastcache by caching values with marshaled deadlines and verifying deadlines after reading these values from the cache.

Why `fastcache` doesn't support advanced features such as thundering herd protection or callbacks on entries' eviction?

Because these features would complicate the code and would make it slower. Fastcache source code is simple - just copy-paste it and implement the feature you want on top of it.

Author: VictoriaMetrics
Source Code: https://github.com/VictoriaMetrics/fastcache
License: MIT License

#go #golang #caching #cache