Guidelines on Benchmarking in Rust

This post covers:

• Benchmark reports for contributors
• Benchmark reports for users
• Profiling with valgrind / kcachegrind
• Reproducible benchmarks and graphics
• Tips for benchmark behavior and benchmarking other languages

Lots of libraries advertise how performant they are with phrases like “blazingly fast”, “lightning fast”, “10x faster than y” – oftentimes written in the project’s main description. If performance is a library’s main selling point then I expect for there to be instructions for reproducible benchmarks and lucid visualizations. Nothing less. Else it’s an all talk and no action situation, especially because great benchmark frameworks exist in nearly all languages.

I find performance touting libraries without a benchmark foundation analogous to GUI libraries without screenshots.

This post mainly focuses on creating satisfactory benchmarks in Rust, but the main points here can be extrapolated.

Use Criterion

If there is one thing to takeaway from this post: benchmark with Criterion.

Never written a Rust benchmark? Use Criterion.

Only written benchmarks against Rust’s built in bench harness? Switch to Criterion:

• Benchmark on stable Rust (I personally have eschewed nightly Rust for the last few months!)
• Reports statistically significant changes between runs (to test branches or varying implementations).
• Criterion is actively developed

Get started with Criterion

When running benchmarks, the commandline output will look something like:

sixtyfour_bits/bitter_byte_checked
                        time:   [1.1052 us 1.1075 us 1.1107 us]
                        thrpt:  [6.7083 GiB/s 6.7274 GiB/s 6.7416 GiB/s]
                 change:
                        time:   [-1.0757% -0.0366% +0.8695%] (p = 0.94 > 0.05)
                        thrpt:  [-0.8621% +0.0367% +1.0874%]
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild
  6 (6.00%) high severe

This output is good for contributors in pull requests or issues, but I better not see this in a project’s readme! Criterion generates reports automatically that are 100x better than console output.

Criterion Reports

Below is a criterion generated plot from one of my projects: bitter. I’m only including one of the nearly 1800 graphics generated by criterion, the one chosen captures the heart of a single benchmark measuring Rust bit parsing libraries across read sizes (in bits).

This chart shows the mean measured time for each function as the input (or the size of the input) increases.

Out of all the auto-generated graphics, I would consider this the only visualization that could be displayed for a more general audience, but I still wouldn’t use it this way. This chart lacks context, and it’s not clear what graphic is trying to convey. I’d even be worried about one drawing inappropriate conclusions (pop quiz time: there is a superior library for all parameters, which one is it?).

It’s my opinion that the graphics that criterion generates are perfect for contributors of the project as there is no dearth of info. Criterion generates graphics that break down mean, median, standard deviation, MAD, etc, which are invaluable when trying to pinpoint areas of improvement.

As a comparison, here is the graphic I created using the same data:

It may be hard to believe that the same data, but here are the improvements:

• A more self-explanatory title
• Stylistically differentiate “us vs them”. In the above graphic, bitter methods are solid lines while “them” are dashed
• More accessible x, y axis values
• Eyes are drawn to the upper right, as the throughput value stands out which is desirable as it shows bitter in a good light. It’s more clear which libraries perform better.

These add context that Criterion shouldn’t be expected to know. I recommend spending the time to dress reports up before presenting it to a wider audience.

#rust #developer

What is GEEK

Buddha Community

Serde Rust: Serialization Framework for Rust

Serde

*Serde is a framework for serializing and deserializing Rust data structures efficiently and generically.*

You may be looking for:

• An overview of Serde
• Data formats supported by Serde
• Setting up #[derive(Serialize, Deserialize)]
• Examples
• API documentation
• Release notes

Serde in action

Click to show Cargo.toml. Run this code in the playground.

[dependencies]

# The core APIs, including the Serialize and Deserialize traits. Always
# required when using Serde. The "derive" feature is only required when
# using #[derive(Serialize, Deserialize)] to make Serde work with structs
# and enums defined in your crate.
serde = { version = "1.0", features = ["derive"] }

# Each data format lives in its own crate; the sample code below uses JSON
# but you may be using a different one.
serde_json = "1.0"

 

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Debug)]
struct Point {
    x: i32,
    y: i32,
}

fn main() {
    let point = Point { x: 1, y: 2 };

    // Convert the Point to a JSON string.
    let serialized = serde_json::to_string(&point).unwrap();

    // Prints serialized = {"x":1,"y":2}
    println!("serialized = {}", serialized);

    // Convert the JSON string back to a Point.
    let deserialized: Point = serde_json::from_str(&serialized).unwrap();

    // Prints deserialized = Point { x: 1, y: 2 }
    println!("deserialized = {:?}", deserialized);
}

Getting help

Serde is one of the most widely used Rust libraries so any place that Rustaceans congregate will be able to help you out. For chat, consider trying the #rust-questions or #rust-beginners channels of the unofficial community Discord (invite: https://discord.gg/rust-lang-community), the #rust-usage or #beginners channels of the official Rust Project Discord (invite: https://discord.gg/rust-lang), or the #general stream in Zulip. For asynchronous, consider the [rust] tag on StackOverflow, the /r/rust subreddit which has a pinned weekly easy questions post, or the Rust Discourse forum. It's acceptable to file a support issue in this repo but they tend not to get as many eyes as any of the above and may get closed without a response after some time.

Download Details:
Author: serde-rs
Source Code: https://github.com/serde-rs/serde
License: View license

#rust  #rustlang 

Serde JSON: JSON Support for Serde Framework

Serde JSON

Serde is a framework for serializing and deserializing Rust data structures efficiently and generically.

[dependencies]
serde_json = "1.0"

You may be looking for:

• JSON API documentation
• Serde API documentation
• Detailed documentation about Serde
• Setting up #[derive(Serialize, Deserialize)]
• Release notes

JSON is a ubiquitous open-standard format that uses human-readable text to transmit data objects consisting of key-value pairs.

{
    "name": "John Doe",
    "age": 43,
    "address": {
        "street": "10 Downing Street",
        "city": "London"
    },
    "phones": [
        "+44 1234567",
        "+44 2345678"
    ]
}

There are three common ways that you might find yourself needing to work with JSON data in Rust.

• As text data. An unprocessed string of JSON data that you receive on an HTTP endpoint, read from a file, or prepare to send to a remote server.
• As an untyped or loosely typed representation. Maybe you want to check that some JSON data is valid before passing it on, but without knowing the structure of what it contains. Or you want to do very basic manipulations like insert a key in a particular spot.
• As a strongly typed Rust data structure. When you expect all or most of your data to conform to a particular structure and want to get real work done without JSON's loosey-goosey nature tripping you up.

Serde JSON provides efficient, flexible, safe ways of converting data between each of these representations.

Operating on untyped JSON values

Any valid JSON data can be manipulated in the following recursive enum representation. This data structure is serde_json::Value.

enum Value {
    Null,
    Bool(bool),
    Number(Number),
    String(String),
    Array(Vec<Value>),
    Object(Map<String, Value>),
}

A string of JSON data can be parsed into a serde_json::Value by the serde_json::from_str function. There is also from_slice for parsing from a byte slice &[u8] and from_reader for parsing from any io::Read like a File or a TCP stream.

use serde_json::{Result, Value};

fn untyped_example() -> Result<()> {
    // Some JSON input data as a &str. Maybe this comes from the user.
    let data = r#"
        {
            "name": "John Doe",
            "age": 43,
            "phones": [
                "+44 1234567",
                "+44 2345678"
            ]
        }"#;

    // Parse the string of data into serde_json::Value.
    let v: Value = serde_json::from_str(data)?;

    // Access parts of the data by indexing with square brackets.
    println!("Please call {} at the number {}", v["name"], v["phones"][0]);

    Ok(())
}

The result of square bracket indexing like v["name"] is a borrow of the data at that index, so the type is &Value. A JSON map can be indexed with string keys, while a JSON array can be indexed with integer keys. If the type of the data is not right for the type with which it is being indexed, or if a map does not contain the key being indexed, or if the index into a vector is out of bounds, the returned element is Value::Null.

When a Value is printed, it is printed as a JSON string. So in the code above, the output looks like Please call "John Doe" at the number "+44 1234567". The quotation marks appear because v["name"] is a &Value containing a JSON string and its JSON representation is "John Doe". Printing as a plain string without quotation marks involves converting from a JSON string to a Rust string with as_str() or avoiding the use of Value as described in the following section.

The Value representation is sufficient for very basic tasks but can be tedious to work with for anything more significant. Error handling is verbose to implement correctly, for example imagine trying to detect the presence of unrecognized fields in the input data. The compiler is powerless to help you when you make a mistake, for example imagine typoing v["name"] as v["nmae"] in one of the dozens of places it is used in your code.

Parsing JSON as strongly typed data structures

Serde provides a powerful way of mapping JSON data into Rust data structures largely automatically.

use serde::{Deserialize, Serialize};
use serde_json::Result;

#[derive(Serialize, Deserialize)]
struct Person {
    name: String,
    age: u8,
    phones: Vec<String>,
}

fn typed_example() -> Result<()> {
    // Some JSON input data as a &str. Maybe this comes from the user.
    let data = r#"
        {
            "name": "John Doe",
            "age": 43,
            "phones": [
                "+44 1234567",
                "+44 2345678"
            ]
        }"#;

    // Parse the string of data into a Person object. This is exactly the
    // same function as the one that produced serde_json::Value above, but
    // now we are asking it for a Person as output.
    let p: Person = serde_json::from_str(data)?;

    // Do things just like with any other Rust data structure.
    println!("Please call {} at the number {}", p.name, p.phones[0]);

    Ok(())
}

This is the same serde_json::from_str function as before, but this time we assign the return value to a variable of type Person so Serde will automatically interpret the input data as a Person and produce informative error messages if the layout does not conform to what a Person is expected to look like.

Any type that implements Serde's Deserialize trait can be deserialized this way. This includes built-in Rust standard library types like Vec<T> and HashMap<K, V>, as well as any structs or enums annotated with #[derive(Deserialize)].

Once we have p of type Person, our IDE and the Rust compiler can help us use it correctly like they do for any other Rust code. The IDE can autocomplete field names to prevent typos, which was impossible in the serde_json::Value representation. And the Rust compiler can check that when we write p.phones[0], then p.phones is guaranteed to be a Vec<String> so indexing into it makes sense and produces a String.

The necessary setup for using Serde's derive macros is explained on the Using derive page of the Serde site.

Constructing JSON values

Serde JSON provides a json! macro to build serde_json::Value objects with very natural JSON syntax.

use serde_json::json;

fn main() {
    // The type of `john` is `serde_json::Value`
    let john = json!({
        "name": "John Doe",
        "age": 43,
        "phones": [
            "+44 1234567",
            "+44 2345678"
        ]
    });

    println!("first phone number: {}", john["phones"][0]);

    // Convert to a string of JSON and print it out
    println!("{}", john.to_string());
}

The Value::to_string() function converts a serde_json::Value into a String of JSON text.

One neat thing about the json! macro is that variables and expressions can be interpolated directly into the JSON value as you are building it. Serde will check at compile time that the value you are interpolating is able to be represented as JSON.

let full_name = "John Doe";
let age_last_year = 42;

// The type of `john` is `serde_json::Value`
let john = json!({
    "name": full_name,
    "age": age_last_year + 1,
    "phones": [
        format!("+44 {}", random_phone())
    ]
});

This is amazingly convenient, but we have the problem we had before with Value: the IDE and Rust compiler cannot help us if we get it wrong. Serde JSON provides a better way of serializing strongly-typed data structures into JSON text.

Creating JSON by serializing data structures

A data structure can be converted to a JSON string by serde_json::to_string. There is also serde_json::to_vec which serializes to a Vec<u8> and serde_json::to_writer which serializes to any io::Write such as a File or a TCP stream.

use serde::{Deserialize, Serialize};
use serde_json::Result;

#[derive(Serialize, Deserialize)]
struct Address {
    street: String,
    city: String,
}

fn print_an_address() -> Result<()> {
    // Some data structure.
    let address = Address {
        street: "10 Downing Street".to_owned(),
        city: "London".to_owned(),
    };

    // Serialize it to a JSON string.
    let j = serde_json::to_string(&address)?;

    // Print, write to a file, or send to an HTTP server.
    println!("{}", j);

    Ok(())
}

Any type that implements Serde's Serialize trait can be serialized this way. This includes built-in Rust standard library types like Vec<T> and HashMap<K, V>, as well as any structs or enums annotated with #[derive(Serialize)].

Performance

It is fast. You should expect in the ballpark of 500 to 1000 megabytes per second deserialization and 600 to 900 megabytes per second serialization, depending on the characteristics of your data. This is competitive with the fastest C and C++ JSON libraries or even 30% faster for many use cases. Benchmarks live in the serde-rs/json-benchmark repo.

Getting help

Serde is one of the most widely used Rust libraries, so any place that Rustaceans congregate will be able to help you out. For chat, consider trying the #rust-questions or #rust-beginners channels of the unofficial community Discord (invite: https://discord.gg/rust-lang-community), the #rust-usage or #beginners channels of the official Rust Project Discord (invite: https://discord.gg/rust-lang), or the #general stream in Zulip. For asynchronous, consider the [rust] tag on StackOverflow, the /r/rust subreddit which has a pinned weekly easy questions post, or the Rust Discourse forum. It's acceptable to file a support issue in this repo, but they tend not to get as many eyes as any of the above and may get closed without a response after some time.

No-std support

As long as there is a memory allocator, it is possible to use serde_json without the rest of the Rust standard library. This is supported on Rust 1.36+. Disable the default "std" feature and enable the "alloc" feature:

[dependencies]
serde_json = { version = "1.0", default-features = false, features = ["alloc"] }

For JSON support in Serde without a memory allocator, please see the serde-json-core crate.

Link: https://crates.io/crates/serde_json

#rust  #rustlang  #encode   #json 

How Benchmarking Your Code Will Improve Your Ruby Skills

Learning to code is a path full of struggles, and learning Ruby isn’t the exception. But you’ll agree with me, that practice is the best way to learn and develop your skills.

Out there are plenty of sites to choose from where you can achieve this. But after n amount of solved challenges, you’ll start questioning your solutions.

You’ll start feeling, that is not enough to come up with an answer to the challenge. And you’ll be wondering if there is a better, faster, and less consuming memory approach.

If this is your case, if you have reached this stage, then you’re more than ready to begin benchmarking your code. And if you’re not, well, let me tell you that learning this will step up your game.

So, what is this “benchmarking” thing anyway?..

Benchmarking is the process of measuring the performance of something… in this case the performance of your code.

Benchmarking is the process of measuring the performance of something.

And that’s it, there is nothing more to add.

Yeah, I know what you’re thinking, that knowing the meaning doesn’t serve your purpose. But I’m positive that at least it will give you a broad idea.

With this in mind, the next question to answer is, “How benchmarking my code, will help me improve my coding skills?”.

This is an easy one. Benchmarking gives you knowledge, and knowledge is power. The better understanding of your code, the better code you’ll start programming. Is that simple!.

Benchmarking gives you knowledge, and knowledge is power.

Benchmarking gives you a simple view of how your code is performing. This view focuses on three main areas: elapsed time, memory, and iterations per second.

1. Elapsed time: is the amount of time your code takes to finish a task.
2. Memory: is the allocated space in your drive that your code occupies to solve the task.
3. Iterations per second: is the total number of repetitions your code can do the same task, over and over, in a second.

Ok, at this point I assume you have understood the basics of benchmarking. Hopefully, you are also saying to yourself - “Yeah, this is what I was missing, this will help me become a better programmer!”.

If you don’t think that way yet; I hope knowing how it works will do.

Now, how do we apply this?

Ruby makes it easy for us, as it already has a “Benchmarking” class, but we’ll complement it with two other ruby gems.

1. Benchmarking-ips
2. Benchmarking-memory

Before describing the steps to benchmark your code. I’m assuming you are using a linux/unix distribution, and have a working version of Ruby installed.

If you’re running some other OS like Windows, the next series of commands, probably, won’t work. But the benchmarking steps are going to be identical.

Let’s begin!.

a. The first step is to install this rubygems in your system; for that, you can use this command:

gem install benchmark-ips benchmark-memory

b. The second **step **is creating a ruby file, I’m going to call it “benchmarking_ruby.rb”, but you can name it in any way you want:

touch benchmarking_ruby.rb

b.1 Open the file with your preferred text editor. This file will contain three main sections:

- Section 1: Here we use require to include and grant our file access to the gems recently installed and the Ruby Benchmark Class. Doing so, will allow us to use the benchmark methods to measure our code performance.

require 'benchmark'
require 'benchmark/memory'
require 'benchmark/ips'

- Section 2: We write the code that we want to benchmark.

def method_1(params)
    ## some code
end

def method_2(params)
    ## some code
end

Note:_ Benchmark works if you want to measure a single method but it’s best if you test at least two or more. Don’t worry too much about this right now, later on, I’ll show you an example and you’ll understand what I meant._

- Section 3: We set up the test.

This part is tricky but not complicated. Look at the image below and try to identify what’s happening.

def benchmark(params)
    Benchmark.bmbm do |x|
        x.report ("Method One") { method_1(params) }
        x.report ("Method Two") { method_2(params) }
    end

    Benchmark.memory do |x|
        x.report ("Method One") { method_1(params) }
        x.report ("Method Two") { method_2(params) }
        x.compare!
    end

    Benchmark.ips do |x|
        x.report ("Method One") { method_1(params) }
        x.report ("Method Two") { method_2(params) }
        x.compare!
    end
end

benchmark(params)

We are declaring a method called “benchmark” that accepts some parameters (or arguments).

def benchmark(params)
    ...
end

#ruby #benchmark #learn-to-code-ruby #rubygems #code-performance #how-to-benchmark-ruby #benchmark-ips #benchmark-memory

Rust Lang Course For Beginner In 2021: Guessing Game

 What we learn in this chapter:
- Rust number types and their default
- First exposure to #Rust modules and the std::io module to read input from the terminal
- Rust Variable Shadowing
- Rust Loop keyword
- Rust if/else
- First exposure to #Rust match keyword

=== Content:
00:00 - Intro & Setup
02:11 - The Plan
03:04 - Variable Secret
04:03 - Number Types
05:45 - Mutability recap
06:22 - Ask the user
07:45 - First intro to module std::io
08:29 - Rust naming conventions
09:22 - Read user input io:stdin().read_line(&mut guess)
12:46 - Break & Understand
14:20 - Parse string to number
17:10 - Variable Shadowing
18:46 - If / Else - You Win, You Loose
19:28 - Loop
20:38 - Match
23:19 - Random with rand
26:35 - Run it all
27:09 - Conclusion and next episode

#rust 

ULTIMATE Rust Lang Tutorial! - Publishing a Rust Crate

The ultimate Rust lang tutorial. Follow along as we go through the Rust lang book chapter by chapter.

📝Get the FREE Rust Cheatsheet: https://letsgetrusty.com/cheatsheet

The Rust book: https://doc.rust-lang.org/stable/book/​​

Chapters:
0:00​ Intro
0:43 Release Profiles
3:00 Documentation Comments
4:32 Commonly Used Sections
5:04 Documentation Comments as Tests
5:50 Commenting Contained Items
6:29 Exporting a Public API
8:44 Setting up Creates.io Account
9:54 Adding Metadata to a New Create
12:14 Publishing to Crates.io
12:49 Removing Version from Crates.io
13:37 Outro

#letsgetrusty​​ #rust​lang​ #tutorial

#rust #rust lang #rust crate