Garry Taylor

Garry Taylor

1642049491

Sharpliner: Use C# Instead of YAML to Define Your Azure DevOps Pipelines

Sharpliner is a .NET library that lets you use C# for Azure DevOps pipeline definition. Exchange YAML indentation problems for the type-safe environment of C# and let the intellisense speed up your work!

Getting started

All you have to do is reference our NuGet package in your project, override a class with your definition and dotnet build the project! Dead simple!

For more detailed steps, check our documentation.

Example

// Just override prepared abstract classes and `dotnet build` the project, nothing else is needed!
// You can also generate collections of definitions dynamically
//    see https://github.com/sharpliner/sharpliner/blob/main/docs/AzureDevOps/DefinitionCollections.md
// For a full list of classes you can override
//    see https://github.com/sharpliner/sharpliner/blob/main/src/Sharpliner/AzureDevOps/PublicDefinitions.cs
class PullRequestPipeline : SingleStagePipelineDefinition
{
    // Say where to publish the YAML to
    public override string TargetFile => "azure-pipelines.yml";
    public override TargetPathType TargetPathType => TargetPathType.RelativeToGitRoot;

    public override SingleStagePipeline Pipeline => new()
    {
        Pr = new PrTrigger("main"),

        Variables =
        {
            // YAML ${{ if }} conditions are available with handy macros that expand into the
            // expressions such as comparing branch names. We also have "else"
            If.IsBranch("net-6.0")
                .Variable("DotnetVersion", "6.0.100")
                .Group("net6-kv")
            .Else
                .Variable("DotnetVersion", "5.0.202"),
        },

        Jobs =
        {
            new Job("Build")
            {
                Pool = new HostedPool("Azure Pipelines", "windows-latest"),
                Steps =
                {
                    // Many tasks have helper methods for shorter notation
                    DotNet.Install.Sdk("$(DotnetVersion)").DisplayAs("Install .NET SDK"),

                    // You can also specify any pipeline task in full
                    Task("DotNetCoreCLI@2", "Build and test") with
                    {
                        Inputs = new()
                        {
                            { "command", "test" },
                            { "projects", "src/MyProject.sln" },
                        }
                    },

                    // If statements supported (almost) everywhere
                    If.IsPullRequest
                        // You can load script contents from a .ps1 file and inline them into YAML
                        // This way you can write scripts with syntax highlighting separately
                        .Step(Powershell.FromResourceFile("New-Report.ps1", "Create build report")),
                }
            }
        },
    };
}

Sharpliner features

Apart from the obvious benefits of using static type language with IDE support, not having to have to deal with indentation problems ever again, being able to split the code easily or the ability to generate YAML programatically, there are several other benefits of using Sharpliner.

Intellisense

One of the best things when using Sharpliner is that you won't have to go the YAML reference every time you're adding a new piece of your pipeline. Having everything strongly typed will make your IDE give you hints all the way!

Imagine you want to install the .NET SDK. For that, Azure Pipelines have the DotNetCoreCLI@2 task. However, this task's specification is quite long since the task does many things:

# .NET Core
# Build, test, package, or publish a dotnet application, or run a custom dotnet command
# https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/build/dotnet-core-cli?view=azure-devops
- task: DotNetCoreCLI@2
  inputs:
    command: 'build' # Options: build, push, pack, publish, restore, run, test, custom
    publishWebProjects: true # Required when command == Publish
    projects: # Optional
    custom: # Required when command == Custom
    arguments: # Optional
    publishTestResults: true # Optional
    testRunTitle: # Optional
    zipAfterPublish: true # Optional
    modifyOutputPath: true # Optional
    feedsToUse: 'select' # Options: select, config
    vstsFeed: # Required when feedsToUse == Select
    feedRestore: # Required when command == restore. projectName/feedName for project-scoped feed. FeedName only for organization-scoped feed.
    includeNuGetOrg: true # Required when feedsToUse == Select
    nugetConfigPath: # Required when feedsToUse == Config
    externalFeedCredentials: # Optional
    noCache: false
    restoreDirectory:
    restoreArguments: # Optional
    verbosityRestore: 'Detailed' # Options: -, quiet, minimal, normal, detailed, diagnostic
    packagesToPush: '$(Build.ArtifactStagingDirectory)/*.nupkg' # Required when command == Push
    nuGetFeedType: 'internal' # Required when command == Push# Options: internal, external
    publishVstsFeed: # Required when command == Push && NuGetFeedType == Internal
    publishPackageMetadata: true # Optional
    publishFeedCredentials: # Required when command == Push && NuGetFeedType == External
    packagesToPack: '**/*.csproj' # Required when command == Pack
    packDirectory: '$(Build.ArtifactStagingDirectory)' # Optional
    nobuild: false # Optional
    includesymbols: false # Optional
    includesource: false # Optional
    versioningScheme: 'off' # Options: off, byPrereleaseNumber, byEnvVar, byBuildNumber
    versionEnvVar: # Required when versioningScheme == byEnvVar
    majorVersion: '1' # Required when versioningScheme == ByPrereleaseNumber
    minorVersion: '0' # Required when versioningScheme == ByPrereleaseNumber
    patchVersion: '0' # Required when versioningScheme == ByPrereleaseNumber
    buildProperties: # Optional
    verbosityPack: 'Detailed' # Options: -, quiet, minimal, normal, detailed, diagnostic
    workingDirectory:

Notice how some of the properties are only valid in a specific combination with other. With Sharpliner, we remove some of this complexity using nice fluent APIs:

DotNet.Install.Sdk(parameters["version"]),

DotNet.Restore.FromFeed("dotnet-7-preview-feed", includeNuGetOrg: false) with
{
    ExternalFeedCredentials = "feeds/dotnet-7",
    NoCache = true,
    RestoreDirectory = ".packages",
},

DotNet.Build("src/MyProject.csproj") with
{
    Timeout = TimeSpan.FromMinutes(20)
},

Useful macros

Some very common pipeline patterns such as comparing the current branch name or detecting pull requests are very cumbersome to do in YAML (long conditions full of complicated ${{ if }} syntax). For many of these, we have handy macros so that you get more readable and shorter code.

For example this YAML

${{ if eq(variables['Build.SourceBranch'], 'refs/heads/production') }}:
    name: rg-suffix
    value: -pr
${{ if ne(variables['Build.SourceBranch'], 'refs/heads/production') }}:
    name: rg-suffix
    value: -prod

can become this C#

If.IsBranch("production")
    .Variable("rg-suffix", "-pr")
.Else
    .Variable("rg-suffix", "-prod")

Re-usable pipeline blocks

Sharpliner lets you re-use code more easily than YAML templates do. Apart from obvious C# code re-use, you can also define sets of C# building blocks and re-use them in your pipelines:

class ProjectBuildSteps : StepLibrary
{
    public override List<Conditioned<Step>> Steps => new()
    {
        DotNet.Install.Sdk("6.0.100"),

        If.IsBranch("main")
            .Step(DotNet.Restore.Projects("src/MyProject.sln")),

        DotNet.Build("src/MyProject.sln"),
    };
}

You can then reference this library in between build steps and it will get expanded into the pipeline's YAML:

...
    new Job("Build")
    {
        Steps =
        {
            Script.Inline("echo 'Hello World'"),

            StepLibrary<ProjectBuildSteps>(),

            Script.Inline("echo 'Goodbye World'"),
        }
    }
...

More about this feature can be found here (DefinitionLibraries.md).

Sourcing scripts from files

When you need to add cmd, PowerShell or bash steps into your pipeline, mainatining these bits inside YAML can be error prone. With Sharpliner you can keep scripts in their own files (.ps1, .sh..) where you get the natural environment you're used to such as syntax highlighting. Sharpliner gives you APIs to load these on build time and include them inline:

Steps =
{
    Bash.FromResourceFile("embedded-script.sh") with
    {
        DisplayName = "Run post-build clean-up",
        Timeout = TimeSpan.FromMinutes(5),
    }
}

Pipeline validation

Your pipeline definition can be validated during publishing and you can uncover issues, such as typos inside dependsOn, you would only find by trying to run the pipeline in CI. This gives you a faster dev loop and greater productivity.

Something missing?

If you find a missing feature / API / property / use case, file an issue in project's repository. We try to be very responsive and for small asks can deliver you a new version very fast.

If you want to start contributing, either you already know about something missing or you can choose from some of the open issues. We will help you review your first change so that you can continue with something advanced!

Another way to start is to try out Sharpliner to define your own, already existing pipeline. This way you can uncover missing features or you can introduce shortcuts for definitions of build tasks or similar that you use frequently. Contributions like these are also very welcome! In these cases, it is worth starting with describing your intent in an issue first.

Project status

Azure DevOps

Azure DevOps pipelines are production ready and have been tested on several pipelines in the wild already. The Sharpliner project itself is self-hosted and is using Sharpliner for its PR and publish pipelines. You can take inspiration there, however it's a bit more complex than needed since we also test some of the features there.

Status:

  • All of the known models from the official documentation have now appropriate C# counterparts.
  • The ${{ if }} statements are working and they can be put (and nested) almost everywhere. There are places where if statemets didn't make sense but in case you need to "if" some property or value, let us know by opening an issue, it should be easy to add.
  • Defining and using templates works!
  • It can happen that we have some of the default values or property names wrong - please let us know (by creating an issue for example)!

GitHub Actions

Currently the development is stalled and very behind compared to Azure DevOps:

  • Model is not complete
  • Actions cannot be currently defined as a whole
  • Documentation is missing

Contributors have already written a large portion of the model but there is still considerable amount of work needed to make GitHub actions work. We welcome all contributors, no contribution is small enough!

Download Details: 
Author: sharpliner
Source Code: https://github.com/sharpliner/sharpliner 
License: MIT
#yaml #csharp #azure #devops #github

What is GEEK

Buddha Community

Sharpliner: Use C# Instead of YAML to Define Your Azure DevOps Pipelines

Generis: Versatile Go Code Generator

Generis

Versatile Go code generator.

Description

Generis is a lightweight code preprocessor adding the following features to the Go language :

  • Generics.
  • Free-form macros.
  • Conditional compilation.
  • HTML templating.
  • Allman style conversion.

Sample

package main;

// -- IMPORTS

import (
    "html"
    "io"
    "log"
    "net/http"
    "net/url"
    "strconv"
    );

// -- DEFINITIONS

#define DebugMode
#as true

// ~~

#define HttpPort
#as 8080

// ~~

#define WriteLine( {{text}} )
#as log.Println( {{text}} )

// ~~

#define local {{variable}} : {{type}};
#as var {{variable}} {{type}};

// ~~

#define DeclareStack( {{type}}, {{name}} )
#as
    // -- TYPES

    type {{name}}Stack struct
    {
        ElementArray []{{type}};
    }

    // -- INQUIRIES

    func ( stack * {{name}}Stack ) IsEmpty(
        ) bool
    {
        return len( stack.ElementArray ) == 0;
    }

    // -- OPERATIONS

    func ( stack * {{name}}Stack ) Push(
        element {{type}}
        )
    {
        stack.ElementArray = append( stack.ElementArray, element );
    }

    // ~~

    func ( stack * {{name}}Stack ) Pop(
        ) {{type}}
    {
        local
            element : {{type}};

        element = stack.ElementArray[ len( stack.ElementArray ) - 1 ];

        stack.ElementArray = stack.ElementArray[ : len( stack.ElementArray ) - 1 ];

        return element;
    }
#end

// ~~

#define DeclareStack( {{type}} )
#as DeclareStack( {{type}}, {{type:PascalCase}} )

// -- TYPES

DeclareStack( string )
DeclareStack( int32 )

// -- FUNCTIONS

func HandleRootPage(
    response_writer http.ResponseWriter,
    request * http.Request
    )
{
    local
        boolean : bool;
    local
        natural : uint;
    local
        integer : int;
    local
        real : float64;
    local
        escaped_html_text,
        escaped_url_text,
        text : string;
    local
        integer_stack : Int32Stack;

    boolean = true;
    natural = 10;
    integer = 20;
    real = 30.0;
    text = "text";
    escaped_url_text = "&escaped text?";
    escaped_html_text = "<escaped text/>";

    integer_stack.Push( 10 );
    integer_stack.Push( 20 );
    integer_stack.Push( 30 );

    #write response_writer
        <!DOCTYPE html>
        <html lang="en">
            <head>
                <meta charset="utf-8">
                <title><%= request.URL.Path %></title>
            </head>
            <body>
                <% if ( boolean ) { %>
                    <%= "URL : " + request.URL.Path %>
                    <br/>
                    <%@ natural %>
                    <%# integer %>
                    <%& real %>
                    <br/>
                    <%~ text %>
                    <%^ escaped_url_text %>
                    <%= escaped_html_text %>
                    <%= "<%% ignored %%>" %>
                    <%% ignored %%>
                <% } %>
                <br/>
                Stack :
                <br/>
                <% for !integer_stack.IsEmpty() { %>
                    <%# integer_stack.Pop() %>
                <% } %>
            </body>
        </html>
    #end
}

// ~~

func main()
{
    http.HandleFunc( "/", HandleRootPage );

    #if DebugMode
        WriteLine( "Listening on http://localhost:HttpPort" );
    #end

    log.Fatal(
        http.ListenAndServe( ":HttpPort", nil )
        );
}

Syntax

#define directive

Constants and generic code can be defined with the following syntax :

#define old code
#as new code

#define old code
#as
    new
    code
#end

#define
    old
    code
#as new code

#define
    old
    code
#as
    new
    code
#end

#define parameter

The #define directive can contain one or several parameters :

{{variable name}} : hierarchical code (with properly matching brackets and parentheses)
{{variable name#}} : statement code (hierarchical code without semicolon)
{{variable name$}} : plain code
{{variable name:boolean expression}} : conditional hierarchical code
{{variable name#:boolean expression}} : conditional statement code
{{variable name$:boolean expression}} : conditional plain code

They can have a boolean expression to require they match specific conditions :

HasText text
HasPrefix prefix
HasSuffix suffix
HasIdentifier text
false
true
!expression
expression && expression
expression || expression
( expression )

The #define directive must not start or end with a parameter.

#as parameter

The #as directive can use the value of the #define parameters :

{{variable name}}
{{variable name:filter function}}
{{variable name:filter function:filter function:...}}

Their value can be changed through one or several filter functions :

LowerCase
UpperCase
MinorCase
MajorCase
SnakeCase
PascalCase
CamelCase
RemoveComments
RemoveBlanks
PackStrings
PackIdentifiers
ReplacePrefix old_prefix new_prefix
ReplaceSuffix old_suffix new_suffix
ReplaceText old_text new_text
ReplaceIdentifier old_identifier new_identifier
AddPrefix prefix
AddSuffix suffix
RemovePrefix prefix
RemoveSuffix suffix
RemoveText text
RemoveIdentifier identifier

#if directive

Conditional code can be defined with the following syntax :

#if boolean expression
    #if boolean expression
        ...
    #else
        ...
    #end
#else
    #if boolean expression
        ...
    #else
        ...
    #end
#end

The boolean expression can use the following operators :

false
true
!expression
expression && expression
expression || expression
( expression )

#write directive

Templated HTML code can be sent to a stream writer using the following syntax :

#write writer expression
    <% code %>
    <%@ natural expression %>
    <%# integer expression %>
    <%& real expression %>
    <%~ text expression %>
    <%= escaped text expression %>
    <%! removed content %>
    <%% ignored tags %%>
#end

Limitations

  • There is no operator precedence in boolean expressions.
  • The --join option requires to end the statements with a semicolon.
  • The #writer directive is only available for the Go language.

Installation

Install the DMD 2 compiler (using the MinGW setup option on Windows).

Build the executable with the following command line :

dmd -m64 generis.d

Command line

generis [options]

Options

--prefix # : set the command prefix
--parse INPUT_FOLDER/ : parse the definitions of the Generis files in the input folder
--process INPUT_FOLDER/ OUTPUT_FOLDER/ : reads the Generis files in the input folder and writes the processed files in the output folder
--trim : trim the HTML templates
--join : join the split statements
--create : create the output folders if needed
--watch : watch the Generis files for modifications
--pause 500 : time to wait before checking the Generis files again
--tabulation 4 : set the tabulation space count
--extension .go : generate files with this extension

Examples

generis --process GS/ GO/

Reads the Generis files in the GS/ folder and writes Go files in the GO/ folder.

generis --process GS/ GO/ --create

Reads the Generis files in the GS/ folder and writes Go files in the GO/ folder, creating the output folders if needed.

generis --process GS/ GO/ --create --watch

Reads the Generis files in the GS/ folder and writes Go files in the GO/ folder, creating the output folders if needed and watching the Generis files for modifications.

generis --process GS/ GO/ --trim --join --create --watch

Reads the Generis files in the GS/ folder and writes Go files in the GO/ folder, trimming the HTML templates, joining the split statements, creating the output folders if needed and watching the Generis files for modifications.

Version

2.0

Author: Senselogic
Source Code: https://github.com/senselogic/GENERIS 
License: View license

#go #golang #code 

Emilie  Okumu

Emilie Okumu

1651546800

Klib | A Standalone and Lightweight C Library

Klib is a standalone and lightweight C library distributed under MIT/X11 license. Most components are independent of external libraries, except the standard C library, and independent of each other. To use a component of this library, you only need to copy a couple of files to your source code tree without worrying about library dependencies.

Klib strives for efficiency and a small memory footprint. Some components, such as khash.h, kbtree.h, ksort.h and kvec.h, are among the most efficient implementations of similar algorithms or data structures in all programming languages, in terms of both speed and memory use.

A new documentation is available here which includes most information in this README file.

Common components

Components for more specific use cases

Methodology

For the implementation of generic containers, klib extensively uses C macros. To use these data structures, we usually need to instantiate methods by expanding a long macro. This makes the source code look unusual or even ugly and adds difficulty to debugging. Unfortunately, for efficient generic programming in C that lacks template, using macros is the only solution. Only with macros, we can write a generic container which, once instantiated, compete with a type-specific container in efficiency. Some generic libraries in C, such as Glib, use the void* type to implement containers. These implementations are usually slower and use more memory than klib (see this benchmark).

To effectively use klib, it is important to understand how it achieves generic programming. We will use the hash table library as an example:

#include "khash.h"
KHASH_MAP_INIT_INT(m32, char)        // instantiate structs and methods
int main() {
    int ret, is_missing;
    khint_t k;
    khash_t(m32) *h = kh_init(m32);  // allocate a hash table
    k = kh_put(m32, h, 5, &ret);     // insert a key to the hash table
    if (!ret) kh_del(m32, h, k);
    kh_value(h, k) = 10;             // set the value
    k = kh_get(m32, h, 10);          // query the hash table
    is_missing = (k == kh_end(h));   // test if the key is present
    k = kh_get(m32, h, 5);
    kh_del(m32, h, k);               // remove a key-value pair
    for (k = kh_begin(h); k != kh_end(h); ++k)  // traverse
        if (kh_exist(h, k))          // test if a bucket contains data
			kh_value(h, k) = 1;
    kh_destroy(m32, h);              // deallocate the hash table
    return 0;
}

In this example, the second line instantiates a hash table with unsigned as the key type and char as the value type. m32 names such a type of hash table. All types and functions associated with this name are macros, which will be explained later. Macro kh_init() initiates a hash table and kh_destroy() frees it. kh_put() inserts a key and returns the iterator (or the position) in the hash table. kh_get() and kh_del() get a key and delete an element, respectively. Macro kh_exist() tests if an iterator (or a position) is filled with data.

An immediate question is this piece of code does not look like a valid C program (e.g. lacking semicolon, assignment to an apparent function call and apparent undefined m32 'variable'). To understand why the code is correct, let's go a bit further into the source code of khash.h, whose skeleton looks like:

#define KHASH_INIT(name, SCOPE, key_t, val_t, is_map, _hashf, _hasheq) \
  typedef struct { \
    int n_buckets, size, n_occupied, upper_bound; \
    unsigned *flags; \
    key_t *keys; \
    val_t *vals; \
  } kh_##name##_t; \
  SCOPE inline kh_##name##_t *init_##name() { \
    return (kh_##name##_t*)calloc(1, sizeof(kh_##name##_t)); \
  } \
  SCOPE inline int get_##name(kh_##name##_t *h, key_t k) \
  ... \
  SCOPE inline void destroy_##name(kh_##name##_t *h) { \
    if (h) { \
      free(h->keys); free(h->flags); free(h->vals); free(h); \
    } \
  }

#define _int_hf(key) (unsigned)(key)
#define _int_heq(a, b) (a == b)
#define khash_t(name) kh_##name##_t
#define kh_value(h, k) ((h)->vals[k])
#define kh_begin(h, k) 0
#define kh_end(h) ((h)->n_buckets)
#define kh_init(name) init_##name()
#define kh_get(name, h, k) get_##name(h, k)
#define kh_destroy(name, h) destroy_##name(h)
...
#define KHASH_MAP_INIT_INT(name, val_t) \
	KHASH_INIT(name, static, unsigned, val_t, is_map, _int_hf, _int_heq)

KHASH_INIT() is a huge macro defining all the structs and methods. When this macro is called, all the code inside it will be inserted by the C preprocess to the place where it is called. If the macro is called multiple times, multiple copies of the code will be inserted. To avoid naming conflict of hash tables with different key-value types, the library uses token concatenation, which is a preprocessor feature whereby we can substitute part of a symbol based on the parameter of the macro. In the end, the C preprocessor will generate the following code and feed it to the compiler (macro kh_exist(h,k) is a little complex and not expanded for simplicity):

typedef struct {
  int n_buckets, size, n_occupied, upper_bound;
  unsigned *flags;
  unsigned *keys;
  char *vals;
} kh_m32_t;
static inline kh_m32_t *init_m32() {
  return (kh_m32_t*)calloc(1, sizeof(kh_m32_t));
}
static inline int get_m32(kh_m32_t *h, unsigned k)
...
static inline void destroy_m32(kh_m32_t *h) {
  if (h) {
    free(h->keys); free(h->flags); free(h->vals); free(h);
  }
}

int main() {
	int ret, is_missing;
	khint_t k;
	kh_m32_t *h = init_m32();
	k = put_m32(h, 5, &ret);
	if (!ret) del_m32(h, k);
	h->vals[k] = 10;
	k = get_m32(h, 10);
	is_missing = (k == h->n_buckets);
	k = get_m32(h, 5);
	del_m32(h, k);
	for (k = 0; k != h->n_buckets; ++k)
		if (kh_exist(h, k)) h->vals[k] = 1;
	destroy_m32(h);
	return 0;
}

This is the C program we know.

From this example, we can see that macros and the C preprocessor plays a key role in klib. Klib is fast partly because the compiler knows the key-value type at the compile time and is able to optimize the code to the same level as type-specific code. A generic library written with void* will not get such performance boost.

Massively inserting code upon instantiation may remind us of C++'s slow compiling speed and huge binary size when STL/boost is in use. Klib is much better in this respect due to its small code size and component independency. Inserting several hundreds lines of code won't make compiling obviously slower.

Resources

  • Library documentation, if present, is available in the header files. Examples can be found in the test/ directory.
  • Obsolete documentation of the hash table library can be found at SourceForge. This README is partly adapted from the old documentation.
  • Blog post describing the hash table library.
  • Blog post on why using void* for generic programming may be inefficient.
  • Blog post on the generic stream buffer.
  • Blog post evaluating the performance of kvec.h.
  • Blog post arguing B-tree may be a better data structure than a binary search tree.
  • Blog post evaluating the performance of khash.h and kbtree.h among many other implementations. An older version of the benchmark is also available.
  • Blog post benchmarking internal sorting algorithms and implementations.
  • Blog post on the k-small algorithm.
  • Blog post on the Hooke-Jeeve's algorithm for nonlinear programming.

Download Details:
 

Author: attractivechaos
Download Link: Download The Source Code
Official Website: https://github.com/attractivechaos/klib 
License: MIT License

#cprogramming #c 

Ruth  Clarke

Ruth Clarke

1649959200

MagickWnd: ImageMagick Program interfaces for C

The MagickWand API is the recommended interface between the C programming language and the ImageMagick image processing libraries. Unlike the MagickCore C API, MagickWand uses only a few opaque types. Accessors are available to set or get important wand properties. A description of the MagickWand public methods are found here:

After you write your MagickWand program, compile it like this:

cc -o wand wand.c `pkg-config --cflags --libs MagickWand`

Set the PKG_CONFIG_PATH environment variable if ImageMagick is not in your default system path:

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

Here is a example program that utilizes the MagickWand API to get you started, wand.c. It reads an image, creates a thumbnail, and writes the result to disk.

#include <stdio.h>
#include <stdlib.h>
#include <MagickWand/MagickWand.h>

int main(int argc,char **argv)
{
#define ThrowWandException(wand) \
{ \
  char \
    *description; \
 \
  ExceptionType \
    severity; \
 \
  description=MagickGetException(wand,&severity); \
  (void) fprintf(stderr,"%s %s %lu %s\n",GetMagickModule(),description); \
  description=(char *) MagickRelinquishMemory(description); \
  exit(-1); \
}

  MagickBooleanType
    status;

  MagickWand
    *magick_wand;

  if (argc != 3)
    {
      (void) fprintf(stdout,"Usage: %s image thumbnail\n",argv[0]);
      exit(0);
    }
  /*
    Read an image.
  */
  MagickWandGenesis();
  magick_wand=NewMagickWand();
  status=MagickReadImage(magick_wand,argv[1]);
  if (status == MagickFalse)
    ThrowWandException(magick_wand);
  /*
    Turn the images into a thumbnail sequence.
  */
  MagickResetIterator(magick_wand);
  while (MagickNextImage(magick_wand) != MagickFalse)
    MagickResizeImage(magick_wand,106,80,LanczosFilter,1.0);
  /*
    Write the image then destroy it.
  */
  status=MagickWriteImages(magick_wand,argv[2],MagickTrue);
  if (status == MagickFalse)
    ThrowWandException(magick_wand);
  magick_wand=DestroyMagickWand(magick_wand);
  MagickWandTerminus();
  return(0);
}

Here is another program that shows one way to get and set image pixels with the MagickWand API, contrast.c. It reads an image, applies sigmoidal non-linearity contrast control, and writes the result to disk.

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <MagickWand/MagickWand.h>

int main(int argc,char **argv)
{
#define QuantumScale  ((MagickRealType) 1.0/(MagickRealType) QuantumRange)
#define SigmoidalContrast(x) \
  (QuantumRange*(1.0/(1+exp(10.0*(0.5-QuantumScale*x)))-0.0066928509)*1.0092503)
#define ThrowWandException(wand) \
{ \
  char \
    *description; \
 \
  ExceptionType \
    severity; \
 \
  description=MagickGetException(wand,&severity); \
  (void) fprintf(stderr,"%s %s %lu %s\n",GetMagickModule(),description); \
  description=(char *) MagickRelinquishMemory(description); \
  exit(-1); \
}

  long
    y;

  MagickBooleanType
    status;

  MagickPixelPacket
    pixel;

  MagickWand
    *contrast_wand,
    *image_wand;

  PixelIterator
    *contrast_iterator,
    *iterator;

  PixelWand
    **contrast_pixels,
    **pixels;

  register long
    x;

  unsigned long
    width;

  if (argc != 3)
    {
      (void) fprintf(stdout,"Usage: %s image sigmoidal-image\n",argv[0]);
      exit(0);
    }
  /*
    Read an image.
  */
  MagickWandGenesis();
  image_wand=NewMagickWand();
  status=MagickReadImage(image_wand,argv[1]);
  if (status == MagickFalse)
    ThrowWandException(image_wand);
  contrast_wand=CloneMagickWand(image_wand);
  /*
    Sigmoidal non-linearity contrast control.
  */
  iterator=NewPixelIterator(image_wand);
  contrast_iterator=NewPixelIterator(contrast_wand);
  if ((iterator == (PixelIterator *) NULL) ||
      (contrast_iterator == (PixelIterator *) NULL))
    ThrowWandException(image_wand);
  for (y=0; y < (long) MagickGetImageHeight(image_wand); y++)
  {
    pixels=PixelGetNextIteratorRow(iterator,&width);
    contrast_pixels=PixelGetNextIteratorRow(contrast_iterator,&width);
    if ((pixels == (PixelWand **) NULL) ||
        (contrast_pixels == (PixelWand **) NULL))
      break;
    for (x=0; x < (long) width; x++)
    {
      PixelGetMagickColor(pixels[x],&pixel);
      pixel.red=SigmoidalContrast(pixel.red);
      pixel.green=SigmoidalContrast(pixel.green);
      pixel.blue=SigmoidalContrast(pixel.blue);
      pixel.index=SigmoidalContrast(pixel.index);
      PixelSetMagickColor(contrast_pixels[x],&pixel);
    }
    (void) PixelSyncIterator(contrast_iterator);
  }
  if (y < (long) MagickGetImageHeight(image_wand))
    ThrowWandException(image_wand);
  contrast_iterator=DestroyPixelIterator(contrast_iterator);
  iterator=DestroyPixelIterator(iterator);
  image_wand=DestroyMagickWand(image_wand);
  /*
    Write the image then destroy it.
  */
  status=MagickWriteImages(contrast_wand,argv[2],MagickTrue);
  if (status == MagickFalse)
    ThrowWandException(image_wand);
  contrast_wand=DestroyMagickWand(contrast_wand);
  MagickWandTerminus();
  return(0);
}

Now lets perform the same contrast enhancement while taking advantage of our dual or quad-core processing system by running the algorithm in parallel utilizing wand views. The sigmoidal-contrast.c module reads an image, applies sigmoidal non-linearity contrast control, and writes the result to disk just like the previous contrast enhancement program, but now it does its work in parallel (assumes ImageMagick is built with OpenMP support).

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <MagickWand/MagickWand.h>

static MagickBooleanType SigmoidalContrast(WandView *pixel_view,
  const ssize_t y,const int id,void *context)
{
#define QuantumScale  ((MagickRealType) 1.0/(MagickRealType) QuantumRange)
#define SigmoidalContrast(x) \
  (QuantumRange*(1.0/(1+exp(10.0*(0.5-QuantumScale*x)))-0.0066928509)*1.0092503)

  RectangleInfo
    extent;

  MagickPixelPacket
    pixel;

  PixelWand
    **pixels;

  register long
    x;

  extent=GetWandViewExtent(contrast_view);
  pixels=GetWandViewPixels(contrast_view);
  for (x=0; x < (long) (extent.width-extent.height); x++)
  {
    PixelGetMagickColor(pixels[x],&pixel);
    pixel.red=SigmoidalContrast(pixel.red);
    pixel.green=SigmoidalContrast(pixel.green);
    pixel.blue=SigmoidalContrast(pixel.blue);
    pixel.index=SigmoidalContrast(pixel.index);
    PixelSetMagickColor(contrast_pixels[x],&pixel);
  }
  return(MagickTrue);
}

int main(int argc,char **argv)
{
#define ThrowViewException(view) \
{ \
  description=GetWandViewException(view,&severity); \
  (void) fprintf(stderr,"%s %s %lu %s\n",GetMagickModule(),description); \
  description=(char *) MagickRelinquishMemory(description); \
  exit(-1); \
}
#define ThrowWandException(wand) \
{ \
  description=MagickGetException(wand,&severity); \
  (void) fprintf(stderr,"%s %s %lu %s\n",GetMagickModule(),description); \
  description=(char *) MagickRelinquishMemory(description); \
  exit(-1); \
}

  char
    *description;

  ExceptionType
    severity;

  MagickBooleanType
    status;

  MagickPixelPacket
    pixel;

  MagickWand
    *contrast_wand;

  WandView
    *contrast_view;

  if (argc != 3)
    {
      (void) fprintf(stdout,"Usage: %s image sigmoidal-image\n",argv[0]);
      exit(0);
    }
  /*
    Read an image.
  */
  MagickWandGenesis();
  contrast_wand=NewMagickWand();
  status=MagickReadImage(contrast_wand,argv[1]);
  if (status == MagickFalse)
    ThrowWandException(contrast_wand);
  /*
    Sigmoidal non-linearity contrast control.
  */
  contrast_view=NewWandView(contrast_wand);
  if (contrast_view == (WandView *) NULL)
    ThrowWandException(contrast_wand);
  status=UpdateWandViewIterator(contrast_view,SigmoidalContrast,(void *) NULL);
  if (status == MagickFalse)
    ThrowWandException(contrast_wand);
  contrast_view=DestroyWandView(contrast_view);
  /*
    Write the image then destroy it.
  */
  status=MagickWriteImages(contrast_wand,argv[2],MagickTrue);
  if (status == MagickFalse)
    ThrowWandException(contrast_wand);
  contrast_wand=DestroyMagickWand(contrast_wand);
  MagickWandTerminus();
  return(0);
}

MagickWand Examples in C illustrates how to use the ImageMagick MagickWand API. Each example is presented as a C function, complete with headers, so that it can be copied to a file and then included in your own C project.


Source code: https://imagemagick.org/script/magick-wand.php

#cpluplus #c 
 

Abdullah  Kozey

Abdullah Kozey

1649329200

Minimp3: Minimalistic MP3 Decoder Single Header Library

minimp3

Minimalistic, single-header library for decoding MP3. minimp3 is designed to be small, fast (with SSE and NEON support), and accurate (ISO conformant). You can find a rough benchmark below, measured using perf on an i7-6700K, IO included, no CPU heat to address speedstep:

VectorHzSamplesSecClockticksClockticks per secondPSNRMax diff
compl.bit480002488325.184143066842.759M124.221
he_32khz.bit320001728005.484261581.560M139.671
he_44khz.bit4410047232010.710212963001.988M144.041
he_48khz.bit480001728003.684538462.348M139.671
hecommon.bit44100691201.56731697152.022M133.931
he_free.bit441001566723.55257984181.632M137.481
he_mode.bit441002626565.95598823141.659M118.001
si.bit441001359363.08271705202.326M120.301
si_block.bit44100737281.67142331362.533M125.181
si_huff.bit44100864001.95947853222.442M107.981
sin1k0db.bit4410072576016.457248429771.509M111.031

Conformance test passed on all vectors (PSNR > 96db).

Comparison with keyj's minimp3

Comparison by features:

Keyj minimp3Current
Fixed pointFloating point
source: 84kb70kb
binary: 34kb (20kb compressed)30kb (20kb)
no vector optsSSE/NEON intrinsics
no free formatfree format support

Below, you can find the benchmark and conformance test for keyj's minimp3:

VectorHzSamplesSecClockticksClockticks per secondPSNRMax diff
compl.bit480002488325.184318493736.143M71.5041
he_32khz.bit320001728005.4263023194.870M71.6324
he_44khz.bit4410047232010.710416288613.886M71.6324
he_48khz.bit480001728003.6258995277.194M71.6324
hecommon.bit44100691201.5672043777913.039M71.5825
he_free.bit4410000----
he_mode.bit441002626565.955309889845.203M71.7827
si.bit441001359363.082240962237.817M72.3536
si_block.bit44100737281.6712072201712.394M71.8426
si_huff.bit44100864001.9592112137610.780M27.8065535
sin1k0db.bit4410073036816.561555696363.355M0.1558814

Keyj minimp3 conformance test fails on all vectors (PSNR < 96db), and free format is unsupported. This caused some problems when it was used here, and was the main motivation for this work.

Usage

First, we need to initialize the decoder structure:

//#define MINIMP3_ONLY_MP3
//#define MINIMP3_ONLY_SIMD
//#define MINIMP3_NO_SIMD
//#define MINIMP3_NONSTANDARD_BUT_LOGICAL
//#define MINIMP3_FLOAT_OUTPUT
#define MINIMP3_IMPLEMENTATION
#include "minimp3.h"
...
    static mp3dec_t mp3d;
    mp3dec_init(&mp3d);

Note that you must define MINIMP3_IMPLEMENTATION in exactly one source file. You can #include minimp3.h in as many files as you like. Also you can use MINIMP3_ONLY_MP3 define to strip MP1/MP2 decoding code. MINIMP3_ONLY_SIMD define controls generic (non SSE/NEON) code generation (always enabled on x64/arm64 targets). In case you do not want any platform-specific SIMD optimizations, you can define MINIMP3_NO_SIMD. MINIMP3_NONSTANDARD_BUT_LOGICAL define saves some code bytes, and enforces non-standard but logical behaviour of mono-stereo transition (rare case). MINIMP3_FLOAT_OUTPUT makes mp3dec_decode_frame() output to be float instead of short and additional function mp3dec_f32_to_s16 will be available for float->short conversion if needed.

Then. we decode the input stream frame-by-frame:

    /*typedef struct
    {
        int frame_bytes;
        int channels;
        int hz;
        int layer;
        int bitrate_kbps;
    } mp3dec_frame_info_t;*/
    mp3dec_frame_info_t info;
    short pcm[MINIMP3_MAX_SAMPLES_PER_FRAME];
    /*unsigned char *input_buf; - input byte stream*/
    samples = mp3dec_decode_frame(&mp3d, input_buf, buf_size, pcm, &info);

The mp3dec_decode_frame() function decodes one full MP3 frame from the input buffer, which must be large enough to hold one full frame.

The decoder will analyze the input buffer to properly sync with the MP3 stream, and will skip ID3 data, as well as any data which is not valid. Short buffers may cause false sync and can produce 'squealing' artefacts. The bigger the size of the input buffer, the more reliable the sync procedure. We recommend having as many as 10 consecutive MP3 frames (~16KB) in the input buffer at a time.

At end of stream just pass rest of the buffer, sync procedure should work even with just 1 frame in stream (except for free format and garbage at the end can mess things up, so id3v1 and ape tags must be removed first).

For free format there minimum 3 frames needed to do proper sync: 2 frames to detect frame length and 1 next frame to check detect is good.

The size of the consumed MP3 data is returned in the mp3dec_frame_info_t field of the frame_bytes struct; you must remove the data corresponding to the frame_bytes field from the input buffer before the next decoder invocation.

The decoding function returns the number of decoded samples. The following cases are possible:

  • 0: No MP3 data was found in the input buffer
  • 384: Layer 1
  • 576: MPEG 2 Layer 3
  • 1152: Otherwise

The following is a description of the possible combinations of the number of samples and frame_bytes field values:

  • More than 0 samples and frame_bytes > 0: Succesful decode
  • 0 samples and frame_bytes > 0: The decoder skipped ID3 or invalid data
  • 0 samples and frame_bytes == 0: Insufficient data

If frame_bytes == 0, the other fields may be uninitialized or unchanged; if frame_bytes != 0, the other fields are available. The application may call mp3dec_init() when changing decode position, but this is not necessary.

As a special case, the decoder supports already split MP3 streams (for example, after doing an MP4 demux). In this case, the input buffer must contain exactly one non-free-format frame.

Seeking

You can seek to any byte in the stream and call mp3dec_decode_frame; this will work in almost all cases, but is not completely guaranteed. Probablility of sync procedure failure lowers when MAX_FRAME_SYNC_MATCHES value grows. Default MAX_FRAME_SYNC_MATCHES=10 and probablility of sync failure should be very low. If granule data is accidentally detected as a valid MP3 header, short audio artefacting is possible.

High-level mp3dec_ex_seek function supports precise seek to sample (MP3D_SEEK_TO_SAMPLE) using index and binary search.

Track length detect

If the file is known to be cbr, then all frames have equal size and lack ID3 tags, which allows us to decode the first frame and calculate all frame positions as frame_bytes * N. However, because of padding, frames can differ in size even in this case.

In general case whole stream scan is needed to calculate it's length. Scan can be omitted if vbr tag is present (added by encoders like lame and ffmpeg), which contains length info. High-level functions automatically use the vbr tag if present.

High-level API

If you need only decode file/buffer or use precise seek, you can use optional high-level API. Just #include minimp3_ex.h instead and use following additional functions:

#define MP3D_SEEK_TO_BYTE   0
#define MP3D_SEEK_TO_SAMPLE 1

#define MINIMP3_PREDECODE_FRAMES 2 /* frames to pre-decode and skip after seek (to fill internal structures) */
/*#define MINIMP3_SEEK_IDX_LINEAR_SEARCH*/ /* define to use linear index search instead of binary search on seek */
#define MINIMP3_IO_SIZE (128*1024) /* io buffer size for streaming functions, must be greater than MINIMP3_BUF_SIZE */
#define MINIMP3_BUF_SIZE (16*1024) /* buffer which can hold minimum 10 consecutive mp3 frames (~16KB) worst case */
#define MINIMP3_ENABLE_RING 0      /* enable hardware magic ring buffer if available, to make less input buffer memmove(s) in callback IO mode */

#define MP3D_E_MEMORY  -1
#define MP3D_E_IOERROR -2

typedef struct
{
    mp3d_sample_t *buffer;
    size_t samples; /* channels included, byte size = samples*sizeof(mp3d_sample_t) */
    int channels, hz, layer, avg_bitrate_kbps;
} mp3dec_file_info_t;

typedef size_t (*MP3D_READ_CB)(void *buf, size_t size, void *user_data);
typedef int (*MP3D_SEEK_CB)(uint64_t position, void *user_data);

typedef struct
{
    MP3D_READ_CB read;
    void *read_data;
    MP3D_SEEK_CB seek;
    void *seek_data;
} mp3dec_io_t;

typedef struct
{
    uint64_t samples;
    mp3dec_frame_info_t info;
    int last_error;
    ...
} mp3dec_ex_t;

typedef int (*MP3D_ITERATE_CB)(void *user_data, const uint8_t *frame, int frame_size, int free_format_bytes, size_t buf_size, uint64_t offset, mp3dec_frame_info_t *info);
typedef int (*MP3D_PROGRESS_CB)(void *user_data, size_t file_size, uint64_t offset, mp3dec_frame_info_t *info);

/* decode whole buffer block */
int mp3dec_load_buf(mp3dec_t *dec, const uint8_t *buf, size_t buf_size, mp3dec_file_info_t *info, MP3D_PROGRESS_CB progress_cb, void *user_data);
int mp3dec_load_cb(mp3dec_t *dec, mp3dec_io_t *io, uint8_t *buf, size_t buf_size, mp3dec_file_info_t *info, MP3D_PROGRESS_CB progress_cb, void *user_data);
/* iterate through frames */
int mp3dec_iterate_buf(const uint8_t *buf, size_t buf_size, MP3D_ITERATE_CB callback, void *user_data);
int mp3dec_iterate_cb(mp3dec_io_t *io, uint8_t *buf, size_t buf_size, MP3D_ITERATE_CB callback, void *user_data);
/* streaming decoder with seeking capability */
int mp3dec_ex_open_buf(mp3dec_ex_t *dec, const uint8_t *buf, size_t buf_size, int seek_method);
int mp3dec_ex_open_cb(mp3dec_ex_t *dec, mp3dec_io_t *io, int seek_method);
void mp3dec_ex_close(mp3dec_ex_t *dec);
int mp3dec_ex_seek(mp3dec_ex_t *dec, uint64_t position);
size_t mp3dec_ex_read(mp3dec_ex_t *dec, mp3d_sample_t *buf, size_t samples);
#ifndef MINIMP3_NO_STDIO
/* stdio versions of file load, iterate and stream */
int mp3dec_load(mp3dec_t *dec, const char *file_name, mp3dec_file_info_t *info, MP3D_PROGRESS_CB progress_cb, void *user_data);
int mp3dec_iterate(const char *file_name, MP3D_ITERATE_CB callback, void *user_data);
int mp3dec_ex_open(mp3dec_ex_t *dec, const char *file_name, int seek_method);
#ifdef _WIN32
int mp3dec_load_w(mp3dec_t *dec, const wchar_t *file_name, mp3dec_file_info_t *info, MP3D_PROGRESS_CB progress_cb, void *user_data);
int mp3dec_iterate_w(const wchar_t *file_name, MP3D_ITERATE_CB callback, void *user_data);
int mp3dec_ex_open_w(mp3dec_ex_t *dec, const wchar_t *file_name, int seek_method);
#endif
#endif

Use MINIMP3_NO_STDIO define to exclude STDIO functions. MINIMP3_ALLOW_MONO_STEREO_TRANSITION allows mixing mono and stereo in same file. In that case mp3dec_frame_info_t->channels = 0 is reported on such files and correct channels number passed to progress_cb callback for each frame in mp3dec_frame_info_t structure. MP3D_PROGRESS_CB is optional and can be NULL, example of file decoding:

    mp3dec_t mp3d;
    mp3dec_file_info_t info;
    if (mp3dec_load(&mp3d, input_file_name, &info, NULL, NULL))
    {
        /* error */
    }
    /* mp3dec_file_info_t contains decoded samples and info,
       use free(info.buffer) to deallocate samples */

Example of file decoding with seek capability:

    mp3dec_ex_t dec;
    if (mp3dec_ex_open(&dec, input_file_name, MP3D_SEEK_TO_SAMPLE))
    {
        /* error */
    }
    /* dec.samples, dec.info.hz, dec.info.layer, dec.info.channels should be filled */
    if (mp3dec_ex_seek(&dec, position))
    {
        /* error */
    }
    mp3d_sample_t *buffer = malloc(dec.samples*sizeof(mp3d_sample_t));
    size_t readed = mp3dec_ex_read(&dec, buffer, dec.samples);
    if (readed != dec.samples) /* normal eof or error condition */
    {
        if (dec.last_error)
        {
            /* error */
        }
    }

Bindings

Interesting links

Download Details: 
Author: lieff
Source Code: https://github.com/lieff/minimp3 
License: CC0-1.0 License
#programming #developer #cplusplus #cpp

Tamale  Moses

Tamale Moses

1651075200

A C++ Header-only Library Of Statistical Distribution Functions.

StatsLib 

StatsLib is a templated C++ library of statistical distribution functions, featuring unique compile-time computing capabilities and seamless integration with several popular linear algebra libraries.

Features:

  • A header-only library of probability density functions, cumulative distribution functions, quantile functions, and random sampling methods.
  • Functions are written in C++11 constexpr format, enabling the library to operate as both a compile-time and run-time computation engine.
  • Designed with a simple R-like syntax.
  • Optional vector-matrix functionality with wrappers to support:
  • Matrix-based operations are parallelizable with OpenMP.
  • Released under a permissive, non-GPL license.

Contents:

Distributions

Functions to compute the cdf, pdf, quantile, as well as random sampling methods, are available for the following distributions:

  • Bernoulli
  • Beta
  • Binomial
  • Cauchy
  • Chi-squared
  • Exponential
  • F
  • Gamma
  • Inverse-Gamma
  • Laplace
  • Logistic
  • Log-Normal
  • Normal (Gaussian)
  • Poisson
  • Student's t
  • Uniform
  • Weibull

In addition, pdf and random sampling functions are available for several multivariate distributions:

  • inverse-Wishart
  • Multivariate Normal
  • Wishart

Installation and Dependencies

StatsLib is a header-only library. Simply add the header files to your project using

#include "stats.hpp"

The only dependency is the latest version of GCEM and a C++11 compatible compiler.

Documentation

Full documentation is available online:

Documentation Status

A PDF version of the documentation is available here.

Jupyter Notebook

You can test the library online using an interactive Jupyter notebook:

Binder

Options

The following options should be declared before including the StatsLib header files.

  • For inline-only functionality (i.e., no constexpr specifiers):
#define STATS_GO_INLINE
  • OpenMP functionality is enabled by default if the _OPENMP macro is detected (e.g., by invoking -fopenmp with GCC or Clang). To explicitly enable OpenMP features use:
#define STATS_USE_OPENMP
  • To disable OpenMP functionality:
#define STATS_DONT_USE_OPENMP
  • To use StatsLib with Armadillo, Blaze or Eigen:
#define STATS_ENABLE_ARMA_WRAPPERS
#define STATS_ENABLE_BLAZE_WRAPPERS
#define STATS_ENABLE_EIGEN_WRAPPERS
  • To enable wrappers for std::vector:
#define STATS_ENABLE_STDVEC_WRAPPERS

Syntax and Examples

Functions are called using an R-like syntax. Some general rules:

  • density functions: stats::d*. For example, the Normal (Gaussian) density is called using
stats::dnorm(<value>,<mean parameter>,<standard deviation>);
  • cumulative distribution functions: stats::p*. For example, the Gamma CDF is called using
stats::pgamma(<value>,<shape parameter>,<scale parameter>);
  • quantile functions: stats::q*. For example, the Beta quantile is called using
stats::qbeta(<value>,<a parameter>,<b parameter>);
  • random sampling: stats::r*. For example, to generate a single draw from the Logistic distribution:
stats::rlogis(<location parameter>,<scale parameter>,<seed value or random number engine>);


 

All of these functions have matrix-based equivalents using Armadillo, Blaze, and Eigen dense matrices.

  • The pdf, cdf, and quantile functions can take matrix-valued arguments. For example,
// Using Armadillo:
arma::mat norm_pdf_vals = stats::dnorm(arma::ones(10,20),1.0,2.0);
  • The randomization functions (r*) can output random matrices of arbitrary size. For example, For example, the following code will generate a 100-by-50 matrix of iid draws from a Gamma(3,2) distribution:
// Armadillo:
arma::mat gamma_rvs = stats::rgamma<arma::mat>(100,50,3.0,2.0);
// Blaze:
blaze::DynamicMatrix<double> gamma_rvs = stats::rgamma<blaze::DynamicMatrix<double>>(100,50,3.0,2.0);
// Eigen:
Eigen::MatrixXd gamma_rvs = stats::rgamma<Eigen::MatrixXd>(100,50,3.0,2.0);
  • All matrix-based operations are parallelizable with OpenMP. For GCC and Clang compilers, simply include the -fopenmp option during compilation.

Seeding Values

Random number seeding is available in two forms: seed values and random number engines.

  • Seed values are passed as unsigned integers. For example, to generate a draw from a normal distribution N(1,2) with seed value 1776:
stats::rnorm(1,2,1776);
  • Random engines in StatsLib use the 64-bit Mersenne-Twister generator (std::mt19937_64) and are passed by reference. Example:
std::mt19937_64 engine(1776);
stats::rnorm(1,2,engine);

Examples

More examples with code:

// evaluate the normal PDF at x = 1, mu = 0, sigma = 1
double dval_1 = stats::dnorm(1.0,0.0,1.0);
 
// evaluate the normal PDF at x = 1, mu = 0, sigma = 1, and return the log value
double dval_2 = stats::dnorm(1.0,0.0,1.0,true);
 
// evaluate the normal CDF at x = 1, mu = 0, sigma = 1
double pval = stats::pnorm(1.0,0.0,1.0);
 
// evaluate the Laplacian quantile at p = 0.1, mu = 0, sigma = 1
double qval = stats::qlaplace(0.1,0.0,1.0);

// draw from a t-distribution dof = 30
double rval = stats::rt(30);

// matrix output
arma::mat beta_rvs = stats::rbeta<arma::mat>(100,100,3.0,2.0);
// matrix input
arma::mat beta_cdf_vals = stats::pbeta(beta_rvs,3.0,2.0);

Compile-time Computing Capabilities

StatsLib is designed to operate equally well as a compile-time computation engine. Compile-time computation allows the compiler to replace function calls (e.g., dnorm(0,0,1)) with static values in the source code. That is, functions are evaluated during the compilation process, rather than at run-time. This capability is made possible due to the templated constexpr design of the library and can be verified by inspecting the assembly code generated by the compiler.

The compile-time features are enabled using the constexpr specifier. The example below computes the pdf, cdf, and quantile function of the Laplace distribution.

#include "stats.hpp"

int main()
{
    
    constexpr double dens_1  = stats::dlaplace(1.0,1.0,2.0); // answer = 0.25
    constexpr double prob_1  = stats::plaplace(1.0,1.0,2.0); // answer = 0.5
    constexpr double quant_1 = stats::qlaplace(0.1,1.0,2.0); // answer = -2.218875...

    return 0;
}

Assembly code generated by Clang without any optimization:

LCPI0_0:
	.quad	-4611193153885729483    ## double -2.2188758248682015
LCPI0_1:
	.quad	4602678819172646912     ## double 0.5
LCPI0_2:
	.quad	4598175219545276417     ## double 0.25000000000000006
	.section	__TEXT,__text,regular,pure_instructions
	.globl	_main
	.p2align	4, 0x90
_main:                                  ## @main
	push	rbp
	mov	rbp, rsp
	xor	eax, eax
	movsd	xmm0, qword ptr [rip + LCPI0_0] ## xmm0 = mem[0],zero
	movsd	xmm1, qword ptr [rip + LCPI0_1] ## xmm1 = mem[0],zero
	movsd	xmm2, qword ptr [rip + LCPI0_2] ## xmm2 = mem[0],zero
	mov	dword ptr [rbp - 4], 0
	movsd	qword ptr [rbp - 16], xmm2
	movsd	qword ptr [rbp - 24], xmm1
	movsd	qword ptr [rbp - 32], xmm0
	pop	rbp
	ret

Author

Keith O'Hara


Author: kthohr
Source Code: https://github.com/kthohr/stats
License: Apache-2.0 License

#cpluplus #c