FastGaussQuadrature.jl: Julia Package for Gaussian Quadrature

FastGaussQuadrature.jl   

A Julia package to compute n-point Gauss quadrature nodes and weights to 16-digit accuracy and in O(n) time. So far the package includes gausschebyshev(), gausslegendre(), gaussjacobi(), gaussradau(), gausslobatto(), gausslaguerre(), and gausshermite(). This package is heavily influenced by Chebfun.

An introduction to Gauss quadrature can be found here. For a quirky account on the history of computing Gauss-Legendre quadrature, see [6].

Our Aims

• The fastest Julia code for Gauss quadrature nodes and weights (without tabulation).
• Change the perception that Gauss quadrature rules are expensive to compute.

Example usage

julia> @time nodes, weights = gausslegendre( 100000 );
  0.002192 seconds (10 allocations: 2.289 MiB)

# integrates f(x) = x^2 from -1 to 1
julia> @time dot( weights, nodes.^2 )
  0.000184 seconds (7 allocations: 781.422 KiB)
0.6666666666666665

Download Details:

Author: JuliaApproximation
Source Code: https://github.com/JuliaApproximation/FastGaussQuadrature.jl 
License: MIT license

#julia 

What is GEEK

Buddha Community

FastGaussQuadrature.jl: Julia Package for Gaussian Quadrature

FastGaussQuadrature.jl   

A Julia package to compute n-point Gauss quadrature nodes and weights to 16-digit accuracy and in O(n) time. So far the package includes gausschebyshev(), gausslegendre(), gaussjacobi(), gaussradau(), gausslobatto(), gausslaguerre(), and gausshermite(). This package is heavily influenced by Chebfun.

An introduction to Gauss quadrature can be found here. For a quirky account on the history of computing Gauss-Legendre quadrature, see [6].

Our Aims

• The fastest Julia code for Gauss quadrature nodes and weights (without tabulation).
• Change the perception that Gauss quadrature rules are expensive to compute.

Example usage

julia> @time nodes, weights = gausslegendre( 100000 );
  0.002192 seconds (10 allocations: 2.289 MiB)

# integrates f(x) = x^2 from -1 to 1
julia> @time dot( weights, nodes.^2 )
  0.000184 seconds (7 allocations: 781.422 KiB)
0.6666666666666665

Download Details:

Author: JuliaApproximation
Source Code: https://github.com/JuliaApproximation/FastGaussQuadrature.jl 
License: MIT license

#julia 

JuliaWebAPI.jl: Julia Package for Deploying APIs

JuliaWebAPI.jl

Facilitates wrapping Julia functions into a remote callable API via message queues (e.g. ZMQ, RabbitMQ) and HTTP.

It can plug in to a different messaging infrastructure through an implementation of transport (AbstractTransport) and message format (AbstractMsgFormat). Multiple instances of the front (HTTP API) and back (Julia methods) end can help scale an application. Bundled with the package are implementations for:

• ZMQTransport: use ZMQ for transport
• InProcTransport: use Julia Channel for transport within the same process
• JSONMsgFormat: JSON as message format
• SerializedMsgFormat: Julia serialization as message format
• DictMsgFormat: Julia Dict as message format, for use within the same process

Combined with a HTTP/Messaging frontend (like JuliaBox), it helps deploy Julia packages and code snippets as hosted, auto-scaling HTTP APIs.

Some amount of basic request filtering and pre-processing is possible by registering a pre-processor with the HTTP frontend. The pre-processor is run at the HTTP server side, where it has access to the complete request. It can examine headers and data and take decision whether to allow calling the service or respond directly and immediately. It can also rewrite the request before passing it on to the service.

A pre-processor can be used to implement features like authentication, request rewriting and such. See example below.

Example Usage

Create a file srvr.jl with the following code

# Load required packages
using JuliaWebAPI

# Define functions testfn1 and testfn2 that we shall expose
function testfn1(arg1, arg2; narg1="1", narg2="2")
    return (parse(Int, arg1) * parse(Int, narg1)) + (parse(Int, arg2) * parse(Int, narg2))
end

testfn2(arg1, arg2; narg1="1", narg2="2") = testfn1(arg1, arg2; narg1=narg1, narg2=narg2)

# Expose testfn1 and testfn2 via a ZMQ listener
process(
    JuliaWebAPI.create_responder([
        (testfn1, true),
        (testfn2, false)
    ], "tcp://127.0.0.1:9999", true, "")
)

Start the server process in the background. This process will run the ZMQ listener.

julia srvr.jl &

Then, on a Julia REPL, run the following code

using JuliaWebAPI   #Load package

#Create the ZMQ client that talks to the ZMQ listener above
const apiclnt = APIInvoker("tcp://127.0.0.1:9999");

#Start the HTTP server in current process (Ctrl+C to interrupt)
run_http(apiclnt, 8888)

Then, on your browser, navigate to http://localhost:8888/testfn1/4/5?narg1=6&narg2=4

This will return the following JSON response to your browser, which is the result of running the testfn1 function defined above: {"data"=>44,"code"=>0}

Example of an authentication filter implemented using a pre-processor:

function auth_preproc(req::HTTP.Request)
    if !validate(req)
        return HTTP.Response(401)
    end
    return nothing
end
run_http(apiclnt, 8888, auth_preproc)

Download Details:

Author: JuliaWeb
Source Code: https://github.com/JuliaWeb/JuliaWebAPI.jl 
License: View license

#julia #web #package #api 

Diversity.jl: Julia Package for Diversity Measurement

Diversity

A package for measuring and partitioning diversity

Summary

Diversity is a Julia package that provides functionality for measuring alpha, beta and gamma diversity of metacommunities (e.g. ecosystems) and their constituent subcommunities. It uses the diversity measures described in the arXiv paper arXiv:1404.6520 (q-bio.QM), How to partition diversity. It also provides a series of other older diversity measures through sub-modules. Currently these are all ecological diversity measures, but this will be expanded through interfacing to EcoJulia and BioJulia.

This package is in beta now, but is cross-validated against our R package boydorr/rdiversity, which is developed independently, so please raise an issue if you find any problems. We now use a DataFrame as the common output format for all of the diversity calculations to provide consistency with our R package rdiversity. The code is not optimised for speed at the moment due to the substantial changes that have happened to it under the hood, and the Phylogenetics submodule is also recently revised, and may need further improvements.

Installation

The package is registered in the General registry on v1.x and so can be installed with add. For example on Julia v1.6:

(@v1.6) pkg> add Diversity
    Resolving package versions...
    Updating `~/.julia/environments/v1.6/Project.toml`
  [d3d5718d] + Diversity v0.5.5
    Updating `~/.julia/environments/v1.6/Manifest.toml`
  [d3d5718d] + Diversity v0.5.5
  
(@v1.6) pkg>

Project Status

The package is confirmed to work against the current LTS Julia v1.4 release and the latest release on Linux, macOS, and Windows. It is also tested against nightly.

Contributing and Questions

Contributions are very welcome, as are feature requests and suggestions. Please open an issue if you encounter any problems or would just like to ask a question.

Usage

Diversity Measures

The main package provides basic numbers-equivalent diversity measures (described in Hill, 1973), similarity-sensitive diversity measures (generalised from Hill, and described in Leinster and Cobbold, 2012), and related alpha, beta and gamma diversity measures at the level of the metacommunity and its component subcommunities (generalised in turn from Leinster and Cobbold, and described in arXiv:1404.6520 (q-bio.QM)). The diversity functions exist both with unicode names (e.g. ᾱ()), which are not automatically exported as we feel they are too short and with matching ascii names (e.g. NormalisedAlpha()), which are. We also provide a general function for extract any diversity measure for a series of subcommunity relative abundances.

Getting started

Before calculating diversity a Metacommunity object must be created. This object contains all the information needed to calculate diversity.

# Load the package into Julia
using Diversity

# Example population
pop = [1 1 0; 2 0 0; 3 1 4]
pop = pop / sum(pop)

# Create Metacommunity object
meta = Metacommunity(pop)

Calculating diversity

First we need to calculate the low-level diversity component seperately, by passing a metacommunity object to the appropriate function; RawAlpha(), NormalisedAlpha(), RawBeta(), NormalisedBeta(), RawRho(), NormalisedRho(), or Gamma().

# First, calculate the normalised alpha component
component = NormalisedAlpha(meta)

Afterwhich, subdiv() or metadiv() are used to calculate subcommunity or metacommunity diversity, respectively (since both subcommunity and metacommunity diversity measures are transformations of the same low-level components, this is computationally more efficient).

# Then, calculate species richness of the subcommunities
subdiv(component, 0)

# or the average (alpha) species richness across the whole population
metadiv(component, 0)

# We can also generate a diversity profile by calculating multiple q-values simultaneously
df = subdiv(component, 0:30)

In some instances, it may be useful to calculate all subcommunity (or metacommunity) measures. In which case, a Metacommunity object may be passed directly to subdiv() or metadiv():

# To calculate all subcommunity diversity measures
subdiv(meta, 0:2)

# To calculate all metacommunity diversity measures
metadiv(meta, 0:2)

Alternatively, if computational efficiency is not an issue, a single measure of diversity may be calculated directly by calling a wrapper function:

norm_sub_alpha(meta, 0:2)

A complete list of these functions is shown below:

• raw_sub_alpha() : per-subcommunity estimate of naive-community metacommunity diversity
• norm_sub_alpha() : similarity-sensitive diversity of each subcommunity in isolation
• raw_sub_rho() : redundancy of individual subcommunities
• norm_sub_rho() : representativeness of individual subcommunities
• raw_sub_beta() : distinctiveness of individual subcommunities
• norm_sub_beta() : per-subcommunity estimate of effective number of distinct subcommunities
• sub_gamma() : contribution per individual in a subcommunity toward metacommunity diversity
• raw_meta_alpha() : naive-community metacommunity diversity
• norm_meta_alpha() : average similarity-sensitive diversity of subcommunities
• raw_meta_rho() : average redundancy of subcommunities
• norm_meta_rho() : average representativeness of subcommunities
• raw_meta_beta() : average distinctiveness of subcommunities
• norm_meta_beta() : effective number of distinct subcommunities
• meta_gamma() : metacommunity similarity-sensitive diversity

Phylogenetic diversity

Phylogenetic diversity (described here) is automatically included in the Diversity module when the Phylo package is loaded. Documentation for these diversity measures can be found here. The phylogenetics code relies on the Phylo package to generate trees to incorporate into the diversity code, and the ppropriate code will be created when both main packages are loaded:

julia> using Diversity, Phylo

julia> communities = [4 1; 3 2; 1 0; 0 1] / 12;

julia> nt = rand(Nonultrametric(4))
RootedTree with 4 tips, 7 nodes and 6 branches.
Leaf names are tip 1, tip 2, tip 4 and tip 3


julia> metaphylo = Metacommunity(communities, PhyloBranches(nt));

julia> raw_meta_rho(metaphylo, [1, 2])
2×8 DataFrame
│ Row │ div_type            │ measure │ q     │ type_level │ type_name │ partition_level │ partition_name │ diversity │
│     │ String              │ String  │ Int64 │ String     │ String    │ String          │ String         │ Float64   │
├─────┼─────────────────────┼─────────┼───────┼────────────┼───────────┼─────────────────┼────────────────┼───────────┤
│ 1   │ Phylogenetic Branch │ RawRho  │ 1     │ types      │           │ metacommunity   │                │ 1.7787    │
│ 2   │ Phylogenetic Branch │ RawRho  │ 2     │ types      │           │ metacommunity   │                │ 1.66749   │

The package also provides some other sub-modules for related measures:

Diversity.Ecology

Many existing ecological diversity measures can be derived from our diversity measures, and so we provide them in the Diversity.Ecology submodule along with generalised versions of them that relate to our general measures of alpha, beta and gamma diversity at subcommunity and metacommunity levels. The generalisations of species richness, Shannon entropy and Simpson's index are the only standard measures we are aware of whose subcommunity components sum directly to the corresponding metacommunity measure (although note that Simpson's index decreases for increased diversity, so small components are more diverse). Documentation for these diversity measures can be found here.

Diversity.Hill

Hill numbers are found in the Diversity.Hill sub-module. Documentation for these diversity measures can be found here.

Diversity.Jost

Lou Jost's diversity measures are found in the Diversity.Jost sub-module. Documentation for these diversity measures is here.

Documentation

Documentation is generated by the Base documentation in Julia and online via the Documenter package.

Using the docs

Accessing the documentation in Julia is easy:

using Diversity

# Returns any documentation for the subdiv() function
?subdiv

The documentation is also available online.

Latest release

The online documentation for the current stable release is here.

Development branch

The online documentation for the latest dev (unreleased) branch is here.

Download Details:

Author: EcoJulia
Source Code: https://github.com/EcoJulia/Diversity.jl 
License: BSD-2-Clause license

#julia #packaging 

Aqua.jl: Auto QUality Assurance for Julia Packages

Aqua.jl: Auto QUality Assurance for Julia packages

Aqua.jl provides functions to run a few automatable checks for Julia packages:

• There are no method ambiguities.
• There are no undefined exports.
• There are no unbound type parameters.
• There are no stale dependencies listed in Project.toml.
• Check that test target of the root project Project.toml and test project (test/Project.toml) are consistent.
• Check that all external packages listed in deps have corresponding compat entry.
• Project.toml formatting is compatible with Pkg.jl output.

See more in the documentation.

Quick usage

Call Aqua.test_all(YourPackage) from test/runtests.jl, e.g.,

using YourPackage
using Aqua
Aqua.test_all(YourPackage)

Specifying Aqua version

To avoid breaking test when a new Aqua.jl version is released, it is recommended to add version bound for Aqua.jl in test/Project.toml:

[deps]
Aqua = "4c88cf16-eb10-579e-8560-4a9242c79595"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[compat]
Aqua = "0.5"

Badge

You can add the following line in README.md to include Aqua.jl badge:

[![Aqua QA](https://raw.githubusercontent.com/JuliaTesting/Aqua.jl/master/badge.svg)](https://github.com/JuliaTesting/Aqua.jl)

which is rendered as

Download Details:

Author: JuliaTesting
Source Code: https://github.com/JuliaTesting/Aqua.jl 
License: MIT license

#julia #packages #functions 

TiledIteration.jl: Julia Package to Facilitate Writing Mulithreaded

TiledIteration

This Julia package handles some of the low-level details for writing cache-efficient, possibly-multithreaded code for multidimensional arrays. A "tile" corresponds to a chunk of a larger array, typically a region that is large enough to encompass any "local" computations you need to perform; some of these computations may require temporary storage.

Usage

This package offers two basic kinds of functionality: the management of temporary buffers for processing on tiles, and the iteration over disjoint tiles of a larger array.

Iteration

SplitAxis and SplitAxes

The main use for these simple types is in distributing work across threads, usually in circumstances that do not require multidimensional locality as provided by TileIterator. SplitAxis splits a single array axis, and SplitAxes splits multidimensional axes along the final axis. For example:

julia> using TiledIteration

julia> A = rand(3, 20);

julia> collect(SplitAxes(axes(A), 4))
4-element Vector{Tuple{UnitRange{Int64}, UnitRange{Int64}}}:
 (1:3, 1:5)
 (1:3, 6:10)
 (1:3, 11:15)
 (1:3, 16:20)

You can also reduce the amount of work assigned to thread 1 (often the main thread is responsible for scheduling the other threads):

julia> collect(SplitAxes(axes(A), 3.5))
4-element Vector{Tuple{UnitRange{Int64}, UnitRange{Int64}}}:
 (1:3, 1:2)
 (1:3, 3:8)
 (1:3, 9:14)
 (1:3, 15:20)

Using "3.5 chunks" forces the later workers to perform 6 columns of work (rounding 20/3.5 up to the next integer), leaving only two columns remaining for the first thread.

TileIterator

More general iteration over disjoint tiles of a larger array can be done with TileIterator:

using TiledIteration

A = rand(1000,1000);   # our big array
for tileaxs in TileIterator(axes(A), (128,8))
    @show tileaxs
end

This produces

tileaxs = (1:128,1:8)
tileaxs = (129:256,1:8)
tileaxs = (257:384,1:8)
tileaxs = (385:512,1:8)
tileaxs = (513:640,1:8)
tileaxs = (641:768,1:8)
tileaxs = (769:896,1:8)
tileaxs = (897:1000,1:8)
tileaxs = (1:128,9:16)
tileaxs = (129:256,9:16)
tileaxs = (257:384,9:16)
tileaxs = (385:512,9:16)
...

You can see that the total axes range is split up into chunks, which are of size (128,8) except at the edges of A. Naturally, these axes serve as the basis for processing individual chunks of the array.

As a further example, suppose you've started julia with JULIA_NUM_THREADS=4; then

function fillid!(A, tilesz)
    tileinds_all = collect(TileIterator(axes(A), tilesz))
    Threads.@threads for i = 1:length(tileinds_all)
        tileaxs = tileinds_all[i]
        A[tileaxs...] .= Threads.threadid()
    end
    A
end

A = zeros(Int, 8, 8)
fillid!(A, (2,2))

would yield

8×8 Array{Int64,2}:
 1  1  2  2  3  3  4  4
 1  1  2  2  3  3  4  4
 1  1  2  2  3  3  4  4
 1  1  2  2  3  3  4  4
 1  1  2  2  3  3  4  4
 1  1  2  2  3  3  4  4
 1  1  2  2  3  3  4  4
 1  1  2  2  3  3  4  4

See also "EdgeIterator" below.

Determining the chunk size

Stencil computations typically require "padding" values, so the inputs to a computation may be of a different size than the resulting outputs. Naturally, you can set the tile size manually; a simple convenience function, padded_tilesize, attempts to pick reasonable choices for you depending on the size of your kernel (stencil) and element type you'll be using:

julia> padded_tilesize(UInt8, (3,3))
(768,18)

julia> padded_tilesize(UInt8, (3,3), 4)  # we want 4 of these to fit in L1 cache at once
(512,12)

julia> padded_tilesize(Float64, (3,3))
(96,18)

julia> padded_tilesize(Float32, (3,3,3))
(64,6,6)

Allocating and managing temporary storage

To allocate temporary storage while working with tiles, use TileBuffer:

julia> tileaxs = (-1:15, 0:7)  # really this might have come from TileIterator

julia> buf = TileBuffer(Float32, tileaxs)
TiledIteration.TileBuffer{Float32,2,2} with indices -1:15×0:7:
 0.0  0.0          2.38221f-44  0.0          0.0          0.0          9.3887f-44   0.0
 0.0  1.26117f-44  0.0          0.0          0.0          8.26766f-44  0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          6.02558f-44  0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          7.28675f-44  0.0          0.0          0.0
 0.0  1.54143f-44  0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          9.94922f-44  0.0
 0.0  0.0          0.0          0.0          0.0          8.82818f-44  0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          9.10844f-44  0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          1.03696f-43  0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0

This returns an uninitialized buffer for use over the indicated domain. You can reuse this same storage for the next tile, even if the tile is smaller because it corresponds to the edge of the original array:

julia> pointer(buf)
Ptr{Float32} @0x00007f79131fd550

julia> buf = TileBuffer(buf, (16:20, 0:7))
TiledIteration.TileBuffer{Float32,2,2} with indices 16:20×0:7:
 0.0  0.0  0.0  0.0          0.0          0.0  0.0          0.0
 0.0  0.0  0.0  0.0          0.0          0.0  0.0          0.0
 0.0  0.0  0.0  0.0          1.54143f-44  0.0  0.0          0.0
 0.0  0.0  0.0  1.26117f-44  0.0          0.0  0.0          0.0
 0.0  0.0  0.0  0.0          0.0          0.0  2.38221f-44  0.0

julia> pointer(buf)
Ptr{Float32} @0x00007f79131fd550

When you use it again at the top of the next block of columns, it returns to its original size while still reusing the same memory:

julia> buf = TileBuffer(buf, (-1:15, 8:15))
TiledIteration.TileBuffer{Float32,2,2} with indices -1:15×8:15:
 0.0  0.0          2.38221f-44  0.0          0.0          0.0          9.3887f-44   0.0
 0.0  1.26117f-44  0.0          0.0          0.0          8.26766f-44  0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          6.02558f-44  0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          7.28675f-44  0.0          0.0          0.0
 0.0  1.54143f-44  0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          9.94922f-44  0.0
 0.0  0.0          0.0          0.0          0.0          8.82818f-44  0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          9.10844f-44  0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          1.03696f-43  0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0
 0.0  0.0          0.0          0.0          0.0          0.0          0.0          0.0

julia> pointer(buf)
Ptr{Float32} @0x00007f79131fd550

EdgeIterator

When performing stencil operations, oftentimes the edge of the array requires special treatment. Several approaches to handling the edges (adding explicit padding, or executing special code just when on the boundaries) can slow your algorithm down because of extra steps or branches.

This package helps support implementations which first handle the "interior" of an array (for example using TiledIterator over just the interior) using a "fast path," and then handle just the edges by a (possibly) less carefully optimized algorithm. The key component of this is EdgeIterator:

outerrange = CartesianIndices((-1:4, 0:3))
innerrange = CartesianIndices(( 1:3, 1:2))
julia> for I in EdgeIterator(outerrange, innerrange)
           @show I
       end
I = CartesianIndex(-1, 0)
I = CartesianIndex(0, 0)
I = CartesianIndex(1, 0)
I = CartesianIndex(2, 0)
I = CartesianIndex(3, 0)
I = CartesianIndex(4, 0)
I = CartesianIndex(-1, 1)
I = CartesianIndex(0, 1)
I = CartesianIndex(4, 1)
I = CartesianIndex(-1, 2)
I = CartesianIndex(0, 2)
I = CartesianIndex(4, 2)
I = CartesianIndex(-1, 3)
I = CartesianIndex(0, 3)
I = CartesianIndex(1, 3)
I = CartesianIndex(2, 3)
I = CartesianIndex(3, 3)
I = CartesianIndex(4, 3)

The time required to visit these edge sites is on the order of the number of edge sites, not the order of the number of sites encompassed by outerrange, and consequently is efficient.

Download Details:

Author: JuliaArrays
Source Code: https://github.com/JuliaArrays/TiledIteration.jl 
License: View license

#julia #package #cache