Duktape.cr: Evaluate JavaScript From Crystal!

Duktape.cr

Duktape.cr provides Crystal bindings to the Duktape javascript engine.

Installation

Duktape.cr is best installed using Shards.

Add this to your shard.yml:

name: example  # your project's name
version: 1.0.0 # your project's version

dependencies:
  duktape:
    github: jessedoyle/duktape.cr
    version: ~> 1.0.0

then execute:

shards install

Shards will automatically make the native library. You can make the library manually by calling make libduktape.

Usage

You must first create a Duktape context:

require "duktape"

sbx = Duktape::Sandbox.new

sbx.eval! <<-JS
  var birthYear = 1990;

  function calcAge(birthYear){
    var current = new Date();
    var year = current.getFullYear();
    return year - birthYear;
  }

  print("You are " + calcAge(birthYear) + " years old.");
JS

An overwhelming majority of the Duktape API has been implemented. You can call the API functions directly on a Duktape::Sandbox or Duktape::Context instance:

sbx = Duktape::Sandbox.new
sbx.push_global_object   # [ global ]
sbx.push_string "Math"   # [ global "Math" ]
sbx.get_prop -2          # [ global Math ]
sbx.push_string "PI"     # [ global Math "PI" ]
sbx.get_prop -2          # [ global Math PI ]
pi = sbx.get_number -1
puts "PI: #{pi}"         # => PI: 3.14159
sbx.pop_3

Eval vs Eval!

All of the evaluation API methods have a corresponding bang-method (!). The bang method calls will raise when a javascript error occurs, the non-bang methods will not raise on invalid javascript.

For example:

sbx = Duktape::Context.new
sbx.eval <<-JS
  var a =
JS

will not raise any errors, but will return a non-zero error code.

The following code:

sbx = Duktape::Context.new
sbx.eval! <<-JS
  __invalid();
JS

will raise Duktape::SyntaxError.

Sandbox vs Context

You should only execute untrusted javascript code from within a Duktape::Sandbox instance. A sandbox isolates code from insecure operations such as Duktape's internal require mechanism and the Duktape global javascript object.

Creating a Duktape::Context gives code access to internal Duktape properties:

ctx = Duktape::Context.new
ctx.eval! <<-JS
  print(Duktape.version);
JS

Setting a Timeout

Duktape::Sandbox instances may optionally take an execution timeout limit in milliseconds. This provides protection against infinite loops when executing untrusted code.

A Duktape::RangeError exception is raised when the following code executes for longer than specified:

sbx = Duktape::Sandbox.new 500 # 500ms execution time limit
sbx.eval! "while (true) {}"    # => RangeError

Duktape::Runtime

An alternative interface for evaluating JS code is available via the Duktape::Runtime class. This class provides a streamlined evaluation API (similar to ExecJS) that allows easier access to javascript values without the need to call many low-level Duktape API functions.

The entire Runtime API is as follows:

• call(property, *args) - Call the property or function with the given arguments and return the result.
• call([properties], *args) - Call the property that is nested within an array of string property names.
• eval(source) - Evaluate the javascript source and return the last value.
• exec(source) - Evaluate the javascript source and always return nil.

Duktape::Runtime instances can also be provided an initialization block when created.

Here's an example:

  require "duktape/runtime"

  # A Runtime (optionally) accepts an initialization block
  rt = Duktape::Runtime.new do |sbx|
    sbx.eval! <<-JS
      function test(a, b, c) { return a + b + c; }
    JS
  end

  rt.call("test", 3, 4, 5) # => 12.0 (same as test(3, 4, 5);)
  rt.call(["Math", "PI"])  # => 3.14159
  rt.eval("1 + 1")         # => 2.0
  rt.exec("1 + 1")         # => nil

Note that duktape/runtime is not loaded by the base duktape require, and may be used standalone if necessary (ie. replace your require "duktape" calls with require "duktape/runtime" if you want this functionality).

Calling Crystal Code from Javascript

Note: This functionality is considered experimental and syntax/functionality may change dramatically between releases.

It is possible to call Crystal code from your javascript:

  sbx = Duktape::Sandbox.new

  # Push a global function named "add_together"
  # that accepts two arguments.
  sbx.push_global_proc("add_together", 2) do |ptr|
    env = Duktape::Sandbox.new ptr

    # Get the two arguments
    # from the stack
    a = env.require_number 0
    b = env.require_number 1

    env.push_number a + b # Push the return value to the stack
    env.call_success      # call_success -> stack top is value returned
  end

  sbx.eval! "print(add_together(2, 3));" # => 5

The proc object that is pushed to the Duktape stack accepts a pointer to a Context instance. We must wrap this pointer by calling env = Duktape::Sandbox.new ptr. The proc must also return an Int32 status code - env.call_failure and env.call_success will provide the proper integer values.

Note: Because it is currently not possible to pass closures to C bindings in Crystal, one must be careful that any variables used in the proc must not be referenced or initialized outside the scope of the proc. This is why variable names such as env are used.

Exceptions

The following exceptions may be thrown at runtime and may be rescued normally:

• Duktape::Error
• Duktape::EvalError
• Duktape::RangeError
• Duktape::ReferenceError
• Duktape::SyntaxError
• Duktape::TypeError
• Duktape::URIError

These exceptions all inherit from Duktape::Error, so it may be used as a catch-all for runtime errors.

The following exceptions represent errors internal to the Duktape engine and are generally not recoverable when thrown from a context:

• Duktape::InternalError
• Duktape::HeapError

These exceptions all inherit from Duktape::InternalError.

Contributing

I'll accept any pull requests that are well tested for bugs/features with Duktape.cr.

You should fork the main repo, create a feature branch, write tests and submit a pull request.

Download Details:

Author: jessedoyle
Source Code: https://github.com/jessedoyle/duktape.cr 
License: MIT license

#crystal #duktape 

Crystal-hunspell: Crystal Bindings for Hunspell

Crystal-hunspell

Crystal bindings for Hunspell.

Installation

Before installing crystal-hunspell ensure you have hunspell already installed:

Add the dependency to your shard.yml:

dependencies:
  hunspell:
    github: mamantoha/crystal-hunspell

Run shards install

Usage

require "hunspell"

Open a dictionary:

hunspell = Hunspell.new("/usr/share/hunspell/en_US.aff", "/usr/share/hunspell/en_US.dic")

or

hunspell = Hunspell.new("en_US")

Check if a word is valid:

hunspell.spellcheck("crystal")
# => true

hunspell.spellcheck("cristal")
# => false

Find the stems of a word:

hunsell.stem("fishing")
# => ["fishing", "fish"]

Suggest alternate spellings for a word:

hunspell.suggest("arbitrage")
# => ["arbitrage", "arbitrages", "arbitrager", "arbitraged", "arbitrate"]

Ensure to close Hunspell instance after using.

hunspell.close

Development

sudo apt install libclang-dev libhunspell-dev

Generate new bindings for Hunspell

crystal ./lib/crystal_lib/src/main.cr src/hunspell/lib_hunspell.cr.in > src/hunspell/lib_hunspell.cr

Contributing

1. Fork it (https://github.com/mamantoha/crystal-hunspell/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• Anton Maminov - creator and maintainer

Download Details:

Author: Mamantoha
Source Code: https://github.com/mamantoha/crystal-hunspell 
License: MIT license

#crystal #wrapper #binding 

Crt.cr: Bindings for Libncursesw and Crt Class

Crt.cr

Bindings for libncursesw and crt class

• crystal: 0.27.2 0.31.1 0.32.1 0.33.0

Installation

Add this to your application's shard.yml:

dependencies:
  crt:
    github: maiha/crt.cr
    version: 0.4.2

Usage

require "crt"

Crt.init

window = Crt::Window.new(5, 21)
window.print(2, 4, "Hello, world!")
window.refresh

sleep 10.seconds

Crt.done

See ./examples/ for more usages.

Supported C Functions

• see: https://github.com/maiha/crt.cr/blob/master/src/libncursesw.cr

Development

• for ubuntu

apt-get install libncursesw5-dev
apt-get install libgpm-dev        # needs only for static link

Contributing

1. Fork it ( https://github.com/maiha/crt.cr/fork )
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• maiha maiha - creator, maintainer

Download Details:

Author: Maiha
Source Code: https://github.com/maiha/crt.cr 
License: MIT license

#crystal #binding 

Clang.cr: Libclang Bindings for Crystal

libclang bindings for Crystal

Usage:

require "clang"

index = Clang::Index.new

files = [
  #Clang::UnsavedFile.new("input.c", "#include <pcre.h>\n"),
  Clang::UnsavedFile.new("input.c", "#include <clang/Basic/ABI.h>\n"),
]
tu = Clang::TranslationUnit.from_source(index, files, [
  "-I/usr/include",
  "-I/usr/lib/llvm-5.0/include",
])

tu.cursor.visit_children do |cursor|
  p cursor

  Clang::ChildVisitResult::Continue
end

Samples

See the samples folder for some example usages:

• samples/debug.cr will print the AST of C or C++ headers as they are parsed;
• samples/c2cr.cr will automatically generate Crystal bindings for a C header.

For example:

$ shards build --release

$ bin/c2cr -I/usr/lib/llvm-5.0/include llvm-c/Core.h \
    --remove-enum-prefix=LLVM --remove-enum-suffix > llvm-c/Core.cr

$ bin/c2cr -I/usr/lib/llvm-5.0/include clang-c/Index.h \
    --remove-enum-prefix > clang-c/Index.cr

$ bin/c2cr gtk-2.0/gtk/gtkenums.h --remove-enum-prefix > gtk/enums.cr

Reference

• C interface to Clang

Download Details:

Author: Crystal-lang
Source Code: https://github.com/crystal-lang/clang.cr 
License: View license

#crystal #lang 

Augeas.cr: Crystal C bindings for Augeas

Augeas

Crystal C Bindings for Augeas

Installation

Pre-Requisites

Before adding this shard, need to install the libaugeas.

On Mac brew install augeas

On Linux apt-get install libaugeas-dev

Crystal

Add the dependency to your shard.yml:

dependencies:
  augeas:
    github: fernandes/augeas

Run shards install

Usage

require "augeas"

Initializing Augeas

Initial step is always setup Augeas to your root path:

augeas = Augeas.new(root: "/path/to/root")

Getting a Value

Consider getting value for a postgresql conf setting:

augeas_file = augeas.load_file("/var/lib/pgsql/data/postgresql.conf")
augeas_file.get("data_directory")
# => "/var/lib/postgresql/8.4/main"

Setting a value

augeas_file = augeas.load_file("/var/lib/pgsql/data/postgresql.conf")
augeas_file.set("data_directory", "/my/new/path/to/postgresql/8.4/main")
# => true
augeas.save # this is a important step, do not forget to save

Development

Install the libraries on your local environment, write a spec, make it pass and :shipit:

LibAugeas has all the methods mapped, the comments there are from the C header.

Augeas needs to pass a pointer to Augeas on every method, that's why we pass the Augeas crystal object to Augeas::File, so we can encapsulate the logic to manipulate a file inside it. We have the penalty on needing to call Augeas#save, but this is something we can abstract in the future with kind of DSL/block.

This library is on early stages, the next steps are (to be developed as needed):

•  Better error handling
  •  Specially when there are more than 1 node
•  Support augeas operations
  •  Single Node Operations (aug_rm, aug_mv, aug_cp)
  •  Deal with matches (aug_match)
  •  Support aug_ns_* functions

About error handling, as we need to check the error messages on Augeas, the idea is to centralize all the error message / exceptions in one place, for now keeping in one method works, maybe a refactor to a new file can make sense.

Contributing

1. Fork it (https://github.com/fernandes/augeas/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• Celso Fernandes

Download Details:

Author: Fernandes
Source Code: https://github.com/fernandes/augeas.cr 
License: MIT license

#devops #crystal #binding 

Native Library Implementing Secp256k1 Purely for The Crystal Language

Secp256k1.cr

A library implementing the Secp256k1 elliptic curve natively in pure Crystal. Secp256k1 is the elliptic curve used in the public-private-key cryptography required by Bitcoin, Ethereum, and Polkadot.

This library allows for:

• providing a Secp256k1 cryptographic context, see Secp256k1::Context
• managing Secp256k1 signatures and verification, see Secp256k1::Signature
• managing private-public keypairs, see Secp256k1::Key
• generating public keys, see Secp256k1::Point
• generating private keys, see Secp256k1::Num

Installation

Add the Secp256k1 library to your shard.yml

dependencies:
  secp256k1:
    github: q9f/secp256k1.cr
    version: "~> 0.5"

Usage

Import and expose the Secp256k1 module.

require "secp256k1"

This library exposes the following modules and classes (in logical order):

• Secp256k1: necessary constants and data structures, including:
  • Secp256k1::Num: for managing big numerics (private keys)
  • Secp256k1::Point: for handling of elliptic curve points (public keys)
  • Secp256k1::Key: for managing private-public keypairs (accounts)
  • Secp256k1::Signature: for handling ECDSA signatures (r, s, v)
• Secp256k1::Context: providing a cryptographic context for signing and verification
• Secp256k1::Curve: the entire core mathematics behind the elliptic curve cryptography
• Secp256k1::Util: binding of various hashing algorithms for convenience

Basic usage:

# generates a new, random keypair
key = Secp256k1::Key.new
# => #<Secp256k1::Key:0x7fad7235aee0
#          @private_key=#<Secp256k1::Num:0x7fad7235d300
#              @hex="3ccf84820c20d5e8c536ba84c52ba410375b29b1812b5f7e722445c969a0fb30",
#              @dec=27505422793993207218034260454067205887515304192802142316084292370834437241648,
#              @bin=Bytes[60, 207, 132, 130, 12, 32, 213, 232, 197, 54, 186, 132, 197, 43, 164, 16, 55, 91, 41, 177, 129, 43, 95, 126, 114, 36, 69, 201, 105, 160, 251, 48]>,
#          @public_key=#<Secp256k1::Point:0x7fad7235ad20
#              @x=#<Secp256k1::Num:0x7fad69294ec0
#                  @hex="cd4a8712ee6efc15b5abe37c0dbfa979d89c427d3fe24b076008decefe94dba2",
#                  @dec=92855812888509048668847240903552964511053624688683992093822247249407942908834,
#                  @bin=Bytes[205, 74, 135, 18, 238, 110, 252, 21, 181, 171, 227, 124, 13, 191, 169, 121, 216, 156, 66, 125, 63, 226, 75, 7, 96, 8, 222, 206, 254, 148, 219, 162]>,
#              @y=#<Secp256k1::Num:0x7fad69294e80
#                  @hex="81363d298e4a40ebcb13f1afa85a0b94b967f243ee59a59010cb5deaf0d7b66c",
#                  @dec=58444189335609256006902338825877424261513225250255958585656342678587884156524,
#                  @bin=Bytes[129, 54, 61, 41, 142, 74, 64, 235, 203, 19, 241, 175, 168, 90, 11, 148, 185, 103, 242, 67, 238, 89, 165, 144, 16, 203, 93, 234, 240, 215, 182, 108]>>>

# gets the private key
key.private_hex
# => "3ccf84820c20d5e8c536ba84c52ba410375b29b1812b5f7e722445c969a0fb30"

# gets the compressed public key with prefix
key.public_hex_compressed
# => "02cd4a8712ee6efc15b5abe37c0dbfa979d89c427d3fe24b076008decefe94dba2"

Signature generation and verification:

# sign a message with a private key
ctx = Secp256k1::Context.new
priv = Secp256k1::Num.new "1f0c122d41ff536b19bfd83537c0dfc290e45cd3c375a43237c8b8fff7ac8af7"
key = Secp256k1::Key.new priv
hash = Secp256k1::Util.sha256 "Henlo, Wordl"
sig = ctx.sign key, hash
# => #<Secp256k1::Signature:0x7f5332e1d9c0
#          @r=#<Secp256k1::Num:0x7f5332decac0
#              @hex="c4079db44240b7afe94985c69fc89602e33629fd9b8623d711c30ce6378b33df",
#              @dec=88666774685717741514025410921892109286073075687452443491001272268566542627807,
#              @bin=Bytes[196, 7, 157, 180, 66, 64, 183, 175, 233, 73, 133, 198, 159, 200, 150, 2, 227, 54, 41, 253, 155, 134, 35, 215, 17, 195, 12, 230, 55, 139, 51, 223]>,
#          @s=#<Secp256k1::Num:0x7f5332deca80
#              @hex="6842c1b63c94bdb8e4f5ae88fb65f7a98b77b197c8323004fb47ef57fab29053",
#              @dec=47158485109070227797431103290229472044663017260590156038384319099500326195283,
#              @bin=Bytes[104, 66, 193, 182, 60, 148, 189, 184, 228, 245, 174, 136, 251, 101, 247, 169, 139, 119, 177, 151, 200, 50, 48, 4, 251, 71, 239, 87, 250, 178, 144, 83]>,
#          @v=#<Secp256k1::Num:0x7f5332deca40
#              @hex="00",
#              @dec=0,
#              @bin=Bytes[0]>>

# verify a signature with a public key
r = Secp256k1::Num.new "c4079db44240b7afe94985c69fc89602e33629fd9b8623d711c30ce6378b33df"
s = Secp256k1::Num.new "6842c1b63c94bdb8e4f5ae88fb65f7a98b77b197c8323004fb47ef57fab29053"
v = Secp256k1::Num.new "00"
sig = Secp256k1::Signature.new r, s, v
hash = Secp256k1::Util.sha256 "Henlo, Wordl"
publ = Secp256k1::Point.new "0416008a369439f1a8a75cf974860bed5b10180518d6b1dd3ac847f423fd375d6aa29474394f0cd79d2ea543507d069e97339284f01bdbfd27392daec0ec553816"
ctx.verify sig, hash, publ
# => true

There are example scripts for generating Bitcoin and Ethereum accounts in src/bitcoin.cr and src/ethereum.cr.

Documentation

The full library documentation can be found here: q9f.github.io/secp256k1.cr

Generate a local copy with:

crystal docs

Testing

The library is entirely specified through tests in ./spec; run:

crystal spec --verbose

Understand

Private keys are just scalars (Secp256k1::Num) and public keys are points (Secp256k1::Point) with x and y coordinates.

Bitcoin public keys can be uncompressed p|x|y or compressed p|x. both come with a prefix p which is useless for uncompressed keys but necessary for compressed keys to recover the y coordinate on the Secp256k1 elliptic curve field.

Ethereum public keys are uncompressed x|y without any prefix. The last 20 bytes slice of the y coordinate is actually used as address without any checksum. A checksum was later added in EIP-55 using a keccak256 hash and indicating character capitalization.

Neither Bitcoin nor Ethereum allow for recovering public keys from an address unless there exists a transaction with a valid signature on the blockchain.

Known issues

Note: this library should not be used in production without proper auditing. It should be considered slow and insecure.

• This library is not constant time and might be subject to side-channel attacks. (#4)
• This library does unnecessary big-integer math and should someday rather correctly implement the Secp256k1 prime field (#5)
• This library is slow in recovering signatures. Future versions should respect the recovery ID to quickly identify the correct public key from a signature.

Found any other issue? Report it: github.com/q9f/secp256k1.cr/issues

Contribute

Create a pull request, and make sure tests and linter pass.

This pure crystal implementation is based on the python implementation wobine/blackboard101 which is also used as reference to write tests against. It's a complete rewrite of the abandoned packetzero/bitcoinutils for educational purposes.

Honerable mention for the bitcoin wiki and the ethereum stackexchange for providing so many in-depth resources that supported this project in reimplementing everything.

Contributors: @q9f, @cserb, MrSorcus

Download Details:

Author: q9f
Source Code: https://github.com/q9f/secp256k1.cr 
License: Apache-2.0 license

#crystal #bitcoin #ethereum

Cocol: Rapid Blockchain Prototyping

COCOL!

Rapid blockchain prototyping

About 🌐

The Cocol Project has the goal to lower the entry barrier for developers interested in building blockchains and dApps. There is still a long way to go and your help is needed.

Installation 🏹

Cocol is written in Crystal, so make sure to follow the installation instructions first.

After setting up Crystal you can clone the Cocol repository and install the dependencies:

> git clone https://github.com/cocol-project/cocol.git
> cd cocol
> shards install

Usage ⚔

Make your changes to the code-base and than build Cocol

> make

The binary ./cocol offers the following CLI options

Options:

-p --port            The port your Cocol node is going to run on
-m --master          Making this node a master (there can only be one)
--max-connections    Setting the max-connections for peers.
--miner              Making this node a miner
--update             Triggering an update on launch (will catch up with the current height)

There is also a script that starts multiple nodes and the master for you

> ./script/start.sh 66 5

First option is the amount of nodes and the second amount of miner. It will start the master node with the port 3000 and every other node with 3000 + n, while all miner have port 4000 + n

Now go ahead and open the explorer in a browser:

> open ./explorer/index.html

You should see 66 nodes and a miner (red border)

Each one of the nodes has a REST API on the corresponding port (e.g. 3001)

Start creating live fake transactions and watch the network come alive

> crystal script/live.cr

Development 👩‍💻👨‍💻

Cocol is in a very early stage. Expect changes, bugs and messy code. Test coverage sucks atm.

Contributing ️👷‍♀️👷‍♂

1. Fork it ( https://github.com/cocol-project/cocol/fork )
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• github: cserb | twitter: @cerbivore | Cristian Șerb - creator, maintainer

Download Details:

Author: Cocol-project
Source Code: https://github.com/cocol-project/cocol 
License: MPL-2.0 license

#crystal #experimental #toolkit #blockchain 

Axentro: To Be The Go To Platform for Building DApps Quickly

Axentro

To be the go to platform for building dApps quickly and cheaply for business and gaming

Axentro is an original proof of work blockchain platform that empowers developers to build decentralised apps quickly and cheaply. It features the following:

• CPU mining using the award winning Argon2d hashing algorithm
• Decentralised apps platform
• Human readable addresses
• Globally distributed mining rewards
• Unique double helix chain with Slow and Fast transactions

Platform status

The platform phase one is complete and now stable with a live Mainnet. Phase two of the development will start early 2021 with a host of features for building decentralised apps as well as a mobile wallet.

Follow the Getting Started documentation to explore further.

Documentation

• Offical docs
• Roadmap

Blockchain Specification

Community

Questions or suggestions? Join our community here on Telegram

Active Team

• @kingsleyh Kingsley Hendrickse - founder & lead developer
• @aaronbronow Aaron Bronow - core team
• @fenicks Christian Kakesa - core team
• @mathieulaporte Mathieu Laporte - core team
• @m-o-e Moe - core team
• @bararchy Bar Hofesh - Security advisor

Past Contributors

• @tbrand Taichiro Suzuki - original founder
• @jferas John Feras
• @raymanoz Raymond Barlow
• @watzon Chris Watson
• @mamalisk Kostas Mamalis

Download Details:

Author: Axentro
Source Code: https://github.com/Axentro/Axentro 
License: MIT license

#crystal #bitcoin #blockchain 

Splay Tree implementation that conforms to the Hash ducktype

Splay Tree Map

A splay tree is a type of binary search tree that self organizes so that the most frequently accessed items tend to be towards theroot of the tree, where they can be accessed more quickly.

This implementation provides a hash-like interface, and it provides a couple features not typically found in Splay Trees -- efficient removal of the items that are generally least frequently accessed, and an extra fast search option.

Leaf Pruning

Because splay trees tend to organize themselves with the most frequently accessed elements towards the root of the tree, the least frequently accessed items tend to migrate towards the leaves of the tree. This implementation offers a method that can be used to prune its leaves, which generally has the effect of removing the least frequently accessed items from the tree.

This is useful if the data structure is being used to implement a cache, as it can be used to control the size of the cache while generaly keeping the most useful items in the cache without any other extensive bookkeeping.

Search without Splaying

A splay operation is generally performed on any access to a splay tree. This is the operation that moves the most important items towards the root. This operation has a cost to it, however, and there are times when it is desireable to search the hash without a splay operation occuring for the key that is searched. This results in a faster search operation, at the cost of not performing any efficiency improving structural changes to the tree. This should not be the primary search method that is used, but it can be useful at the right time.

Maximum Size

If #maxsize is set to an integer alue, then the splay tree will perform a prune operation when the maximum size of the tree is reached. This is useful for implementing a cache.

Installation

Add the dependency to your shard.yml:

dependencies:
  splay_tree_map:
    github: wyhaines/splay_tree_map.cr

Run shards install

Usage

Full documentation can be found at: https://wyhaines.github.io/splay_tree_map.cr/index.html

require "splay_tree_map"

Generally, the data structure is used like a hash.

stm = SplayTreeMap(String, String).new
stm.maxsize = 10

stm["this"] = "that"
stm["something"] = "else"
stm["junk"] = "pile"

if stm.has_key?("this")
  puts stm["this"]
end

stm.delete("junk")

puts stm.obtain("something") # This finds, but doesn't splay.

stm.prune # remove all leaves

Testing

To run the specs run crystal spec. To run specs with more debugging output use LOG_LEVEL=DEBUG crystal spec.

TODO

Experiment with other variations of splay operations, such as lazy semi-splay to see if performance can be improved. Right now this isn't any better than just using a Hash and arbitrarily deleting half of the hash if it grows too big.

Credits

This implementation is derived from the incomplete and broken implementation in the Crystalline shard found at https://github.com/jtomschroeder/crystalline

Contributing

1. Fork it (https://github.com/wyhaines/splay_tree_map/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• Kirk Haines - creator and maintainer

Download Details:

Author: Wyhaines
Source Code: https://github.com/wyhaines/splay_tree_map.cr 
License: Apache-2.0 license

 #crystal #tree #map 

Secure Remote Password for Crystal Lang

Secure Remote Password for Crystal Lang

This is a pure Crystal implementation of the Secure Remote Password protocol (SRP-6a).

SRP is an authentication method that allows the use of user names and passwords over an insecure network connection without revealing the password. If the client side lacks the user's password or the server side lacks the proper verification key, the authentication will fail.

Unlike other common challenge-response autentication protocols, such as Kerberos and SSL, SRP does not rely on an external infrastructure of trusted key servers or certificate management.

HomeKit Accessory Protocol Specification (HAP)

HomeKit authentication is supported when using SHA-512, these changes are made to the SRP protocol:

• SHA-512 is used as the hash function, replacing SHA-1
• The Modulus, N, and Generator, g, are specified by the 3072-bit group of RFC 5054
• The match, M, hash calculation is not padded

These changes improve security and are used as defaults

References

• RFC 2945
• RFC 5054
• Homekit accessory protocol specification (non-commercial version)
• http://srp.stanford.edu/
• http://srp.stanford.edu/demo/demo.html

Installation

Add the dependency to your shard.yml:

  dependencies:
    secure-remote-password:
      github: spider-gazelle/secure-remote-password

Usage

require 'secure-remote-password'

username = "user"
password = "password"

# The username, verifier and salt should be stored in the server database
server_verifier = SecureRemotePassword::Verifier.new
auth = verifier.generate_user_verifier(username, password)
auth # => {username: username, verifier: ..., salt: ...}

# ~~~ Begin Authentication ~~~

client = SecureRemotePassword::Client.new(username, password)
client_a = client.start_authentication

# Send username and client_a to the server
# Client => Server: username, client_a

# Server retrieves user's verifier and salt from the database.
# auth = lookup_user(username)
salt = auth[:salt]
verifier = auth[:verifier]

# Server generates challenge for the client.
challenge, proof = server_verifier.get_challenge_and_proof(username, verifier, salt, client_a)

# Server sends the challenge containing salt and proof (B) to client.
# Server => Client: challenge.salt, challenge.proof (B)

# Client calculates match (M) as a response to the challenge.
client_m = client.process_challenge(challenge)

# Client => Server: username, client_m

# Verify challenge response M.
# The Verifier state is passed in proof (server should persist this during negotiation)
server_h_amk = verifier.verify_session(proof, client_m)
# is nil if authentication failed.


# At this point, the client and server should have a common session key
# that is secure (i.e. not known to an outside party).  To finish
# authentication, they must prove to each other that their keys are
# identical.


# server to send server_h_amk to the client
# Server => Client: server_h_amk

client.verify(server_h_amk) == true

Credit

The original SRP-6a work was done by lamikae in the srp-rb project. The HomeKit implementation by karlentwistle

Download Details:

Author: Spider-gazelle
Source Code: https://github.com/spider-gazelle/secure-remote-password 
License: MIT license

#crystal #password #remote 

S2_cells: Maps latitude and longitude to S2 Cells

Crystal Lang S2 Cells

Maps Lat Lon coordinates to S2 Cells. Useful for things like storing points in InfluxDB

Installation

Add the dependency to your shard.yml:

dependencies:
  s2_cells:
    github: spider-gazelle/s2_cells

Run shards install

Usage

require "s2_cells"

lat = -33.870456
lon = 151.208889
level = 24

cell = S2Cells.at(lat, lon).parent(level)
token = cell.to_token # => "3ba32f81"

# Or a little more direct
S2Cells::LatLon.new(lat, lon).to_token(level)

Download Details:

Author: Spider-gazelle
Source Code: https://github.com/spider-gazelle/s2_cells 
License: MIT license

#crystal #maps 

Radix Tree implementation for Crystal

Radix Tree

Radix tree implementation for Crystal language

Installation

Add this to your application's shard.yml:

dependencies:
  radix:
    github: luislavena/radix

Usage

Building Trees

You can associate a payload with each path added to the tree:

require "radix"

tree = Radix::Tree(Symbol).new
tree.add "/products", :products
tree.add "/products/featured", :featured

result = tree.find "/products/featured"

if result.found?
  puts result.payload # => :featured
end

The types allowed for payload are defined on Tree definition:

tree = Radix::Tree(Symbol).new

# Good, since Symbol is allowed as payload
tree.add "/", :root

# Compilation error, Int32 is not allowed
tree.add "/meaning-of-life", 42

Can combine multiple types if needed:

tree = Radix::Tree(Int32 | String | Symbol).new

tree.add "/", :root
tree.add "/meaning-of-life", 42
tree.add "/hello", "world"

Lookup and placeholders

You can also extract values from placeholders (as named segments or globbing):

tree.add "/products/:id", :product

result = tree.find "/products/1234"

if result.found?
  puts result.params["id"]? # => "1234"
end

Please see Radix::Tree#add documentation for more usage examples.

Caveats

Pretty much all Radix implementations have their limitations and this project is no exception.

When designing and adding paths to a Tree, please consider that two different named parameters cannot share the same level:

tree.add "/", :root
tree.add "/:post", :post
tree.add "/:category/:post", :category_post # => Radix::Tree::SharedKeyError

This is because different named parameters at the same level will result in incorrect params when lookup is performed, and sometimes the value for post or category parameters will not be stored as expected.

To avoid this issue, usage of explicit keys that differentiate each path is recommended.

For example, following a good SEO practice will be consider /:post as absolute permalink for the post and have a list of categories which links to the permalinks of the posts under that category:

tree.add "/", :root
tree.add "/:post", :post                    # this is post permalink
tree.add "/categories", :categories         # list of categories
tree.add "/categories/:category", :category # listing of posts under each category

Implementation

This project has been inspired and adapted from julienschmidt/httprouter and spriet2000/vertx-http-router Go and Java implementations, respectively.

Changes to logic and optimizations have been made to take advantage of Crystal's features.

Contributing

1. Fork it ( https://github.com/luislavena/radix/fork )
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• Luis Lavena - creator, maintainer

Download Details:

Author: luislavena
Source Code: https://github.com/luislavena/radix 
License: MIT license

#crystal #radix #tree 

Qr-code: A QR Code Implementation Written in Crystal Lang

Crystal Lang QR Code

Native crystal lang QR code, no external dependencies

Installation

Add the dependency to your shard.yml:

dependencies:
  qr-code:
    github: spider-gazelle/qr-code

Run shards install

Usage

require "qr-code"

qr = QRCode.new("my string to generate", size: 4, level: :h)
puts qr.to_s

output

xxxxxxx x  x x   x x  xx  xxxxxxx
x     x  xxx  xxxxxx xxx  x     x
x xxx x  xxxxx x       xx x xxx x
... etc

Doing your own rendering

require "qr-code"

qr = QRCode.new("my string to generate", size: 4, level: :h)
qr.modules.each do |row|
  row.each do |col|
    print col ? '#' : ' '
  end

  print "\n"
end

Rendering a SVG

require "qr-code"

svg_string = QRCode.new("my string to generate").as_svg

Rendering a PNG

you'll need to add stumpy_png to your shard.yml dependencies

require "qr-code"
require "qr-code/export/png"

# size == width, and QR codes are square
png_bytes = QRCode.new("my string to generate").as_png(size: 256)

Credits

Based off the ruby gem: https://github.com/whomwah/rqrcode_core Which was adapted from the javascript library: https://github.com/kazuhikoarase/qrcode-generator

Download Details:

Author: Spider-gazelle
Source Code: https://github.com/spider-gazelle/qr-code 
License: MIT license

#crystal #qrcode 

Multiset.cr: Multiset (bag) Implementation in Crystal

Multiset

A multiset (bag) implementation in Crystal.

Installation

Add this to your application's shard.yml:

dependencies:
  multiset:
    github: tcrouch/multiset.cr

Usage

require "multiset"

ms1 = Multiset{1, 1}
ms1 << 2                          # => Multiset{1, 1, 2}
ms1.merge [3, 4]                  # => Multiset{1, 1, 2, 3, 4}
ms2 = Multiset.new [2, 3, 4]
ms2.subset_of?(ms1)               # => true
ms1 & ms2                         # => Multiset{2, 3, 4}

Development

crystal spec

Contributing

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• [tcrouch] Tom Crouch - creator, maintainer

Download Details:

Author: tcrouch
Source Code: https://github.com/tcrouch/multiset.cr 
License: MIT license

#crystal #set #datastructures 

Crystal Library for Building Markov Chains & Running Markov Processes

⛓ Markov

A Crystal library for building Markov Chains and running Markov Processes.

What is a Markov Chain?

A Markov Chain is essentially a mechanism for guessing probable future events based on a sample of past events. For a great explanation, watch this Khan Academy video.

Visit the API Documentation for a more in-depth look at the library's functionality.

Installation

Add this to your application's shard.yml:

dependencies:
  markov:
    github: mccallofthewild/markov

In your terminal, install Crystal dependencies with:

$ shards install

or

$ crystal deps

Usage

Begin by requiring the Markov module:

require "markov"

Basic -- Hello Markov

A classic Markov text generator. This example will work well for small (array-sized) data sets.

NOTE: Markov::Chain is a generic type which contains, receives and generates elements of LinkType.

We'll start with the sample text:

example_string = "how much wood would a woodchuck chuck if a woodchuck could chuck wood"

There are several Markov::Chain constructors to choose from. The simplest one takes in a LinkType array of elements as sample and a seed of LinkType. seed is the element in sample you want to start the chain with. If not provided, a random element will be chosen.

example_arr = example_string.split(" ") #=> ["how","much","wood","would","a","woodchuck","chuck","if","a","woodchuck","could","chuck","wood"]
seed = example_arr[0] #=> "how"

example_chain = Markov::Chain(String).new sample: example_arr, seed: seed

Finally, we'll generate a probable sequence of elements with the Markov::Chain#generate method:

puts example_chain.generate(10)

Output:

["much", "wood", "would", "a", "woodchuck", "could", "chuck", "if", "a", "woodchuck"]

That's it!

If we wanted to get the elements one at a time, we could use the Markov::Chain#next method instead:

puts example_chain.next #=> "much"
puts example_chain.next #=> "wood"
puts example_chain.next #=> "would"

Advanced

This implementation was built for larger data sets, with asynchronous input in mind.

In this example, we will create a Markov::Chain which can generate realistic movie titles.

To begin, we instantiate a Markov::TransitionTable. A TransitionTable is a mechanism for training and implementing Markov processes.

example_table = Markov::TransitionTable(String).new

Markov::TransitionTable#add

Now we'll add a movie title using the Markov::TransitionTable#add method:

movie_one = %w(the great gatsby) # shortcut syntax for ["the","great","gatsby"]

movie_one.each do |word|
  example_table.add(word)
end

Markov::TransitionTable#add adds elements one at a time. At a deeper level, it's adding each new word to the previous word's Transition Matrix (Markov::TransitionMatrix).

Markov::TransitionTable#fill

For syntactic sugar, if we have an array of elements, we can avoid looping through and #add-ing them by using the Markov::TransitionTable#fill method instead:

movie_one = %w(the great gatsby) # shortcut syntax for ["the","great","gatsby"]

example_table.fill table_with: movie_one

Markov::TransitionTable#reset

A problem arises at this point:

movie_two = %w(great expectations)
example_table.fill table_with: movie_two

The above code sequentially adds each word to the TransitionTable. But The Great Gatsby and Great Expectations are two separate movie titles; the "Great" at the beginning of Great Expectations is not a probable transition from the "Gatsby" at the end of The Great Gatsby.

To solve this, use Markov::TransitionTable#reset. #reset clears the TransitionTable's last added key, allowing us to separate titles like so:

movie_one = %w(the great gatsby)
example_table.fill table_with: movie_one

example_table.reset
movie_two = %w(great expectations)
example_table.fill table_with: movie_two

example_table.reset
movie_three = %w(the great escape)
example_table.fill table_with: movie_three

Implementing the TransitionTable with a Markov::Chain

Finally, we can put the TransitionTable to use by passing it to a Markov::Chain constructor as transition_table:

example_chain = Markov::Chain(String).new transition_table: example_table, seed: "great"

Handling Dead Ends

With small and/or unique data sets, Markov chains are fallible to reaching dead ends. That is, they can often reach a point where there is nothing to transition to.

When this happens in the Markov module, Markov::Exceptions::EmptyTransitionMatrixException is raised.

For example:

dead_end_array = %w(some say the world will end in fire)
dead_end_chain = Markov::Chain(String).new sample: dead_end_array, seed: "fire"
# nothing comes after "fire", so the chain is at a dead end.
dead_end_chain.next # raises `EmptyTransitionMatrixException`

To prevent this, use the Markov::Chain#on_dead_end exception handler.

This method takes in a callback block with arguments of: the Markov::Chain's @transition_table, the Markov::Chain instance, and the EmptyTransitionMatrixException raised.

The block's return value of LinkType fills in as the next item in the chain.

dead_end_array = %w(some say the world will end in fire)
dead_end_chain = Markov::Chain(String).new sample: dead_end_array, seed: "fire"

dead_end_chain.on_dead_end do |transition_table, chain, exception|
  "some"
end

dead_end_chain.next #=> "some"
dead_end_chain.next #=> "say"
dead_end_chain.next #=> "the"

Contributing

1. Fork it ( https://github.com/mccallofthewild/markov/fork )
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• McCall Alexander mccallofthewild - creator, maintainer

Download Details:

Author: Mccallofthewild
Source Code: https://github.com/mccallofthewild/markov 
License: MIT license

#crystal #markov #naturallanguageprocessing