Gordon Taylor

Gordon Taylor

1660682340

Proxyquireify: Browserify >= V2 Version Of Proxyquire

proxyquireify 

browserify >= v2 version of proxyquire.

Proxies browserify's require in order to make overriding dependencies during testing easy while staying totally unobstrusive. To run your tests in both Node and the browser, use proxyquire-universal.

Features

  • no changes to your code are necessary
  • non overriden methods of a module behave like the original
  • mocking framework agnostic, if it can stub a function then it works with proxyquireify
  • "use strict" compliant
  • automatic injection of require calls to ensure the module you are testing gets bundled

Installation

npm install proxyquireify

To use with browserify < 5.1 please npm install proxyquireify@0.5 instead. To run your tests in PhantomJS, you may need to use a shim.

Example

foo.js:

var bar = require('./bar');

module.exports = function () {
  return bar.kinder() + ' ist ' + bar.wunder();
};

foo.test.js:

var proxyquire = require('proxyquireify')(require);

var stubs = { 
  './bar': { 
      wunder: function () { return 'wirklich wunderbar'; }
    , kinder: function () { return 'schokolade'; }
  }
};

var foo = proxyquire('./src/foo', stubs);

console.log(foo());

browserify.build.js:

var browserify = require('browserify');
var proxyquire = require('proxyquireify');

browserify()
  .plugin(proxyquire.plugin)
  .require(require.resolve('./foo.test'), { entry: true })
  .bundle()
  .pipe(fs.createWriteStream(__dirname + '/bundle.js'));

load it in the browser and see:

schokolade ist wirklich wunderbar

With Other Transforms

If you're transforming your source code to JavaScript, you must apply those transforms before applying the proxyquireify plugin:

browserify()
  .transform('coffeeify')
  .plugin(proxyquire.plugin)
  .require(require.resolve('./test.coffee'), { entry: true })
  .bundle()
  .pipe(fs.createWriteStream(__dirname + '/bundle.js'));

proxyquireify needs to parse your code looking for require statements. If you require anything that's not valid JavaScript that acorn can parse (e.g. CoffeeScript, TypeScript), you need to make sure the relevant transform runs before proxyquireify.

API

proxyquire.plugin()

proxyquireify functions as a browserify plugin and needs to be registered with browserify like so:

var browserify = require('browserify');
var proxyquire = require('proxyquireify');

browserify()
  .plugin(proxyquire.plugin)
  .require(require.resolve('./test'), { entry: true })
  .bundle()
  .pipe(fs.createWriteStream(__dirname + '/bundle.js'));

Alternatively you can register proxyquireify as a plugin from the command line like so:

browserify -p proxyquireify/plugin test.js > bundle.js

proxyquire.browserify()

Deprecation Warning

This API to setup proxyquireify was used prior to browserify plugin support.

It has not been removed yet to make upgrading proxyquireify easier for now, but it will be deprecated in future versions. Please consider using the plugin API (above) instead.

To be used in build script instead of browserify(), autmatically adapts browserify to work for tests and injects require overrides into all modules via a browserify transform.

proxyquire.browserify()
  .require(require.resolve('./test'), { entry: true })
  .bundle()
  .pipe(fs.createWriteStream(__dirname + '/bundle.js'));

proxyquire(request: String, stubs: Object)

  • request: path to the module to be tested e.g., ../lib/foo
  • stubs: key/value pairs of the form { modulePath: stub, ... }
    • module paths are relative to the tested module not the test file
    • therefore specify it exactly as in the require statement inside the tested file
    • values themselves are key/value pairs of functions/properties and the appropriate override
var proxyquire =  require('proxyquireify')(require);
var barStub    =  { wunder: function () { 'really wonderful'; } };

var foo = proxyquire('./foo', { './bar': barStub })

Important Magic

In order for browserify to include the module you are testing in the bundle, proxyquireify will inject a require() call for every module you are proxyquireing. So in the above example require('./foo') will be injected at the top of your test file.

noCallThru

By default proxyquireify calls the function defined on the original dependency whenever it is not found on the stub.

If you prefer a more strict behavior you can prevent callThru on a per module or per stub basis.

If callThru is disabled, you can stub out modules that weren't even included in the bundle. Note, that unlike in proxquire, there is no option to prevent call thru globally.

// Prevent callThru for path module only
var foo = proxyquire('./foo', {
    path: {
      extname: function (file) { ... }
    , '@noCallThru': true
    }
  , fs: { readdir: function (..) { .. } }
});

// Prevent call thru for all contained stubs (path and fs)
var foo = proxyquire('./foo', {
    path: {
      extname: function (file) { ... }
    }
  , fs: { readdir: function (..) { .. } }
  , '@noCallThru': true
});

// Prevent call thru for all stubs except path
var foo = proxyquire('./foo', {
    path: {
      extname: function (file) { ... }
    , '@noCallThru': false
    }
  , fs: { readdir: function (..) { .. } }
  , '@noCallThru': true
});

More Examples

Download Details:

Author: Thlorenz
Source Code: https://github.com/thlorenz/proxyquireify 
License: MIT license

#javascript #browserify 

What is GEEK

Buddha Community

Proxyquireify: Browserify >= V2 Version Of Proxyquire
Gordon Taylor

Gordon Taylor

1660682340

Proxyquireify: Browserify >= V2 Version Of Proxyquire

proxyquireify 

browserify >= v2 version of proxyquire.

Proxies browserify's require in order to make overriding dependencies during testing easy while staying totally unobstrusive. To run your tests in both Node and the browser, use proxyquire-universal.

Features

  • no changes to your code are necessary
  • non overriden methods of a module behave like the original
  • mocking framework agnostic, if it can stub a function then it works with proxyquireify
  • "use strict" compliant
  • automatic injection of require calls to ensure the module you are testing gets bundled

Installation

npm install proxyquireify

To use with browserify < 5.1 please npm install proxyquireify@0.5 instead. To run your tests in PhantomJS, you may need to use a shim.

Example

foo.js:

var bar = require('./bar');

module.exports = function () {
  return bar.kinder() + ' ist ' + bar.wunder();
};

foo.test.js:

var proxyquire = require('proxyquireify')(require);

var stubs = { 
  './bar': { 
      wunder: function () { return 'wirklich wunderbar'; }
    , kinder: function () { return 'schokolade'; }
  }
};

var foo = proxyquire('./src/foo', stubs);

console.log(foo());

browserify.build.js:

var browserify = require('browserify');
var proxyquire = require('proxyquireify');

browserify()
  .plugin(proxyquire.plugin)
  .require(require.resolve('./foo.test'), { entry: true })
  .bundle()
  .pipe(fs.createWriteStream(__dirname + '/bundle.js'));

load it in the browser and see:

schokolade ist wirklich wunderbar

With Other Transforms

If you're transforming your source code to JavaScript, you must apply those transforms before applying the proxyquireify plugin:

browserify()
  .transform('coffeeify')
  .plugin(proxyquire.plugin)
  .require(require.resolve('./test.coffee'), { entry: true })
  .bundle()
  .pipe(fs.createWriteStream(__dirname + '/bundle.js'));

proxyquireify needs to parse your code looking for require statements. If you require anything that's not valid JavaScript that acorn can parse (e.g. CoffeeScript, TypeScript), you need to make sure the relevant transform runs before proxyquireify.

API

proxyquire.plugin()

proxyquireify functions as a browserify plugin and needs to be registered with browserify like so:

var browserify = require('browserify');
var proxyquire = require('proxyquireify');

browserify()
  .plugin(proxyquire.plugin)
  .require(require.resolve('./test'), { entry: true })
  .bundle()
  .pipe(fs.createWriteStream(__dirname + '/bundle.js'));

Alternatively you can register proxyquireify as a plugin from the command line like so:

browserify -p proxyquireify/plugin test.js > bundle.js

proxyquire.browserify()

Deprecation Warning

This API to setup proxyquireify was used prior to browserify plugin support.

It has not been removed yet to make upgrading proxyquireify easier for now, but it will be deprecated in future versions. Please consider using the plugin API (above) instead.

To be used in build script instead of browserify(), autmatically adapts browserify to work for tests and injects require overrides into all modules via a browserify transform.

proxyquire.browserify()
  .require(require.resolve('./test'), { entry: true })
  .bundle()
  .pipe(fs.createWriteStream(__dirname + '/bundle.js'));

proxyquire(request: String, stubs: Object)

  • request: path to the module to be tested e.g., ../lib/foo
  • stubs: key/value pairs of the form { modulePath: stub, ... }
    • module paths are relative to the tested module not the test file
    • therefore specify it exactly as in the require statement inside the tested file
    • values themselves are key/value pairs of functions/properties and the appropriate override
var proxyquire =  require('proxyquireify')(require);
var barStub    =  { wunder: function () { 'really wonderful'; } };

var foo = proxyquire('./foo', { './bar': barStub })

Important Magic

In order for browserify to include the module you are testing in the bundle, proxyquireify will inject a require() call for every module you are proxyquireing. So in the above example require('./foo') will be injected at the top of your test file.

noCallThru

By default proxyquireify calls the function defined on the original dependency whenever it is not found on the stub.

If you prefer a more strict behavior you can prevent callThru on a per module or per stub basis.

If callThru is disabled, you can stub out modules that weren't even included in the bundle. Note, that unlike in proxquire, there is no option to prevent call thru globally.

// Prevent callThru for path module only
var foo = proxyquire('./foo', {
    path: {
      extname: function (file) { ... }
    , '@noCallThru': true
    }
  , fs: { readdir: function (..) { .. } }
});

// Prevent call thru for all contained stubs (path and fs)
var foo = proxyquire('./foo', {
    path: {
      extname: function (file) { ... }
    }
  , fs: { readdir: function (..) { .. } }
  , '@noCallThru': true
});

// Prevent call thru for all stubs except path
var foo = proxyquire('./foo', {
    path: {
      extname: function (file) { ... }
    , '@noCallThru': false
    }
  , fs: { readdir: function (..) { .. } }
  , '@noCallThru': true
});

More Examples

Download Details:

Author: Thlorenz
Source Code: https://github.com/thlorenz/proxyquireify 
License: MIT license

#javascript #browserify 

Fredy Larson

Fredy Larson

1601480520

Database Schema Versioning and Migrations Made Simpler For High Speed CI/CD

If you are a back-end developer, you are often faced with having to migrate your database schema with each new release.

The framework called Liquibase can make it easier for you when you need to upgrade your database schema.

In this article, I’ll explain how Liquibase can be used in a Java project, with Spring/Hibernate, to version the database schema.

How does Liquibase work?

Changelog

Liquibase works with Changelog files (the list of all the changes, in order, that need to execute to update the database).

There are 4 supported formats for these Changelogs: SQL, XML, YAML, and JSON.

#liquibase #database #versioning #plugins #java #hackernoon-top-story #database-schema-versioning #database-schema-migration

Python Library

Python Library

1641542520

Vulkan Kompute: General GPU Compute Framework for Graphics Cards

 

Kompute

The general purpose GPU compute framework for cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends)

Blazing fast, mobile-enabled, asynchronous, and optimized for advanced GPU acceleration usecases.

💬 Join the Discord & Community Calls 🔋 Documentation 💻 Blog Post ⌨ Examples 💾

Kompute is backed by the Linux Foundation as a hosted project by the LF AI & Data Foundation.

  

Principles & Features

Getting Started

Below you can find a GPU multiplication example using the C++ and Python Kompute interfaces.

You can join the Discord for questions / discussion, open a github issue, or read the documentation.

Your First Kompute (C++)

The C++ interface provides low level access to the native components of Kompute, enabling for advanced optimizations as well as extension of components.


void kompute(const std::string& shader) {

    // 1. Create Kompute Manager with default settings (device 0, first queue and no extensions)
    kp::Manager mgr; 

    // 2. Create and initialise Kompute Tensors through manager

    // Default tensor constructor simplifies creation of float values
    auto tensorInA = mgr.tensor({ 2., 2., 2. });
    auto tensorInB = mgr.tensor({ 1., 2., 3. });
    // Explicit type constructor supports uint32, int32, double, float and bool
    auto tensorOutA = mgr.tensorT<uint32_t>({ 0, 0, 0 });
    auto tensorOutB = mgr.tensorT<uint32_t>({ 0, 0, 0 });

    std::vector<std::shared_ptr<kp::Tensor>> params = {tensorInA, tensorInB, tensorOutA, tensorOutB};

    // 3. Create algorithm based on shader (supports buffers & push/spec constants)
    kp::Workgroup workgroup({3, 1, 1});
    std::vector<float> specConsts({ 2 });
    std::vector<float> pushConstsA({ 2.0 });
    std::vector<float> pushConstsB({ 3.0 });

    auto algorithm = mgr.algorithm(params,
                                   // See documentation shader section for compileSource
                                   compileSource(shader),
                                   workgroup,
                                   specConsts,
                                   pushConstsA);

    // 4. Run operation synchronously using sequence
    mgr.sequence()
        ->record<kp::OpTensorSyncDevice>(params)
        ->record<kp::OpAlgoDispatch>(algorithm) // Binds default push consts
        ->eval() // Evaluates the two recorded operations
        ->record<kp::OpAlgoDispatch>(algorithm, pushConstsB) // Overrides push consts
        ->eval(); // Evaluates only last recorded operation

    // 5. Sync results from the GPU asynchronously
    auto sq = mgr.sequence();
    sq->evalAsync<kp::OpTensorSyncLocal>(params);

    // ... Do other work asynchronously whilst GPU finishes

    sq->evalAwait();

    // Prints the first output which is: { 4, 8, 12 }
    for (const float& elem : tensorOutA->vector()) std::cout << elem << "  ";
    // Prints the second output which is: { 10, 10, 10 }
    for (const float& elem : tensorOutB->vector()) std::cout << elem << "  ";

} // Manages / releases all CPU and GPU memory resources

int main() {

    // Define a raw string shader (or use the Kompute tools to compile to SPIRV / C++ header
    // files). This shader shows some of the main components including constants, buffers, etc
    std::string shader = (R"(
        #version 450

        layout (local_size_x = 1) in;

        // The input tensors bind index is relative to index in parameter passed
        layout(set = 0, binding = 0) buffer buf_in_a { float in_a[]; };
        layout(set = 0, binding = 1) buffer buf_in_b { float in_b[]; };
        layout(set = 0, binding = 2) buffer buf_out_a { uint out_a[]; };
        layout(set = 0, binding = 3) buffer buf_out_b { uint out_b[]; };

        // Kompute supports push constants updated on dispatch
        layout(push_constant) uniform PushConstants {
            float val;
        } push_const;

        // Kompute also supports spec constants on initalization
        layout(constant_id = 0) const float const_one = 0;

        void main() {
            uint index = gl_GlobalInvocationID.x;
            out_a[index] += uint( in_a[index] * in_b[index] );
            out_b[index] += uint( const_one * push_const.val );
        }
    )");

    // Run the function declared above with our raw string shader
    kompute(shader);
}

Your First Kompute (Python)

The Python package provides a high level interactive interface that enables for experimentation whilst ensuring high performance and fast development workflows.


from .utils import compile_source # using util function from python/test/utils

def kompute(shader):
    # 1. Create Kompute Manager with default settings (device 0, first queue and no extensions)
    mgr = kp.Manager()

    # 2. Create and initialise Kompute Tensors through manager

    # Default tensor constructor simplifies creation of float values
    tensor_in_a = mgr.tensor([2, 2, 2])
    tensor_in_b = mgr.tensor([1, 2, 3])
    # Explicit type constructor supports uint32, int32, double, float and bool
    tensor_out_a = mgr.tensor_t(np.array([0, 0, 0], dtype=np.uint32))
    tensor_out_b = mgr.tensor_t(np.array([0, 0, 0], dtype=np.uint32))

    params = [tensor_in_a, tensor_in_b, tensor_out_a, tensor_out_b]

    # 3. Create algorithm based on shader (supports buffers & push/spec constants)
    workgroup = (3, 1, 1)
    spec_consts = [2]
    push_consts_a = [2]
    push_consts_b = [3]

    # See documentation shader section for compile_source
    spirv = compile_source(shader)

    algo = mgr.algorithm(params, spirv, workgroup, spec_consts, push_consts_a)

    # 4. Run operation synchronously using sequence
    (mgr.sequence()
        .record(kp.OpTensorSyncDevice(params))
        .record(kp.OpAlgoDispatch(algo)) # Binds default push consts provided
        .eval() # evaluates the two recorded ops
        .record(kp.OpAlgoDispatch(algo, push_consts_b)) # Overrides push consts
        .eval()) # evaluates only the last recorded op

    # 5. Sync results from the GPU asynchronously
    sq = mgr.sequence()
    sq.eval_async(kp.OpTensorSyncLocal(params))

    # ... Do other work asynchronously whilst GPU finishes

    sq.eval_await()

    # Prints the first output which is: { 4, 8, 12 }
    print(tensor_out_a)
    # Prints the first output which is: { 10, 10, 10 }
    print(tensor_out_b)

if __name__ == "__main__":

    # Define a raw string shader (or use the Kompute tools to compile to SPIRV / C++ header
    # files). This shader shows some of the main components including constants, buffers, etc
    shader = """
        #version 450

        layout (local_size_x = 1) in;

        // The input tensors bind index is relative to index in parameter passed
        layout(set = 0, binding = 0) buffer buf_in_a { float in_a[]; };
        layout(set = 0, binding = 1) buffer buf_in_b { float in_b[]; };
        layout(set = 0, binding = 2) buffer buf_out_a { uint out_a[]; };
        layout(set = 0, binding = 3) buffer buf_out_b { uint out_b[]; };

        // Kompute supports push constants updated on dispatch
        layout(push_constant) uniform PushConstants {
            float val;
        } push_const;

        // Kompute also supports spec constants on initalization
        layout(constant_id = 0) const float const_one = 0;

        void main() {
            uint index = gl_GlobalInvocationID.x;
            out_a[index] += uint( in_a[index] * in_b[index] );
            out_b[index] += uint( const_one * push_const.val );
        }
    """

    kompute(shader)

Interactive Notebooks & Hands on Videos

You are able to try out the interactive Colab Notebooks which allow you to use a free GPU. The available examples are the Python and C++ examples below:

Try the interactive C++ Colab from Blog PostTry the interactive Python Colab from Blog Post
  

You can also check out the two following talks presented at the FOSDEM 2021 conference.

Both videos have timestamps which will allow you to skip to the most relevant section for you - the intro & motivations for both is almost the same so you can skip to the more specific content.

Watch the video for C++ EnthusiastsWatch the video for Python & Machine Learning Enthusiasts
  

Architectural Overview

The core architecture of Kompute includes the following:

To see a full breakdown you can read further in the C++ Class Reference.

Full ArchitectureSimplified Kompute Components
 

(very tiny, check the full reference diagram in docs for details) 

Asynchronous and Parallel Operations

Kompute provides flexibility to run operations in an asynrchonous way through vk::Fences. Furthermore, Kompute enables for explicit allocation of queues, which allow for parallel execution of operations across queue families.

The image below provides an intuition on how Kompute Sequences can be allocated to different queues to enable parallel execution based on hardware. You can see the hands on example, as well as the detailed documentation page describing how it would work using an NVIDIA 1650 as an example.

Mobile Enabled

Kompute has been optimized to work in mobile environments. The build system enables for dynamic loading of the Vulkan shared library for Android environments, together with a working Android NDK wrapper for the CPP headers.

For a full deep dive you can read the blog post "Supercharging your Mobile Apps with On-Device GPU Accelerated Machine Learning".

You can also access the end-to-end example code in the repository, which can be run using android studio.

 

More examples

Simple examples

End-to-end examples

Python Package

Besides the C++ core SDK you can also use the Python package of Kompute, which exposes the same core functionality, and supports interoperability with Python objects like Lists, Numpy Arrays, etc.

The only dependencies are Python 3.5+ and Cmake 3.4.1+. You can install Kompute from the Python pypi package using the following command.

pip install kp

You can also install from master branch using:

pip install git+git://github.com/KomputeProject/kompute.git@master

For further details you can read the Python Package documentation or the Python Class Reference documentation.

C++ Build Overview

The build system provided uses cmake, which allows for cross platform builds.

The top level Makefile provides a set of optimized configurations for development as well as the docker image build, but you can start a build with the following command:

   cmake -Bbuild

You also are able to add Kompute in your repo with add_subdirectory - the Android example CMakeLists.txt file shows how this would be done.

For a more advanced overview of the build configuration check out the Build System Deep Dive documentation.

Kompute Development

We appreciate PRs and Issues. If you want to contribute try checking the "Good first issue" tag, but even using Kompute and reporting issues is a great contribution!

Contributing

Dev Dependencies

  • Testing
    • GTest
  • Documentation
    • Doxygen (with Dot)
    • Sphynx

Development

  • Follows Mozilla C++ Style Guide https://www-archive.mozilla.org/hacking/mozilla-style-guide.html
    • Uses post-commit hook to run the linter, you can set it up so it runs the linter before commit
    • All dependencies are defined in vcpkg.json
  • Uses cmake as build system, and provides a top level makefile with recommended command
  • Uses xxd (or xxd.exe windows 64bit port) to convert shader spirv to header files
  • Uses doxygen and sphinx for documentation and autodocs
  • Uses vcpkg for finding the dependencies, it's the recommended set up to retrieve the libraries

If you want to run with debug layers you can add them with the KOMPUTE_ENV_DEBUG_LAYERS parameter as:

export KOMPUTE_ENV_DEBUG_LAYERS="VK_LAYER_LUNARG_api_dump"

Updating documentation

To update the documentation you will need to:

  • Run the gendoxygen target in the build system
  • Run the gensphynx target in the build-system
  • Push to github pages with make push_docs_to_ghpages

Running tests

Running the unit tests has been significantly simplified for contributors.

The tests run on CPU, and can be triggered using the ACT command line interface (https://github.com/nektos/act) - once you install the command line (And start the Docker daemon) you just have to type:

$ act

[Python Tests/python-tests] 🚀  Start image=axsauze/kompute-builder:0.2
[C++ Tests/cpp-tests      ] 🚀  Start image=axsauze/kompute-builder:0.2
[C++ Tests/cpp-tests      ]   🐳  docker run image=axsauze/kompute-builder:0.2 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Python Tests/python-tests]   🐳  docker run image=axsauze/kompute-builder:0.2 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
...

The repository contains unit tests for the C++ and Python code, and can be found under the test/ and python/test folder.

The tests are currently run through the CI using Github Actions. It uses the images found in docker-builders/.

In order to minimise hardware requirements the tests can run without a GPU, directly in the CPU using Swiftshader.

For more information on how the CI and tests are setup, you can go to the CI, Docker and Tests Section in the documentation.

Motivations

This project started after seeing that a lot of new and renowned ML & DL projects like Pytorch, Tensorflow, Alibaba DNN, Tencent NCNN - among others - have either integrated or are looking to integrate the Vulkan SDK to add mobile (and cross-vendor) GPU support.

The Vulkan SDK offers a great low level interface that enables for highly specialized optimizations - however it comes at a cost of highly verbose code which requires 500-2000 lines of code to even begin writing application code. This has resulted in each of these projects having to implement the same baseline to abstract the non-compute related features of the Vulkan SDK. This large amount of non-standardised boiler-plate can result in limited knowledge transfer, higher chance of unique framework implementation bugs being introduced, etc.

We are currently developing Kompute not to hide the Vulkan SDK interface (as it's incredibly well designed) but to augment it with a direct focus on the Vulkan SDK's GPU computing capabilities. This article provides a high level overview of the motivations of Kompute, together with a set of hands on examples that introduce both GPU computing as well as the core Kompute architecture.

Download Details:
Author: KomputeProject
Source Code: https://github.com/KomputeProject/kompute
License: Apache-2.0 License

#python #tensorflow #machine-learning #cpluplus #deep-learning #linux 

Mary Turcotte

Mary Turcotte

1657267200

Rollup Plugin Glsl Optimize: Import GLSL Source Files As Strings

rollup-plugin-glsl-optimize 

Import GLSL source files as strings. Pre-processed, validated and optimized with Khronos Group SPIRV-Tools.

Primary use-case is processing WebGL2 / GLSL ES 300 shaders.

import frag from './shaders/myShader.frag';
console.log(frag);

Features

GLSL Optimizer

For WebGL2 / GLSL ES >= 300

With optimize: true (default) shaders will be compiled to SPIR-V (opengl semantics) and optimized for performance using the Khronos SPIR-V Tools Optimizer before being cross-compiled back to GLSL.

Shader Preprocessor

Shaders are preprocessed and validated using the Khronos Glslang Validator.

Macros are run at build time with support for C-style #include directives: *

#version 300 es
#include "postProcessingShared.glsl"
#include "dofCircle.glsl"

void main() {
  outColor = CircleDof(UVAndScreenPos, Color, ColorCoc);
}

* Via the GL_GOOGLE_include_directive extension. But an #extension directive is not required nor recommended in your final inlined code.

Supports glslify

Specify glslify: true to process shader sources with glslify (a node.js-style module system for GLSL).

And install glslify in your devDependencies with npm i -D glslify

Installation

npm i rollup-plugin-glsl-optimize -D

Khronos tools

This plugin uses the Khronos Glslang Validator, Khronos SPIRV-Tools Optimizer and Khronos SPIRV Cross compiler.

Binaries are automatically installed for:

  • Windows 64bit (MSVC 2017)
  • MacOS x86_64 (clang)
  • Ubuntu Trusty / Debian Buster amd64 (clang)

Paths can be manually provided / overridden with the GLSLANG_VALIDATOR, GLSLANG_OPTIMIZER, GLSLANG_CROSS environment variables.

Usage

// rollup.config.mjs
import {default as glslOptimize} from 'rollup-plugin-glsl-optimize';

export default {
    // ...
    plugins: [
        glslOptimize(),
    ]
};

Shader stages

The following shader stages are supported by the Khronos tools and recognized by file extension:

Shader StageFile Extensions
Vertex.vs, .vert, .vs.glsl, .vert.glsl
Fragment.fs, .frag, .fs.glsl, .frag.glsl
Geometry*.geom, .geom.glsl
Compute*.comp, .comp.glsl
Tess Control*.tesc, .tesc.glsl
Tess Eval*.tese, .tese.glsl

* Unsupported in WebGL2

Options

  • include : PathFilter (default table above) File extensions within rollup to include. Though this option can be reconfigured, shader stage detection still operates based on the table above.
  • exclude : PathFilter (default undefined) File extensions within rollup to exclude.
  • includePaths : string[] (default undefined) Additional search paths for #include directive (source file directory is always searched)

Features

  • optimize : boolean (default true) Optimize via SPIR-V as described in the Optimization section [requires WebGL2 / GLSL ES >= 300]. When disabled simply runs the preprocessor [all supported GLSL versions].
  • compress : boolean (default true) Strip all whitespace in the sources

Debugging

  • sourceMap : boolean (default true) Emit source maps. These contain the final preprocessed/optimized GLSL source (but not stripped of whitespace) to aid debugging.
  • emitLineDirectives : boolean (default false) Emit #line NN "original.file" directives for debugging - useful with #include. Note this requires the GL_GOOGLE_cpp_style_line_directive extension so the shader will fail to run in drivers that lack support.

Preprocessor

  • optimizerPreserveUnusedBindings : boolean (default true) Ensure that the optimizer preserves all declared bindings, even when those bindings are unused.
  • preamble : string (default undefined) Prepended to the shader source (after the #version directive, before the preprocessor runs)

glslify

  • glslify : boolean (default false) Process sources using glslify prior to all preprocessing, validation and optimization.
  • glslifyOptions (default undefined) When glslify enabled, pass these additional options to glslify.compile().

Advanced Options

  • optimizerDebugSkipOptimizer : boolean (default false) When optimize enabled, skip the SPIR-V optimizer - compiles to SPIR-V then cross-compiles back to GLSL immediately.
  • suppressLineExtensionDirective : boolean (default false) When emitLineDirectives enabled, suppress the GL_GOOGLE_cpp_style_line_directive directive.
  • extraValidatorParams, extraOptimizerParams, extraCrossParams : string[] (default undefined) Additional parameters for the Khronos Glslang Validator here, the Khronos SPIR-V Optimizer here, and the Khronos SPIR-V Cross compiler here.
  • glslangValidatorPath, glslangOptimizerPath, glslangCrossPath : string (default undefined) Provide / override binary tool paths.\

It's recommended to instead use the environment variables GLSLANG_VALIDATOR, GLSLANG_OPTIMIZER, GLSLANG_CROSS where needed. They always take precedence if set.

Changelog

Available in CHANGES.md.

Caveats & Known Issues

  • This plugin handles glsl and glslify by itself. Use with conflicting plugins (e.g. rollup-plugin-glsl, rollup-plugin-glslify) will cause unpredictable results.
  • Optimizer: lowp precision qualifier - emitted as mediump
    SPIR-V has a single RelaxedPrecision decoration for 16-32bit precision. However most implementations actually treat mediump and lowp equivalently, hence the lack of need for it in SPIR-V.

License

Released under the MIT license.
Strip whitespace function adapted from code by Vincent Wochnik (rollup-plugin-glsl).

Khronos tool binaries (built by the upstream projects) are distributed and installed with this plugin under the terms of the Apache License Version 2.0. See the corresponding LICENSE files installed in the bin folder and the binary releases.

Author: docd27
Source code: https://github.com/docd27/rollup-plugin-glsl-optimize
License: MIT license

#javascript #Rollup 

Gordon Taylor

Gordon Taylor

1657873620

Browserify/wzrd.in: Browserify As A Service

browserify-as-a-service

What just happened?

Well, in this case, since someone has visited this link before you, the file was cached with leveldb. But if you were to try and grab a bundle that nobody else has tried to grab before, what would happen is this:

  • The module gets pulled down from npm and installed
  • The module gets browserified as a standalone bundle
  • The module gets sent to you, piping hot
  • The module gets cached so that you don't have to wait later on

API

There are a few API endpoints:

GET /bundle/:module

Get the latest version of :module.

GET /bundle/:module@:version

Get a version of :module which satisfies the given :version semver range. Defaults to latest.

GET /debug-bundle/:module

GET /debug-bundle/:module@:version

The same as the prior two, except with --debug passed to browserify.

GET /standalone/:module

GET /standalone/:module@:version

In this case, --standalone is passed to browserify.

GET /debug-standalone/:module

GET /debug-standalone/:module@:version

Both --debug and --standalone are passed to browserify!

POST /multi

POST a body that looks something like this:

{
  "options": {
    "debug": true
  },
  "dependencies": {
    "concat-stream": "0.1.x",
    "hyperstream": "0.2.x"
  }
}

"options" is where you get to set "debug", "standalone", and "fullPaths". Usually, in this case, you'll probably only really care about debug. If you don't define "options", it will default to { "debug": false, "standalone": false, "fullPaths": false }.

What you get in return looks something like this:

HTTP/1.1 200 OK
X-Powered-By: Express
Location: /multi/48GOmL0XvnRZn32bkpz75A==
content-type: application/json
Date: Sat, 22 Jun 2013 22:36:32 GMT
Connection: keep-alive
Transfer-Encoding: chunked

{
  "concat-stream": {
    "package": /* the concat-stream package.json */,
    "bundle": /* the concat-stream bundle */
  },
  "hyperstream": {
    "package": /* the hyperstream package.json */,
    "bundle": /* the hyperstream bundle */
  }
}

The bundle gets permanently cached at /multi/48GOmL0XvnRZn32bkpz75A== for future GETs.

GET /multi/:existing-bundle

If you saved the Location url from the POST earlier, you can just GET it instead of POSTing again.

GET /status/:module

GET /status/:module@:version

Get information on the build status of a module. Returns build information for all versions which satisfy the given semver (or latest in the event of a missing semver).

Blobs generally look something like this:

HTTP/1.1 200 OK
X-Powered-By: Express
Access-Control-Allow-Origin: *
Content-Type: application/json; charset=utf-8
Content-Length: 109
ETag: "-9450086"
Date: Sun, 26 Jan 2014 08:05:59 GMT
Connection: keep-alive

{
  "module": "concat-stream",
  "builds": {
    "1.4.1": {
      "ok": true
    }
  }
}

The "module" and "builds" fields should both exist. Keys for "builds" are the versions. Properties:

  • "ok": Whether the package has last built or not
  • "error": If the package was built insuccessfully ("ok" is false), this property will contain information about the error

Versions which have not been built will not be keyed onto "builds".

Heroku Installation

browserify-cdn is ready to run on Heroku:

heroku create my-browserify-cdn
git push heroku master
heroku ps:scale web=1

Docker Installation

You can build and run an image doing the following:

docker build -t "wzrd.in" /path/to/wzrd.in
docker run -p 8080:8080 wzrd.in

Keep in mind that a new deploy will wipe the cache.

Places

Quick Start

Try visiting this link:

/standalone/concat-stream@latest

Also, wzrd.in has a nice url generating form.

Author: Browserify
Source Code: https://github.com/browserify/wzrd.in 
License: MIT license

#javascript #service #browserify