TSdx: Zero-config CLI for TypeScript Package Development

TSdx

Despite all the recent hype, setting up a new TypeScript (x React) library can be tough. Between Rollup, Jest, tsconfig, Yarn resolutions, ESLint, and getting VSCode to play nicely....there is just a whole lot of stuff to do (and things to screw up). TSDX is a zero-config CLI that helps you develop, test, and publish modern TypeScript packages with ease--so you can focus on your awesome new library and not waste another afternoon on the configuration.

Features

TSDX comes with the "battery-pack included" and is part of a complete TypeScript breakfast:

• Bundles your code with Rollup and outputs multiple module formats (CJS & ESM by default, and also UMD if you want) plus development and production builds
• Comes with treeshaking, ready-to-rock lodash optimizations, and minification/compression
• Live reload / watch-mode
• Works with React
• Human readable error messages (and in VSCode-friendly format)
• Bundle size snapshots
• Opt-in to extract invariant error codes
• Jest test runner setup with sensible defaults via tsdx test
• ESLint with Prettier setup with sensible defaults via tsdx lint
• Zero-config, single dependency
• Escape hatches for customization via .babelrc.js, jest.config.js, .eslintrc.js, and tsdx.config.js

Quick Start

npx tsdx create mylib
cd mylib
yarn start

That's it. You don't need to worry about setting up TypeScript or Rollup or Jest or other plumbing. Just start editing src/index.ts and go!

Below is a list of commands you will probably find useful:

npm start or yarn start

Runs the project in development/watch mode. Your project will be rebuilt upon changes. TSDX has a special logger for your convenience. Error messages are pretty printed and formatted for compatibility VS Code's Problems tab.

Your library will be rebuilt if you make edits.

npm run build or yarn build

Bundles the package to the dist folder. The package is optimized and bundled with Rollup into multiple formats (CommonJS, UMD, and ES Module).

npm test or yarn test

Runs your tests using Jest.

npm run lint or yarn lint

Runs Eslint with Prettier on .ts and .tsx files. If you want to customize eslint you can add an eslint block to your package.json, or you can run yarn lint --write-file and edit the generated .eslintrc.js file.

prepare script

Bundles and packages to the dist folder. Runs automatically when you run either npm publish or yarn publish. The prepare script will run the equivalent of npm run build or yarn build. It will also be run if your module is installed as a git dependency (ie: "mymodule": "github:myuser/mymodule#some-branch") so it can be depended on without checking the transpiled code into git.

Optimizations

Aside from just bundling your module into different formats, TSDX comes with some optimizations for your convenience. They yield objectively better code and smaller bundle sizes.

After TSDX compiles your code with TypeScript, it processes your code with 3 Babel plugins:

• babel-plugin-annotate-pure-calls: Injects for #__PURE annotations to enable treeshaking
• babel-plugin-dev-expressions: A mirror of Facebook's dev-expression Babel plugin. It reduces or eliminates development checks from production code
• babel-plugin-rename-import: Used to rewrite any lodash imports

Development-only Expressions + Treeshaking

babel-plugin-annotate-pure-calls + babel-plugin-dev-expressions work together to fully eliminate dead code (aka treeshake) development checks from your production code. Let's look at an example to see how it works.

Imagine our source code is just this:

// ./src/index.ts
export const sum = (a: number, b: number) => {
  if (process.env.NODE_ENV !== 'production') {
    console.log('Helpful dev-only error message');
  }
  return a + b;
};

tsdx build will output an ES module file and 3 CommonJS files (dev, prod, and an entry file). If you want to specify a UMD build, you can do that as well. For brevity, let's examine the CommonJS output (comments added for emphasis):

// Entry File
// ./dist/index.js
'use strict';

// This determines which build to use based on the `NODE_ENV` of your end user.
if (process.env.NODE_ENV === 'production') {
  module.exports = require('./mylib.cjs.production.js');
} else {
  module.exports = require('./mylib.development.cjs');
}

// CommonJS Development Build
// ./dist/mylib.development.cjs
'use strict';

const sum = (a, b) => {
  {
    console.log('Helpful dev-only error message');
  }

  return a + b;
};

exports.sum = sum;
//# sourceMappingURL=mylib.development.cjs.map

// CommonJS Production Build
// ./dist/mylib.cjs.production.js
'use strict';
exports.sum = (s, t) => s + t;
//# sourceMappingURL=test-react-tsdx.cjs.production.js.map

AS you can see, TSDX stripped out the development check from the production code. This allows you to safely add development-only behavior (like more useful error messages) without any production bundle size impact.

For ESM build, it's up to end-user to build environment specific build with NODE_ENV replace (done by Webpack 4 automatically).

Rollup Treeshaking

TSDX's rollup config removes getters and setters on objects so that property access has no side effects. Don't do it.

Advanced babel-plugin-dev-expressions

TSDX will use babel-plugin-dev-expressions to make the following replacements before treeshaking.

__DEV__

Replaces

if (__DEV__) {
  console.log('foo');
}

with

if (process.env.NODE_ENV !== 'production') {
  console.log('foo');
}

IMPORTANT: To use __DEV__ in TypeScript, you need to add declare var __DEV__: boolean somewhere in your project's type path (e.g. ./types/index.d.ts).

// ./types/index.d.ts
declare var __DEV__: boolean;

Note: The dev-expression transform does not run when NODE_ENV is test. As such, if you use __DEV__, you will need to define it as a global constant in your test environment.

invariant

Replaces

invariant(condition, 'error message here');

with

if (!condition) {
  if ('production' !== process.env.NODE_ENV) {
    invariant(false, 'error message here');
  } else {
    invariant(false);
  }
}

Note: TSDX doesn't supply an invariant function for you, you need to import one yourself. We recommend https://github.com/alexreardon/tiny-invariant.

To extract and minify invariant error codes in production into a static codes.json file, specify the --extractErrors flag in command line. For more details see Error extraction docs.

warning

Replaces

warning(condition, 'dev warning here');

with

if ('production' !== process.env.NODE_ENV) {
  warning(condition, 'dev warning here');
}

Note: TSDX doesn't supply a warning function for you, you need to import one yourself. We recommend https://github.com/alexreardon/tiny-warning.

Using lodash

If you want to use a lodash function in your package, TSDX will help you do it the right way so that your library does not get fat shamed on Twitter. However, before you continue, seriously consider rolling whatever function you are about to use on your own. Anyways, here is how to do it right.

First, install lodash and lodash-es as dependencies

yarn add lodash lodash-es

Now install @types/lodash to your development dependencies.

yarn add @types/lodash --dev

Import your lodash method however you want, TSDX will optimize it like so.

// ./src/index.ts
import kebabCase from 'lodash/kebabCase';

export const KebabLogger = (msg: string) => {
  console.log(kebabCase(msg));
};

For brevity let's look at the ES module output.

import o from"lodash-es/kebabCase";const e=e=>{console.log(o(e))};export{e as KebabLogger};
//# sourceMappingURL=test-react-tsdx.esm.production.js.map

TSDX will rewrite your import kebabCase from 'lodash/kebabCase' to import o from 'lodash-es/kebabCase'. This allows your library to be treeshakable to end consumers while allowing to you to use @types/lodash for free.

Note: TSDX will also transform destructured imports. For example, import { kebabCase } from 'lodash' would have also been transformed to `import o from "lodash-es/kebabCase".

Error extraction

After running --extractErrors, you will have a ./errors/codes.json file with all your extracted invariant error codes. This process scans your production code and swaps out your invariant error message strings for a corresponding error code (just like React!). This extraction only works if your error checking/warning is done by a function called invariant.

Note: We don't provide this function for you, it is up to you how you want it to behave. For example, you can use either tiny-invariant or tiny-warning, but you must then import the module as a variable called invariant and it should have the same type signature.

⚠️Don't forget: you will need to host the decoder somewhere. Once you have a URL, look at ./errors/ErrorProd.js and replace the reactjs.org URL with yours.

Known issue: our transformErrorMessages babel plugin currently doesn't have sourcemap support, so you will see "Sourcemap is likely to be incorrect" warnings. We would love your help on this.

TODO: Simple guide to host error codes to be completed

Customization

Rollup

❗⚠️❗ Warning: 
These modifications will override the default behavior and configuration of TSDX. As such they can invalidate internal guarantees and assumptions. These types of changes can break internal behavior and can be very fragile against updates. Use with discretion!

TSDX uses Rollup under the hood. The defaults are solid for most packages (Formik uses the defaults!). However, if you do wish to alter the rollup configuration, you can do so by creating a file called tsdx.config.js at the root of your project like so:

// Not transpiled with TypeScript or Babel, so use plain Es6/Node.js!
module.exports = {
  // This function will run for each entry/format/env combination
  rollup(config, options) {
    return config; // always return a config.
  },
};

The options object contains the following:

export interface TsdxOptions {
  // path to file
  input: string;
  // Name of package
  name: string;
  // JS target
  target: 'node' | 'browser';
  // Module format
  format: 'cjs' | 'umd' | 'esm' | 'system';
  // Environment
  env: 'development' | 'production';
  // Path to tsconfig file
  tsconfig?: string;
  // Is error extraction running?
  extractErrors?: boolean;
  // Is minifying?
  minify?: boolean;
  // Is this the very first rollup config (and thus should one-off metadata be extracted)?
  writeMeta?: boolean;
  // Only transpile, do not type check (makes compilation faster)
  transpileOnly?: boolean;
}

Example: Adding Postcss

const postcss = require('rollup-plugin-postcss');
const autoprefixer = require('autoprefixer');
const cssnano = require('cssnano');

module.exports = {
  rollup(config, options) {
    config.plugins.push(
      postcss({
        plugins: [
          autoprefixer(),
          cssnano({
            preset: 'default',
          }),
        ],
        inject: false,
        // only write out CSS for the first bundle (avoids pointless extra files):
        extract: !!options.writeMeta,
      })
    );
    return config;
  },
};

Babel

You can add your own .babelrc to the root of your project and TSDX will merge it with its own Babel transforms (which are mostly for optimization), putting any new presets and plugins at the end of its list.

Jest

You can add your own jest.config.js to the root of your project and TSDX will shallow merge it with its own Jest config.

ESLint

You can add your own .eslintrc.js to the root of your project and TSDX will deep merge it with its own ESLint config.

patch-package

If you still need more customizations, we recommend using patch-package so you don't need to fork. Keep in mind that these types of changes may be quite fragile against version updates.

Inspiration

TSDX was originally ripped out of Formik's build tooling. TSDX has several similarities to @developit/microbundle, but that is because Formik's Rollup configuration and Microbundle's internals had converged around similar plugins.

Comparison with Microbundle

Some key differences include:

• TSDX includes out-of-the-box test running via Jest
• TSDX includes out-of-the-box linting and formatting via ESLint and Prettier
• TSDX includes a bootstrap command with a few package templates
• TSDX allows for some lightweight customization
• TSDX is TypeScript focused, but also supports plain JavaScript
• TSDX outputs distinct development and production builds (like React does) for CJS and UMD builds. This means you can include rich error messages and other dev-friendly goodies without sacrificing final bundle size.

API Reference

tsdx watch

Description
  Rebuilds on any change

Usage
  $ tsdx watch [options]

Options
  -i, --entry           Entry module
  --target              Specify your target environment  (default web)
  --name                Specify name exposed in UMD builds
  --format              Specify module format(s)  (default cjs,esm)
  --tsconfig            Specify your custom tsconfig path (default <root-folder>/tsconfig.json)
  --verbose             Keep outdated console output in watch mode instead of clearing the screen
  --onFirstSuccess      Run a command on the first successful build
  --onSuccess           Run a command on a successful build
  --onFailure           Run a command on a failed build
  --noClean             Don't clean the dist folder
  --transpileOnly       Skip type checking
  -h, --help            Displays this message

Examples
  $ tsdx watch --entry src/foo.tsx
  $ tsdx watch --target node
  $ tsdx watch --name Foo
  $ tsdx watch --format cjs,esm,umd
  $ tsdx watch --tsconfig ./tsconfig.foo.json
  $ tsdx watch --noClean
  $ tsdx watch --onFirstSuccess "echo The first successful build!"
  $ tsdx watch --onSuccess "echo Successful build!"
  $ tsdx watch --onFailure "echo The build failed!"
  $ tsdx watch --transpileOnly

tsdx build

Description
  Build your project once and exit

Usage
  $ tsdx build [options]

Options
  -i, --entry           Entry module
  --target              Specify your target environment  (default web)
  --name                Specify name exposed in UMD builds
  --format              Specify module format(s)  (default cjs,esm)
  --extractErrors       Opt-in to extracting invariant error codes
  --tsconfig            Specify your custom tsconfig path (default <root-folder>/tsconfig.json)
  --transpileOnly       Skip type checking
  -h, --help            Displays this message

Examples
  $ tsdx build --entry src/foo.tsx
  $ tsdx build --target node
  $ tsdx build --name Foo
  $ tsdx build --format cjs,esm,umd
  $ tsdx build --extractErrors
  $ tsdx build --tsconfig ./tsconfig.foo.json
  $ tsdx build --transpileOnly

tsdx test

This runs Jest, forwarding all CLI flags to it. See https://jestjs.io for options. For example, if you would like to run in watch mode, you can run tsdx test --watch. So you could set up your package.json scripts like:

{
  "scripts": {
    "test": "tsdx test",
    "test:watch": "tsdx test --watch",
    "test:coverage": "tsdx test --coverage"
  }
}

tsdx lint

Description
  Run eslint with Prettier

Usage
  $ tsdx lint [options]

Options
  --fix               Fixes fixable errors and warnings
  --ignore-pattern    Ignore a pattern
  --max-warnings      Exits with non-zero error code if number of warnings exceed this number  (default Infinity)
  --write-file        Write the config file locally
  --report-file       Write JSON report to file locally
  -h, --help          Displays this message

Examples
  $ tsdx lint src
  $ tsdx lint src --fix
  $ tsdx lint src test --ignore-pattern test/foo.ts
  $ tsdx lint src test --max-warnings 10
  $ tsdx lint src --write-file
  $ tsdx lint src --report-file report.json

Contributing

Please see the Contributing Guidelines.

Download Details:

Author: jaredpalmer
Source Code: https://github.com/jaredpalmer/tsdx 
License: MIT license

#typescript #react #npm #rollup 

What is GEEK

Buddha Community

How long does it take to develop/build an app?

With more of us using smartphones, the popularity of mobile applications has exploded. In the digital era, the number of people looking for products and services online is growing rapidly. Smartphone owners look for mobile applications that give them quick access to companies’ products and services. As a result, mobile apps provide customers with a lot of benefits in just one device.

Likewise, companies use mobile apps to increase customer loyalty and improve their services. Mobile Developers are in high demand as companies use apps not only to create brand awareness but also to gather information. For that reason, mobile apps are used as tools to collect valuable data from customers to help companies improve their offer.

There are many types of mobile applications, each with its own advantages. For example, native apps perform better, while web apps don’t need to be customized for the platform or operating system (OS). Likewise, hybrid apps provide users with comfortable user experience. However, you may be wondering how long it takes to develop an app.

To give you an idea of how long the app development process takes, here’s a short guide.

App Idea & Research

_Average time spent: two to five weeks _

This is the initial stage and a crucial step in setting the project in the right direction. In this stage, you brainstorm ideas and select the best one. Apart from that, you’ll need to do some research to see if your idea is viable. Remember that coming up with an idea is easy; the hard part is to make it a reality.

All your ideas may seem viable, but you still have to run some tests to keep it as real as possible. For that reason, when Web Developers are building a web app, they analyze the available ideas to see which one is the best match for the targeted audience.

Targeting the right audience is crucial when you are developing an app. It saves time when shaping the app in the right direction as you have a clear set of objectives. Likewise, analyzing how the app affects the market is essential. During the research process, App Developers must gather information about potential competitors and threats. This helps the app owners develop strategies to tackle difficulties that come up after the launch.

The research process can take several weeks, but it determines how successful your app can be. For that reason, you must take your time to know all the weaknesses and strengths of the competitors, possible app strategies, and targeted audience.

The outcomes of this stage are app prototypes and the minimum feasible product.

#android app #frontend #ios app #minimum viable product (mvp) #mobile app development #web development #android app development #app development #app development for ios and android #app development process #ios and android app development #ios app development #stages in app development

Developer Career Path: To Become a Team Lead or Stay a Developer?

For a developer, becoming a team leader can be a trap or open up opportunities for creating software. Two years ago, when I was a developer, I was thinking, “I want to be a team leader. It’s so cool, he’s in charge of everything and gets more money. It’s the next step after a senior.” Back then, no one could tell me how wrong I was. I had to find it out myself.

I Got to Be a Team Leader — Twice

I’m naturally very organized. Whatever I do, I try to put things in order, create systems and processes. So I’ve always been inclined to take on more responsibilities than just coding. My first startup job, let’s call it T, was complete chaos in terms of development processes.

Now I probably wouldn’t work in a place like that, but at the time, I enjoyed the vibe. Just imagine it — numerous clients and a team leader who set tasks to the developers in person (and often privately). We would often miss deadlines and had to work late. Once, my boss called and asked me to come back to work at 8 p.m. to finish one feature — all because the deadline was “the next morning.” But at T, we were a family.

We also did everything ourselves — or at least tried to. I’ll never forget how I had to install Ubuntu on a rack server that we got from one of our investors. When I would turn it on, it sounded like a helicopter taking off!

At T, I became a CTO and managed a team of 10 people. So it was my first experience as a team leader.

Then I came to work at D — as a developer. And it was so different in every way when it came to processes.

They employed classic Scrum with sprints, burndown charts, demos, story points, planning, and backlog grooming. I was amazed by the quality of processes, but at first, I was just coding and minding my own business. Then I became friends with the Scrum master. I would ask him lots of questions, and he would willingly answer them and recommend good books.

My favorite was Scrum and XP from the Trenches by Henrik Kniberg. The process at D was based on its methods. As a result, both managers and sellers knew when to expect the result.

Then I joined Skyeng, also as a developer. Unlike my other jobs, it excels at continuous integration with features shipped every day. Within my team, we used a Kanban-like method.

We were also lucky to have our team leader, Petya. At our F2F meetings, we could discuss anything, from missing deadlines to setting up a task tracker. Sometimes I would just give feedback or he would give me advice.

That’s how Petya got to know I’d had some management experience at T and learned Scrum at D.

So one day, he offered me to host a stand-up.

#software-development #developer #dev-team-leadership #agile-software-development #web-development #mobile-app-development #ios-development #android-development

Offshore Software Development - Best Practices

With the rise of globalization and the worldwide lockdown due to the pandemic, most of the work has been done by remote working processes and professionals from their homes. This lockdown has proved the efficiency of remote development and enhanced the trust in offshore software development industry.

To make the most out of the benefits of offshore software development, you should understand the crucial factors that affect offshore development. This is why you should read this guide for the best practices when hiring an offshore software development company. Despite the size and the industry of the business, offshore software development is not beneficial for every entrepreneur in many aspects to make the optimum use of talents in technology across the globe.

Here are some of the top reasons why offshore development is beneficial for your business.

• Offshore development teams can work on flexible timing to provide you with the best possible software development practices.
• Get access to the talents across the world from your home to develop the top of the line software with the help of offshore development companies.
• Assured high quality and next-generation technology expertise with duly NDA signed with respect to the priorities of the business.
• With flexible recruitment models, you can hire the freelance developers, remote development team, or an entire offshore development company with respect to the size of your business.
• Build high-end software applications from one corner of the world by hiring software developers across the world.
• Get immediate access to the best resources without hiring them on a permanent basis.

To avail of all these benefits, you should have clear goals, a list of requirements, and features that are mandatory for your software product.

Here are a few tips to help you find the best offshore software development company. Build a top-notch software application by following the listed best practices.

#web development #how to start offshore software development company #offshore meaning #offshore software development best practices #offshore software development company #offshore software development company in india #offshore software development cost #offshore software development statistics #outsource software development

Perl Critic: The Leading Static analyzer for Perl

BUILD STATUS

• CPAN Testers

NAME

Perl::Critic - Critique Perl source code for best-practices.

SYNOPSIS

use Perl::Critic;
my $file = shift;
my $critic = Perl::Critic->new();
my @violations = $critic->critique($file);
print @violations;

DESCRIPTION

Perl::Critic is an extensible framework for creating and applying coding standards to Perl source code. Essentially, it is a static source code analysis engine. Perl::Critic is distributed with a number of Perl::Critic::Policy modules that attempt to enforce various coding guidelines. Most Policy modules are based on Damian Conway's book Perl Best Practices. However, Perl::Critic is not limited to PBP and will even support Policies that contradict Conway. You can enable, disable, and customize those Polices through the Perl::Critic interface. You can also create new Policy modules that suit your own tastes.

For a command-line interface to Perl::Critic, see the documentation for perlcritic. If you want to integrate Perl::Critic with your build process, Test::Perl::Critic provides an interface that is suitable for test programs. Also, Test::Perl::Critic::Progressive is useful for gradually applying coding standards to legacy code. For the ultimate convenience (at the expense of some flexibility) see the criticism pragma.

If you'd like to try Perl::Critic without installing anything, there is a web-service available at http://perlcritic.com. The web-service does not yet support all the configuration features that are available in the native Perl::Critic API, but it should give you a good idea of what it does.

Also, ActivePerl includes a very slick graphical interface to Perl-Critic called perlcritic-gui. You can get a free community edition of ActivePerl from http://www.activestate.com.

PREREQUISITES

Perl::Critic runs on Perl back to Perl 5.6.1. It relies on the PPI module to do the heavy work of parsing Perl.

INTERFACE SUPPORT

The Perl::Critic module is considered to be a public class. Any changes to its interface will go through a deprecation cycle.

CONSTRUCTOR

new( [ -profile => $FILE, -severity => $N, -theme => $string, -include => \@PATTERNS, -exclude => \@PATTERNS, -top => $N, -only => $B, -profile-strictness => $PROFILE_STRICTNESS_{WARN|FATAL|QUIET}, -force => $B, -verbose => $N ], -color => $B, -pager => $string, -allow-unsafe => $B, -criticism-fatal => $B)

new()

Returns a reference to a new Perl::Critic object. Most arguments are just passed directly into Perl::Critic::Config, but I have described them here as well. The default value for all arguments can be defined in your .perlcriticrc file. See the "CONFIGURATION" section for more information about that. All arguments are optional key-value pairs as follows:

-profile is a path to a configuration file. If $FILE is not defined, Perl::Critic::Config attempts to find a .perlcriticrc configuration file in the current directory, and then in your home directory. Alternatively, you can set the PERLCRITIC environment variable to point to a file in another location. If a configuration file can't be found, or if $FILE is an empty string, then all Policies will be loaded with their default configuration. See "CONFIGURATION" for more information.

-severity is the minimum severity level. Only Policy modules that have a severity greater than $N will be applied. Severity values are integers ranging from 1 (least severe violations) to 5 (most severe violations). The default is 5. For a given -profile, decreasing the -severity will usually reveal more Policy violations. You can set the default value for this option in your .perlcriticrc file. Users can redefine the severity level for any Policy in their .perlcriticrc file. See "CONFIGURATION" for more information.

If it is difficult for you to remember whether severity "5" is the most or least restrictive level, then you can use one of these named values:

  SEVERITY NAME   ...is equivalent to...   SEVERITY NUMBER
  --------------------------------------------------------
  -severity => 'gentle'                     -severity => 5
  -severity => 'stern'                      -severity => 4
  -severity => 'harsh'                      -severity => 3
  -severity => 'cruel'                      -severity => 2
  -severity => 'brutal'                     -severity => 1

The names reflect how severely the code is criticized: a gentle criticism reports only the most severe violations, and so on down to a brutal criticism which reports even the most minor violations.

-theme is special expression that determines which Policies to apply based on their respective themes. For example, the following would load only Policies that have a 'bugs' AND 'pbp' theme:

  my $critic = Perl::Critic->new( -theme => 'bugs && pbp' );

Unless the -severity option is explicitly given, setting -theme silently causes the -severity to be set to 1. You can set the default value for this option in your .perlcriticrc file. See the "POLICY THEMES" section for more information about themes.

-include is a reference to a list of string @PATTERNS. Policy modules that match at least one m/$PATTERN/ixms will always be loaded, irrespective of all other settings. For example:

  my $critic = Perl::Critic->new(-include => ['layout'], -severity => 4);

This would cause Perl::Critic to apply all the CodeLayout::* Policy modules even though they have a severity level that is less than 4. You can set the default value for this option in your .perlcriticrc file. You can also use -include in conjunction with the -exclude option. Note that -exclude takes precedence over -include when a Policy matches both patterns.

-exclude is a reference to a list of string @PATTERNS. Policy modules that match at least one m/$PATTERN/ixms will not be loaded, irrespective of all other settings. For example:

  my $critic = Perl::Critic->new(-exclude => ['strict'], -severity => 1);

This would cause Perl::Critic to not apply the RequireUseStrict and ProhibitNoStrict Policy modules even though they have a severity level that is greater than 1. You can set the default value for this option in your .perlcriticrc file. You can also use -exclude in conjunction with the -include option. Note that -exclude takes precedence over -include when a Policy matches both patterns.

-single-policy is a string PATTERN. Only one policy that matches m/$PATTERN/ixms will be used. Policies that do not match will be excluded. This option has precedence over the -severity, -theme, -include, -exclude, and -only options. You can set the default value for this option in your .perlcriticrc file.

-top is the maximum number of Violations to return when ranked by their severity levels. This must be a positive integer. Violations are still returned in the order that they occur within the file. Unless the -severity option is explicitly given, setting -top silently causes the -severity to be set to 1. You can set the default value for this option in your .perlcriticrc file.

-only is a boolean value. If set to a true value, Perl::Critic will only choose from Policies that are mentioned in the user's profile. If set to a false value (which is the default), then Perl::Critic chooses from all the Policies that it finds at your site. You can set the default value for this option in your .perlcriticrc file.

-profile-strictness is an enumerated value, one of "$PROFILE_STRICTNESS_WARN" in Perl::Critic::Utils::Constants (the default), "$PROFILE_STRICTNESS_FATAL" in Perl::Critic::Utils::Constants, and "$PROFILE_STRICTNESS_QUIET" in Perl::Critic::Utils::Constants. If set to "$PROFILE_STRICTNESS_FATAL" in Perl::Critic::Utils::Constants, Perl::Critic will make certain warnings about problems found in a .perlcriticrc or file specified via the -profile option fatal. For example, Perl::Critic normally only warns about profiles referring to non-existent Policies, but this value makes this situation fatal. Correspondingly, "$PROFILE_STRICTNESS_QUIET" in Perl::Critic::Utils::Constants makes Perl::Critic shut up about these things.

-force is a boolean value that controls whether Perl::Critic observes the magical "## no critic" annotations in your code. If set to a true value, Perl::Critic will analyze all code. If set to a false value (which is the default) Perl::Critic will ignore code that is tagged with these annotations. See "BENDING THE RULES" for more information. You can set the default value for this option in your .perlcriticrc file.

-verbose can be a positive integer (from 1 to 11), or a literal format specification. See Perl::Critic::Violation for an explanation of format specifications. You can set the default value for this option in your .perlcriticrc file.

-unsafe directs Perl::Critic to allow the use of Policies that are marked as "unsafe" by the author. Such policies may compile untrusted code or do other nefarious things.

-color and -pager are not used by Perl::Critic but is provided for the benefit of perlcritic.

-criticism-fatal is not used by Perl::Critic but is provided for the benefit of criticism.

-color-severity-highest, -color-severity-high, -color-severity- medium, -color-severity-low, and -color-severity-lowest are not used by Perl::Critic, but are provided for the benefit of perlcritic. Each is set to the Term::ANSIColor color specification to be used to display violations of the corresponding severity.

-files-with-violations and -files-without-violations are not used by Perl::Critic, but are provided for the benefit of perlcritic, to cause only the relevant filenames to be displayed.

METHODS

critique( $source_code )

Runs the $source_code through the Perl::Critic engine using all the Policies that have been loaded into this engine. If $source_code is a scalar reference, then it is treated as a string of actual Perl code. If $source_code is a reference to an instance of PPI::Document, then that instance is used directly. Otherwise, it is treated as a path to a local file containing Perl code. This method returns a list of Perl::Critic::Violation objects for each violation of the loaded Policies. The list is sorted in the order that the Violations appear in the code. If there are no violations, this method returns an empty list.

add_policy( -policy => $policy_name, -params => \%param_hash )

Creates a Policy object and loads it into this Critic. If the object cannot be instantiated, it will throw a fatal exception. Otherwise, it returns a reference to this Critic.

-policy is the name of a Perl::Critic::Policy subclass module. The 'Perl::Critic::Policy' portion of the name can be omitted for brevity. This argument is required.

-params is an optional reference to a hash of Policy parameters. The contents of this hash reference will be passed into to the constructor of the Policy module. See the documentation in the relevant Policy module for a description of the arguments it supports.

policies()

Returns a list containing references to all the Policy objects that have been loaded into this engine. Objects will be in the order that they were loaded.

config()

Returns the Perl::Critic::Config object that was created for or given to this Critic.

statistics()

Returns the Perl::Critic::Statistics object that was created for this Critic. The Statistics object accumulates data for all files that are analyzed by this Critic.

FUNCTIONAL INTERFACE

For those folks who prefer to have a functional interface, The critique method can be exported on request and called as a static function. If the first argument is a hashref, its contents are used to construct a new Perl::Critic object internally. The keys of that hash should be the same as those supported by the Perl::Critic::new() method. Here are some examples:

use Perl::Critic qw(critique);

# Use default parameters...
@violations = critique( $some_file );

# Use custom parameters...
@violations = critique( {-severity => 2}, $some_file );

# As a one-liner
%> perl -MPerl::Critic=critique -e 'print critique(shift)' some_file.pm

None of the other object-methods are currently supported as static functions. Sorry.

CONFIGURATION

Most of the settings for Perl::Critic and each of the Policy modules can be controlled by a configuration file. The default configuration file is called .perlcriticrc. Perl::Critic will look for this file in the current directory first, and then in your home directory. Alternatively, you can set the PERLCRITIC environment variable to explicitly point to a different file in another location. If none of these files exist, and the -profile option is not given to the constructor, then all the modules that are found in the Perl::Critic::Policy namespace will be loaded with their default configuration.

The format of the configuration file is a series of INI-style blocks that contain key-value pairs separated by '='. Comments should start with '#' and can be placed on a separate line or after the name-value pairs if you desire.

Default settings for Perl::Critic itself can be set before the first named block. For example, putting any or all of these at the top of your configuration file will set the default value for the corresponding constructor argument.

severity  = 3                                     #Integer or named level
only      = 1                                     #Zero or One
force     = 0                                     #Zero or One
verbose   = 4                                     #Integer or format spec
top       = 50                                    #A positive integer
theme     = (pbp || security) && bugs             #A theme expression
include   = NamingConventions ClassHierarchies    #Space-delimited list
exclude   = Variables  Modules::RequirePackage    #Space-delimited list
criticism-fatal = 1                               #Zero or One
color     = 1                                     #Zero or One
allow-unsafe = 1                                  #Zero or One
pager     = less                                  #pager to pipe output to

The remainder of the configuration file is a series of blocks like this:

[Perl::Critic::Policy::Category::PolicyName]
severity = 1
set_themes = foo bar
add_themes = baz
maximum_violations_per_document = 57
arg1 = value1
arg2 = value2

Perl::Critic::Policy::Category::PolicyName is the full name of a module that implements the policy. The Policy modules distributed with Perl::Critic have been grouped into categories according to the table of contents in Damian Conway's book Perl Best Practices. For brevity, you can omit the 'Perl::Critic::Policy' part of the module name.

severity is the level of importance you wish to assign to the Policy. All Policy modules are defined with a default severity value ranging from 1 (least severe) to 5 (most severe). However, you may disagree with the default severity and choose to give it a higher or lower severity, based on your own coding philosophy. You can set the severity to an integer from 1 to 5, or use one of the equivalent names:

SEVERITY NAME ...is equivalent to... SEVERITY NUMBER
----------------------------------------------------
gentle                                             5
stern                                              4
harsh                                              3
cruel                                              2
brutal                                             1

The names reflect how severely the code is criticized: a gentle criticism reports only the most severe violations, and so on down to a brutal criticism which reports even the most minor violations.

set_themes sets the theme for the Policy and overrides its default theme. The argument is a string of one or more whitespace-delimited alphanumeric words. Themes are case-insensitive. See "POLICY THEMES" for more information.

add_themes appends to the default themes for this Policy. The argument is a string of one or more whitespace-delimited words. Themes are case- insensitive. See "POLICY THEMES" for more information.

maximum_violations_per_document limits the number of Violations the Policy will return for a given document. Some Policies have a default limit; see the documentation for the individual Policies to see whether there is one. To force a Policy to not have a limit, specify "no_limit" or the empty string for the value of this parameter.

The remaining key-value pairs are configuration parameters that will be passed into the constructor for that Policy. The constructors for most Policy objects do not support arguments, and those that do should have reasonable defaults. See the documentation on the appropriate Policy module for more details.

Instead of redefining the severity for a given Policy, you can completely disable a Policy by prepending a '-' to the name of the module in your configuration file. In this manner, the Policy will never be loaded, regardless of the -severity given to the Perl::Critic constructor.

A simple configuration might look like this:

#--------------------------------------------------------------
# I think these are really important, so always load them

[TestingAndDebugging::RequireUseStrict]
severity = 5

[TestingAndDebugging::RequireUseWarnings]
severity = 5

#--------------------------------------------------------------
# I think these are less important, so only load when asked

[Variables::ProhibitPackageVars]
severity = 2

[ControlStructures::ProhibitPostfixControls]
allow = if unless  # My custom configuration
severity = cruel   # Same as "severity = 2"

#--------------------------------------------------------------
# Give these policies a custom theme.  I can activate just
# these policies by saying `perlcritic -theme larry`

[Modules::RequireFilenameMatchesPackage]
add_themes = larry

[TestingAndDebugging::RequireTestLables]
add_themes = larry curly moe

#--------------------------------------------------------------
# I do not agree with these at all, so never load them

[-NamingConventions::Capitalization]
[-ValuesAndExpressions::ProhibitMagicNumbers]

#--------------------------------------------------------------
# For all other Policies, I accept the default severity,
# so no additional configuration is required for them.

For additional configuration examples, see the perlcriticrc file that is included in this examples directory of this distribution.

Damian Conway's own Perl::Critic configuration is also included in this distribution as examples/perlcriticrc-conway.

THE POLICIES

A large number of Policy modules are distributed with Perl::Critic. They are described briefly in the companion document Perl::Critic::PolicySummary and in more detail in the individual modules themselves. Say "perlcritic -doc PATTERN" to see the perldoc for all Policy modules that match the regex m/PATTERN/ixms

There are a number of distributions of additional policies on CPAN. If Perl::Critic doesn't contain a policy that you want, some one may have already written it. See the "SEE ALSO" section below for a list of some of these distributions.

POLICY THEMES

Each Policy is defined with one or more "themes". Themes can be used to create arbitrary groups of Policies. They are intended to provide an alternative mechanism for selecting your preferred set of Policies. For example, you may wish disable a certain subset of Policies when analyzing test programs. Conversely, you may wish to enable only a specific subset of Policies when analyzing modules.

The Policies that ship with Perl::Critic have been broken into the following themes. This is just our attempt to provide some basic logical groupings. You are free to invent new themes that suit your needs.

THEME             DESCRIPTION
--------------------------------------------------------------------------
core              All policies that ship with Perl::Critic
pbp               Policies that come directly from "Perl Best Practices"
bugs              Policies that that prevent or reveal bugs
certrec           Policies that CERT recommends
certrule          Policies that CERT considers rules
maintenance       Policies that affect the long-term health of the code
cosmetic          Policies that only have a superficial effect
complexity        Policies that specifically relate to code complexity
security          Policies that relate to security issues
tests             Policies that are specific to test programs

Any Policy may fit into multiple themes. Say "perlcritic -list" to get a listing of all available Policies and the themes that are associated with each one. You can also change the theme for any Policy in your .perlcriticrc file. See the "CONFIGURATION" section for more information about that.

Using the -theme option, you can create an arbitrarily complex rule that determines which Policies will be loaded. Precedence is the same as regular Perl code, and you can use parentheses to enforce precedence as well. Supported operators are:

Operator    Alternative    Example
-----------------------------------------------------------------
&&          and            'pbp && core'
||          or             'pbp || (bugs && security)'
!           not            'pbp && ! (portability || complexity)'

Theme names are case-insensitive. If the -theme is set to an empty string, then it evaluates as true all Policies.

BENDING THE RULES

Perl::Critic takes a hard-line approach to your code: either you comply or you don't. In the real world, it is not always practical (nor even possible) to fully comply with coding standards. In such cases, it is wise to show that you are knowingly violating the standards and that you have a Damn Good Reason (DGR) for doing so.

To help with those situations, you can direct Perl::Critic to ignore certain lines or blocks of code by using annotations:

require 'LegacyLibaray1.pl';  ## no critic
require 'LegacyLibrary2.pl';  ## no critic

for my $element (@list) {

    ## no critic

    $foo = "";               #Violates 'ProhibitEmptyQuotes'
    $barf = bar() if $foo;   #Violates 'ProhibitPostfixControls'
    #Some more evil code...

    ## use critic

    #Some good code...
    do_something($_);
}

The "## no critic" annotations direct Perl::Critic to ignore the remaining lines of code until a "## use critic" annotation is found. If the "## no critic" annotation is on the same line as a code statement, then only that line of code is overlooked. To direct perlcritic to ignore the "## no critic" annotations, use the --force option.

A bare "## no critic" annotation disables all the active Policies. If you wish to disable only specific Policies, add a list of Policy names as arguments, just as you would for the "no strict" or "no warnings" pragmas. For example, this would disable the ProhibitEmptyQuotes and ProhibitPostfixControls policies until the end of the block or until the next "## use critic" annotation (whichever comes first):

## no critic (EmptyQuotes, PostfixControls)

# Now exempt from ValuesAndExpressions::ProhibitEmptyQuotes
$foo = "";

# Now exempt ControlStructures::ProhibitPostfixControls
$barf = bar() if $foo;

# Still subjected to ValuesAndExpression::RequireNumberSeparators
$long_int = 10000000000;

Since the Policy names are matched against the "## no critic" arguments as regular expressions, you can abbreviate the Policy names or disable an entire family of Policies in one shot like this:

## no critic (NamingConventions)

# Now exempt from NamingConventions::Capitalization
my $camelHumpVar = 'foo';

# Now exempt from NamingConventions::Capitalization
sub camelHumpSub {}

The argument list must be enclosed in parentheses or brackets and must contain one or more comma-separated barewords (e.g. don't use quotes). The "## no critic" annotations can be nested, and Policies named by an inner annotation will be disabled along with those already disabled an outer annotation.

Some Policies like Subroutines::ProhibitExcessComplexity apply to an entire block of code. In those cases, the "## no critic" annotation must appear on the line where the violation is reported. For example:

sub complicated_function {  ## no critic (ProhibitExcessComplexity)
    # Your code here...
}

Policies such as Documentation::RequirePodSections apply to the entire document, in which case violations are reported at line 1.

Use this feature wisely. "## no critic" annotations should be used in the smallest possible scope, or only on individual lines of code. And you should always be as specific as possible about which Policies you want to disable (i.e. never use a bare "## no critic"). If Perl::Critic complains about your code, try and find a compliant solution before resorting to this feature.

THE Perl::Critic PHILOSOPHY

Coding standards are deeply personal and highly subjective. The goal of Perl::Critic is to help you write code that conforms with a set of best practices. Our primary goal is not to dictate what those practices are, but rather, to implement the practices discovered by others. Ultimately, you make the rules -- Perl::Critic is merely a tool for encouraging consistency. If there is a policy that you think is important or that we have overlooked, we would be very grateful for contributions, or you can simply load your own private set of policies into Perl::Critic.

EXTENDING THE CRITIC

The modular design of Perl::Critic is intended to facilitate the addition of new Policies. You'll need to have some understanding of PPI, but most Policy modules are pretty straightforward and only require about 20 lines of code. Please see the Perl::Critic::DEVELOPER file included in this distribution for a step-by-step demonstration of how to create new Policy modules.

If you develop any new Policy modules, feel free to send them to <team@perlcritic.com> and I'll be happy to consider putting them into the Perl::Critic distribution. Or if you would like to work on the Perl::Critic project directly, you can fork our repository at https://github.com/Perl-Critic/Perl-Critic.git.

The Perl::Critic team is also available for hire. If your organization has its own coding standards, we can create custom Policies to enforce your local guidelines. Or if your code base is prone to a particular defect pattern, we can design Policies that will help you catch those costly defects before they go into production. To discuss your needs with the Perl::Critic team, just contact <team@perlcritic.com>.

PREREQUISITES

Perl::Critic requires the following modules:

B::Keywords

Config::Tiny

Exception::Class

File::Spec

File::Spec::Unix

File::Which

IO::String

List::SomeUtils

List::Util

Module::Pluggable

Perl::Tidy

Pod::Spell

PPI

Pod::PlainText

Pod::Select

Pod::Usage

Readonly

Scalar::Util

String::Format

Task::Weaken

Term::ANSIColor

Text::ParseWords

version

CONTACTING THE DEVELOPMENT TEAM

You are encouraged to subscribe to the public mailing list at https://groups.google.com/d/forum/perl-critic. At least one member of the development team is usually hanging around in irc://irc.perl.org/#perlcritic and you can follow Perl::Critic on Twitter, at https://twitter.com/perlcritic.

SEE ALSO

There are a number of distributions of additional Policies available. A few are listed here:

Perl::Critic::More

Perl::Critic::Bangs

Perl::Critic::Lax

Perl::Critic::StricterSubs

Perl::Critic::Swift

Perl::Critic::Tics

These distributions enable you to use Perl::Critic in your unit tests:

Test::Perl::Critic

Test::Perl::Critic::Progressive

There is also a distribution that will install all the Perl::Critic related modules known to the development team:

Task::Perl::Critic

BUGS

Scrutinizing Perl code is hard for humans, let alone machines. If you find any bugs, particularly false-positives or false-negatives from a Perl::Critic::Policy, please submit them at https://github.com/Perl-Critic/Perl-Critic/issues. Thanks.

CREDITS

Adam Kennedy - For creating PPI, the heart and soul of Perl::Critic.

Damian Conway - For writing Perl Best Practices, finally :)

Chris Dolan - For contributing the best features and Policy modules.

Andy Lester - Wise sage and master of all-things-testing.

Elliot Shank - The self-proclaimed quality freak.

Giuseppe Maxia - For all the great ideas and positive encouragement.

and Sharon, my wife - For putting up with my all-night code sessions.

Thanks also to the Perl Foundation for providing a grant to support Chris Dolan's project to implement twenty PBP policies. http://www.perlfoundation.org/april_1_2007_new_grant_awards

Thanks also to this incomplete laundry list of folks who have contributed to Perl::Critic in some way: Gregory Oschwald, Mike O'Regan, Tom Hukins, Omer Gazit, Evan Zacks, Paul Howarth, Sawyer X, Christian Walde, Dave Rolsky, Jakub Wilk, Roy Ivy III, Oliver Trosien, Glenn Fowler, Matt Creenan, Alex Balhatchet, Sebastian Paaske Tørholm, Stuart A Johnston, Dan Book, Steven Humphrey, James Raspass, Nick Tonkin, Harrison Katz, Douglas Sims, Mark Fowler, Alan Berndt, Neil Bowers, Sergey Romanov, Gabor Szabo, Graham Knop, Mike Eldridge, David Steinbrunner, Kirk Kimmel, Guillaume Aubert, Dave Cross, Anirvan Chatterjee, Todd Rinaldo, Graham Ollis, Karen Etheridge, Jonas Brømsø, Olaf Alders, Jim Keenan, Slaven Rezić, Szymon Nieznański.

AUTHOR

Jeffrey Ryan Thalhammer jeff@imaginative-software.com

COPYRIGHT

Copyright (c) 2005-2018 Imaginative Software Systems. All rights reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The full text of this license can be found in the LICENSE file included with this module.

---

Download Details:

Author: Perl-Critic
Source Code: https://github.com/Perl-Critic/Perl-Critic

License: View license

#perl 

Asset Sync: Synchronises Assets Between Rails and S3

Asset Sync

Synchronises Assets between Rails and S3.

Asset Sync is built to run with the new Rails Asset Pipeline feature introduced in Rails 3.1. After you run bundle exec rake assets:precompile your assets will be synchronised to your S3 bucket, optionally deleting unused files and only uploading the files it needs to.

This was initially built and is intended to work on Heroku but can work on any platform.

Upgrading?

Upgraded from 1.x? Read UPGRADING.md

Installation

Since 2.x, Asset Sync depends on gem fog-core instead of fog.
This is due to fog is including many unused storage provider gems as its dependencies.

Asset Sync has no idea about what provider will be used,
so you are responsible for bundling the right gem for the provider to be used.

In your Gemfile:

gem "asset_sync"
gem "fog-aws"

Or, to use Azure Blob storage, configure as this.

gem "asset_sync"
gem "gitlab-fog-azure-rm"

# This gem seems unmaintianed
# gem "fog-azure-rm"

To use Backblaze B2, insert these.

gem "asset_sync"
gem "fog-backblaze"

Extended Installation (Faster sync with turbosprockets)

It's possible to improve asset:precompile time if you are using Rails 3.2.x the main source of which being compilation of non-digest assets.

turbo-sprockets-rails3 solves this by only compiling digest assets. Thus cutting compile time in half.

NOTE: It will be deprecated in Rails 4 as sprockets-rails has been extracted out of Rails and will only compile digest assets by default.

Configuration

Rails

Configure config/environments/production.rb to use Amazon S3 as the asset host and ensure precompiling is enabled.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com"

Or, to use Google Storage Cloud, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['FOG_DIRECTORY']}.storage.googleapis.com"

Or, to use Azure Blob storage, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['AZURE_STORAGE_ACCOUNT_NAME']}.blob.core.windows.net/#{ENV['FOG_DIRECTORY']}"

Or, to use Backblaze B2, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//f000.backblazeb2.com/file/#{ENV['FOG_DIRECTORY']}"

On HTTPS: the exclusion of any protocol in the asset host declaration above will allow browsers to choose the transport mechanism on the fly. So if your application is available under both HTTP and HTTPS the assets will be served to match.

The only caveat with this is that your S3 bucket name must not contain any periods so, mydomain.com.s3.amazonaws.com for example would not work under HTTPS as SSL certificates from Amazon would interpret our bucket name as not a subdomain of s3.amazonaws.com, but a multi level subdomain. To avoid this don't use a period in your subdomain or switch to the other style of S3 URL.

  config.action_controller.asset_host = "//s3.amazonaws.com/#{ENV['FOG_DIRECTORY']}"

Or, to use Google Storage Cloud, configure as this.

  config.action_controller.asset_host = "//storage.googleapis.com/#{ENV['FOG_DIRECTORY']}"

Or, to use Azure Blob storage, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['AZURE_STORAGE_ACCOUNT_NAME']}.blob.core.windows.net/#{ENV['FOG_DIRECTORY']}"

On non default S3 bucket region: If your bucket is set to a region that is not the default US Standard (us-east-1) you must use the first style of url //#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com or amazon will return a 301 permanently moved when assets are requested. Note the caveat above about bucket names and periods.

If you wish to have your assets sync to a sub-folder of your bucket instead of into the root add the following to your production.rb file

  # store assets in a 'folder' instead of bucket root
  config.assets.prefix = "/production/assets"

Also, ensure the following are defined (in production.rb or application.rb)

• config.assets.digest is set to true.
• config.assets.enabled is set to true.

Additionally, if you depend on any configuration that is setup in your initializers you will need to ensure that

• config.assets.initialize_on_precompile is set to true

AssetSync

AssetSync supports the following methods of configuration.

• Built-in Initializer (configured through environment variables)
• Rails Initializer
• A YAML config file

Using the Built-in Initializer is the default method and is supposed to be used with environment variables. It's the recommended approach for deployments on Heroku.

If you need more control over configuration you will want to use a custom rails initializer.

Configuration using a YAML file (a common strategy for Capistrano deployments) is also supported.

The recommend way to configure asset_sync is by using environment variables however it's up to you, it will work fine if you hard code them too. The main reason why using environment variables is recommended is so your access keys are not checked into version control.

Built-in Initializer (Environment Variables)

The Built-in Initializer will configure AssetSync based on the contents of your environment variables.

Add your configuration details to heroku

heroku config:add AWS_ACCESS_KEY_ID=xxxx
heroku config:add AWS_SECRET_ACCESS_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx
heroku config:add FOG_PROVIDER=AWS
# and optionally:
heroku config:add FOG_REGION=eu-west-1
heroku config:add ASSET_SYNC_GZIP_COMPRESSION=true
heroku config:add ASSET_SYNC_MANIFEST=true
heroku config:add ASSET_SYNC_EXISTING_REMOTE_FILES=keep

Or add to a traditional unix system

export AWS_ACCESS_KEY_ID=xxxx
export AWS_SECRET_ACCESS_KEY=xxxx
export FOG_DIRECTORY=xxxx

Rackspace configuration is also supported

heroku config:add RACKSPACE_USERNAME=xxxx
heroku config:add RACKSPACE_API_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx
heroku config:add FOG_PROVIDER=Rackspace

Google Storage Cloud configuration is supported as well. The preferred option is using the GCS JSON API which requires that you create an appropriate service account, generate the signatures and make them accessible to asset sync at the prescribed location

heroku config:add FOG_PROVIDER=Google
heroku config:add GOOGLE_PROJECT=xxxx
heroku config:add GOOGLE_JSON_KEY_LOCATION=xxxx
heroku config:add FOG_DIRECTORY=xxxx

If using the S3 API the following config is required

heroku config:add FOG_PROVIDER=Google
heroku config:add GOOGLE_STORAGE_ACCESS_KEY_ID=xxxx
heroku config:add GOOGLE_STORAGE_SECRET_ACCESS_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx

The Built-in Initializer also sets the AssetSync default for existing_remote_files to keep.

Custom Rails Initializer (config/initializers/asset_sync.rb)

If you want to enable some of the advanced configuration options you will want to create your own initializer.

Run the included Rake task to generate a starting point.

rails g asset_sync:install --provider=Rackspace
rails g asset_sync:install --provider=AWS
rails g asset_sync:install --provider=AzureRM
rails g asset_sync:install --provider=Backblaze

The generator will create a Rails initializer at config/initializers/asset_sync.rb.

AssetSync.configure do |config|
  config.fog_provider = 'AWS'
  config.fog_directory = ENV['FOG_DIRECTORY']
  config.aws_access_key_id = ENV['AWS_ACCESS_KEY_ID']
  config.aws_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
  config.aws_session_token = ENV['AWS_SESSION_TOKEN'] if ENV.key?('AWS_SESSION_TOKEN')

  # Don't delete files from the store
  # config.existing_remote_files = 'keep'
  #
  # Increase upload performance by configuring your region
  # config.fog_region = 'eu-west-1'
  #
  # Set `public` option when uploading file depending on value,
  # Setting to "default" makes asset sync skip setting the option
  # Possible values: true, false, "default" (default: true)
  # config.fog_public = true
  #
  # Change AWS signature version. Default is 4
  # config.aws_signature_version = 4
  #
  # Change canned ACL of uploaded object. Default is unset. Will override fog_public if set.
  # Choose from: private | public-read | public-read-write | aws-exec-read |
  #              authenticated-read | bucket-owner-read | bucket-owner-full-control 
  # config.aws_acl = nil 
  #
  # Change host option in fog (only if you need to)
  # config.fog_host = 's3.amazonaws.com'
  #
  # Change port option in fog (only if you need to)
  # config.fog_port = "9000"
  #
  # Use http instead of https.
  # config.fog_scheme = 'http'
  #
  # Automatically replace files with their equivalent gzip compressed version
  # config.gzip_compression = true
  #
  # Use the Rails generated 'manifest.yml' file to produce the list of files to
  # upload instead of searching the assets directory.
  # config.manifest = true
  #
  # Upload the manifest file also.
  # config.include_manifest = false
  #
  # Upload files concurrently
  # config.concurrent_uploads = false
  #
  # Number of threads when concurrent_uploads is enabled
  # config.concurrent_uploads_max_threads = 10
  #
  # Path to cache file to skip scanning remote
  # config.remote_file_list_cache_file_path = './.asset_sync_remote_file_list_cache.json'
  #
  # Fail silently.  Useful for environments such as Heroku
  # config.fail_silently = true
  #
  # Log silently. Default is `true`. But you can set it to false if more logging message are preferred.
  # Logging messages are sent to `STDOUT` when `log_silently` is falsy
  # config.log_silently = true
  #
  # Allow custom assets to be cacheable. Note: The base filename will be matched
  # If you have an asset with name `app.0b1a4cd3.js`, only `app.0b1a4cd3` will need to be matched
  # only one of `cache_asset_regexp` or `cache_asset_regexps` is allowed.
  # config.cache_asset_regexp = /\.[a-f0-9]{8}$/i
  # config.cache_asset_regexps = [ /\.[a-f0-9]{8}$/i, /\.[a-f0-9]{20}$/i ]
end

YAML (config/asset_sync.yml)

Run the included Rake task to generate a starting point.

rails g asset_sync:install --use-yml --provider=Rackspace
rails g asset_sync:install --use-yml --provider=AWS
rails g asset_sync:install --use-yml --provider=AzureRM
rails g asset_sync:install --use-yml --provider=Backblaze

The generator will create a YAML file at config/asset_sync.yml.

defaults: &defaults
  fog_provider: "AWS"
  fog_directory: "rails-app-assets"
  aws_access_key_id: "<%= ENV['AWS_ACCESS_KEY_ID'] %>"
  aws_secret_access_key: "<%= ENV['AWS_SECRET_ACCESS_KEY'] %>"

  # To use AWS reduced redundancy storage.
  # aws_reduced_redundancy: true
  #
  # You may need to specify what region your storage bucket is in
  # fog_region: "eu-west-1"
  #
  # Change AWS signature version. Default is 4
  # aws_signature_version: 4
  #
  # Change canned ACL of uploaded object. Default is unset. Will override fog_public if set.
  # Choose from: private | public-read | public-read-write | aws-exec-read |
  #              authenticated-read | bucket-owner-read | bucket-owner-full-control 
  # aws_acl: null
  #
  # Change host option in fog (only if you need to)
  # fog_host: "s3.amazonaws.com"
  #
  # Use http instead of https. Default should be "https" (at least for fog-aws)
  # fog_scheme: "http"

  existing_remote_files: keep # Existing pre-compiled assets on S3 will be kept
  # To delete existing remote files.
  # existing_remote_files: delete
  # To ignore existing remote files and overwrite.
  # existing_remote_files: ignore
  # Automatically replace files with their equivalent gzip compressed version
  # gzip_compression: true
  # Fail silently.  Useful for environments such as Heroku
  # fail_silently: true
  # Always upload. Useful if you want to overwrite specific remote assets regardless of their existence
  #  eg: Static files in public often reference non-fingerprinted application.css
  #  note: You will still need to expire them from the CDN's edge cache locations
  # always_upload: ['application.js', 'application.css', !ruby/regexp '/application-/\d{32}\.css/']
  # Ignored files. Useful if there are some files that are created dynamically on the server and you don't want to upload on deploy.
  # ignored_files: ['ignore_me.js', !ruby/regexp '/ignore_some/\d{32}\.css/']
  # Allow custom assets to be cacheable. Note: The base filename will be matched
  # If you have an asset with name "app.0b1a4cd3.js", only "app.0b1a4cd3" will need to be matched
  # cache_asset_regexps: ['cache_me.js', !ruby/regexp '/cache_some\.\d{8}\.css/']

development:
  <<: *defaults

test:
  <<: *defaults

production:
  <<: *defaults

Available Configuration Options

Most AssetSync configuration can be modified directly using environment variables with the Built-in initializer. e.g.

AssetSync.config.fog_provider == ENV['FOG_PROVIDER']

Simply upcase the ruby attribute names to get the equivalent environment variable to set. The only exception to that rule are the internal AssetSync config variables, they must be prepended with ASSET_SYNC_* e.g.

AssetSync.config.gzip_compression == ENV['ASSET_SYNC_GZIP_COMPRESSION']

AssetSync (optional)

• existing_remote_files: ('keep', 'delete', 'ignore') what to do with previously precompiled files. default: 'keep'
• gzip_compression: (true, false) when enabled, will automatically replace files that have a gzip compressed equivalent with the compressed version. default: 'false'
• manifest: (true, false) when enabled, will use the manifest.yml generated by Rails to get the list of local files to upload. experimental. default: 'false'
• include_manifest: (true, false) when enabled, will upload the manifest.yml generated by Rails. default: 'false'
• concurrent_uploads: (true, false) when enabled, will upload the files in different Threads, this greatly improves the upload speed. default: 'false'
• concurrent_uploads_max_threads: when concurrent_uploads is enabled, this determines the number of threads that will be created. default: 10
• remote_file_list_cache_file_path: if present, use this path to cache remote file list to skip scanning remote default: nil
• enabled: (true, false) when false, will disable asset sync. default: 'true' (enabled)
• ignored_files: an array of files to ignore e.g. ['ignore_me.js', %r(ignore_some/\d{32}\.css)] Useful if there are some files that are created dynamically on the server and you don't want to upload on deploy default: []
• cache_asset_regexps: an array of files to add cache headers e.g. ['cache_me.js', %r(cache_some\.\d{8}\.css)] Useful if there are some files that are added to sprockets assets list and need to be set as 'Cacheable' on uploaded server. Only rails compiled regexp is matched internally default: []

Config Method add_local_file_paths

Adding local files by providing a block:

AssetSync.configure do |config|
  # The block should return an array of file paths
  config.add_local_file_paths do
    # Any code that returns paths of local asset files to be uploaded
    # Like Webpacker
    public_root = Rails.root.join("public")
    Dir.chdir(public_root) do
      packs_dir = Webpacker.config.public_output_path.relative_path_from(public_root)
      Dir[File.join(packs_dir, '/**/**')]
    end
  end
end

The blocks are run when local files are being scanned and uploaded

Config Method file_ext_to_mime_type_overrides

It's reported that mime-types 3.x returns application/ecmascript instead of application/javascript
Such change of mime type might cause some CDN to disable asset compression
So this gem has defined a default override for file ext js to be mapped to application/javascript by default

To customize the overrides:

AssetSync.configure do |config|
  # Clear the default overrides
  config.file_ext_to_mime_type_overrides.clear

  # Add/Edit overrides
  # Will call `#to_s` for inputs
  config.file_ext_to_mime_type_overrides.add(:js, :"application/x-javascript")
end

The blocks are run when local files are being scanned and uploaded

Fog (Required)

• fog_provider: your storage provider AWS (S3) or Rackspace (Cloud Files) or Google (Google Storage) or AzureRM (Azure Blob) or Backblaze (Backblaze B2)
• fog_directory: your bucket name

Fog (Optional)

• fog_region: the region your storage bucket is in e.g. eu-west-1 (AWS), ord (Rackspace), japanwest (Azure Blob)
• fog_path_style: To use buckets with dot in names, check fog/fog#2381 (comment)

AWS

• aws_access_key_id: your Amazon S3 access key
• aws_secret_access_key: your Amazon S3 access secret
• aws_acl: set canned ACL of uploaded object, will override fog_public if set

Rackspace

• rackspace_username: your Rackspace username
• rackspace_api_key: your Rackspace API Key.

Google Storage

When using the JSON API

• google_project: your Google Cloud Project name where the Google Cloud Storage bucket resides
• google_json_key_location: path to the location of the service account key. The service account key must be a JSON type key

When using the S3 API

• google_storage_access_key_id: your Google Storage access key
• google_storage_secret_access_key: your Google Storage access secret

Azure Blob

• azure_storage_account_name: your Azure Blob access key
• azure_storage_access_key: your Azure Blob access secret

Backblaze B2

• b2_key_id: Your Backblaze B2 key ID
• b2_key_token: Your Backblaze B2 key token
• b2_bucket_id: Your Backblaze B2 bucket ID

Rackspace (Optional)

• rackspace_auth_url: Rackspace auth URL, for Rackspace London use: https://lon.identity.api.rackspacecloud.com/v2.0

Amazon S3 Multiple Region Support

If you are using anything other than the US buckets with S3 then you'll want to set the region. For example with an EU bucket you could set the following environment variable.

heroku config:add FOG_REGION=eu-west-1

Or via a custom initializer

AssetSync.configure do |config|
  # ...
  config.fog_region = 'eu-west-1'
end

Or via YAML

production:  # ...  fog_region: 'eu-west-1'

Amazon (AWS) IAM Users

Amazon has switched to the more secure IAM User security policy model. When generating a user & policy for asset_sync you must ensure the policy has the following permissions, or you'll see the error:

Expected(200) <=> Actual(403 Forbidden)

IAM User Policy Example with minimum require permissions (replace bucket_name with your bucket):

{
  "Statement": [
    {
      "Action": "s3:ListBucket",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::bucket_name"
    },
    {
      "Action": "s3:PutObject*",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::bucket_name/*"
    }
  ]
}

If you want to use IAM roles you must set config.aws_iam_roles = true in your initializers.

AssetSync.configure do |config|
  # ...
  config.aws_iam_roles = true
end

Automatic gzip compression

With the gzip_compression option enabled, when uploading your assets. If a file has a gzip compressed equivalent we will replace that asset with the compressed version and sets the correct headers for S3 to serve it. For example, if you have a file master.css and it was compressed to master.css.gz we will upload the .gz file to S3 in place of the uncompressed file.

If the compressed file is actually larger than the uncompressed file we will ignore this rule and upload the standard uncompressed version.

Fail Silently

With the fail_silently option enabled, when running rake assets:precompile AssetSync will never throw an error due to missing configuration variables.

With the new user_env_compile feature of Heroku (see above), this is no longer required or recommended. Yet was added for the following reasons:

With Rails 3.1 on the Heroku cedar stack, the deployment process automatically runs rake assets:precompile. If you are using ENV variable style configuration. Due to the methods with which Heroku compile slugs, there will be an error raised by asset_sync as the environment is not available. This causes heroku to install the rails31_enable_runtime_asset_compilation plugin which is not necessary when using asset_sync and also massively slows down the first incoming requests to your app.

To prevent this part of the deploy from failing (asset_sync raising a config error), but carry on as normal set fail_silently to true in your configuration and ensure to run heroku run rake assets:precompile after deploy.

Rake Task

A rake task is included within the asset_sync gem to perform the sync:

  namespace :assets do
    desc "Synchronize assets to S3"
    task :sync => :environment do
      AssetSync.sync
    end
  end

If AssetSync.config.run_on_precompile is true (default), then assets will be uploaded to S3 automatically after the assets:precompile rake task is invoked:

  if Rake::Task.task_defined?("assets:precompile:nondigest")
    Rake::Task["assets:precompile:nondigest"].enhance do
      Rake::Task["assets:sync"].invoke if defined?(AssetSync) && AssetSync.config.run_on_precompile
    end
  else
    Rake::Task["assets:precompile"].enhance do
      Rake::Task["assets:sync"].invoke if defined?(AssetSync) && AssetSync.config.run_on_precompile
    end
  end

You can disable this behavior by setting AssetSync.config.run_on_precompile = false.

Sinatra/Rack Support

You can use the gem with any Rack application, but you must specify two additional options; prefix and public_path.

AssetSync.configure do |config|
  config.fog_provider = 'AWS'
  config.fog_directory = ENV['FOG_DIRECTORY']
  config.aws_access_key_id = ENV['AWS_ACCESS_KEY_ID']
  config.aws_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
  config.prefix = 'assets'
  # Can be a `Pathname` or `String`
  # Will be converted into an `Pathname`
  # If relative, will be converted into an absolute path
  # via `::Rails.root` or `::Dir.pwd`
  config.public_path = Pathname('./public')
end

Then manually call AssetSync.sync at the end of your asset precompilation task.

namespace :assets do
  desc 'Precompile assets'
  task :precompile do
    target = Pathname('./public/assets')
    manifest = Sprockets::Manifest.new(sprockets, './public/assets/manifest.json')

    sprockets.each_logical_path do |logical_path|
      if (!File.extname(logical_path).in?(['.js', '.css']) || logical_path =~ /application\.(css|js)$/) && asset = sprockets.find_asset(logical_path)
        filename = target.join(logical_path)
        FileUtils.mkpath(filename.dirname)
        puts "Write asset: #{filename}"
        asset.write_to(filename)
        manifest.compile(logical_path)
      end
    end

    AssetSync.sync
  end
end

Webpacker (> 2.0) support

1. Add webpacker files and disable run_on_precompile:

AssetSync.configure do |config|
  # Disable automatic run on precompile in order to attach to webpacker rake task
  config.run_on_precompile = false
  # The block should return an array of file paths
  config.add_local_file_paths do
    # Support webpacker assets
    public_root = Rails.root.join("public")
    Dir.chdir(public_root) do
      packs_dir = Webpacker.config.public_output_path.relative_path_from(public_root)
      Dir[File.join(packs_dir, '/**/**')]
    end
  end
end

1. Add a asset_sync.rake in your lib/tasks directory that enhances the correct task, otherwise asset_sync runs before webpacker:compile does:

if defined?(AssetSync)
  Rake::Task['webpacker:compile'].enhance do
    Rake::Task["assets:sync"].invoke
  end
end

Caveat

By adding local files outside the normal Rails assets directory, the uploading part works, however checking that the asset was previously uploaded is not working because asset_sync is only fetching the files in the assets directory on the remote bucket. This will mean additional time used to upload the same assets again on every precompilation.

Running the specs

Make sure you have a .env file with these details:-

# for AWS provider
AWS_ACCESS_KEY_ID=<yourkeyid>
AWS_SECRET_ACCESS_KEY=<yoursecretkey>
FOG_DIRECTORY=<yourbucket>
FOG_REGION=<youbucketregion>

# for AzureRM provider
AZURE_STORAGE_ACCOUNT_NAME=<youraccountname>
AZURE_STORAGE_ACCESS_KEY=<youraccesskey>
FOG_DIRECTORY=<yourcontainer>
FOG_REGION=<yourcontainerregion>

Make sure the bucket has read/write permissions. Then to run the tests:-

foreman run rake

Todo

1. Add some before and after filters for deleting and uploading
2. Support more cloud storage providers
3. Better test coverage
4. Add rake tasks to clean old assets from a bucket

Credits

Inspired by:

• https://github.com/moocode/asset_id
• https://gist.github.com/1053855

License

MIT License. Copyright 2011-2013 Rumble Labs Ltd. rumblelabs.com

---

Author: AssetSync
Source code: https://github.com/AssetSync/asset_sync
License:

#ruby   #ruby-on-rails