How to setup Webpack config - tutorial for beginners

This article was originally published at https://www.blog.duomly.com/what-is-webpack-and-how-to-setup-webpack/


Intro to Webpack tutorial

Webpack is a tool very often used in software development, and it’s pretty useful. For many junior and even mid developers, Webpak can be the darkest nightmare, and it was scary for me as well. By all the power, I tried to avoid setting Webpack by using tools like create-react-app to build a new ReactJS application and never have to do any changes inside the ready config, as it seemed confusing.

But for every developer, it comes the day when you just have to beat the beast and learn how it works, to set Webpack in a more efficient way for a particular project and make your software even better.

In this article, I’d like to help you with this overwhelming and confusing area of development and explain some basics about Webpack, what it is, why we use it, how it works, and the most important, how to setup Webpack.

For those of you who would like to watch instead of reading, as always, I have a video tutorial on our Youtube channel.

Let’s start!

What is webpack

Webpack is a module bundler for web applications that bundles, minify, and transpile JavaScript code to the code understandable by all the browsers. As a Javascript developer, you probably have heard about modules before. Webpack has a broad definition of what modules are, and it takes into consideration: ES modules, CommonJS modules, CSS imports, image URLs, or AMD modules.

It’s important to understand that Webpack is not used on the actual page in the production environment. It’s running during the development process. What we can see life is a code generated as a bundle by the Webpack.

How Webpack works?

Webpack goes through your application and creates a dependency graph consisting of modules that the application needs to work properly. Next, based on the dependency graph, it creates a new package, mostly it’s just one file called bundle.js, but it can also be divided into chunks, so smaller files are loaded as necessary. The package file created by Webpack is imported as a script in the HTML file of the application and makes the app working as expected.

Although since version 4.0, it’s not required to create a config file to use Webpack to bundle your project, it’s good to know what’s inside to adjust the config and get more profit from using the tools.

To be able to create a well-working config, it’s important to understand the core concepts of Webpack:

Webpack entry point indicates where Webpack should start building the dependency graph. From this point, it will find the other dependencies required to provide the proper functionality of the application. By default, it’s ./src/index.js

Webpack output is a point where the Webpack emits bundles and the name of the bundle file. By default, it’s main.js, and the folder is ./dist/, but inside the configuration file, you can place it somewhere else in the application and name the file differently.

Webpack loaders are used to process any other types of files, as Webpack by default, can recognize only JSON and Javascript. That’s why we need some additional settings for images or styles. The principal task of the loaders is to convert those sources to valid modules, which can be added to the dependency graph. There are two properties in high-level Webpack loaders configuration, test to identify which files and folders should be transformed, and use to indicate which loaders should be used for transforming.

Webpack plugins are used to perform a wider range of tasks, like bundle optimization or environment variables injection. To use plugins, we need to import them using require(), and then add it to the plugins array.

Webpack mode is a parameter that lets Webpack know what’s the environment of the application. It can have the following values: development, production or none. By default, the mode is set to production.

Well, when you understand most of the important Webpack concepts, we can jump to a practical Webpack setup.

How to setup Webpack?

At this point, I’d like to go through setting the simple Javascript application with Webpack and set by step show you how it works. For the purpose of this tutorial, I’ll be using Webpack 5.

Webpack install

Let’s start by creating a new project and initialize npm.

mkdir webpackTutorial && cd webpackTutorial

npm init -y

Next, let’s install all the necessary tools: webpack and webpack-cli.

npm install webpack webpack-cli

When this step is finished successfully, let’s create a simple project structure and a file with some dummy code.

So, let’s open the project folder, and let’s add the src folder and index.js file like below.

Duomly - Webpack setup

Inside the index.js file, let’s add a simple Javascript code to use to build our bundle and be injected into our main page.

const title = document.createElement('h3');
title.textContent = 'Webpack made easy!';

const page = document.querySelector('body');
page.append(title);

Webpack config

To start with the Webpack setup, we need one more file. Inside the root folder, let’s create the file webpack.config.js, and let’s start by setting the entry point and output for our application bundle.

const path = require('path')

module.exports = {
  entry: {
    index: path.resolve(__dirname, './src/index.js') 
  }
}

In the first line of the code, we imported the path module, which provides utilities for working with paths of files and directories. It’s a part of Node.js.

Then, we told Webpack where to look for the first file, which will be the base to create a dependency graph.

Next, we can setup the folder and file where we would like Webpack to save our bundle, and we will set it as an output.

const path = require('path')

module.exports = {
  entry: {
    index: path.resolve(__dirname, './src/index.js') 
  },
  output: {
    path: path.resolve(__dirname, './build'),
    filename: '[name].bunde.js'
  }
}

In the output part of our Webpack config, I set the location of the bundle to ./build folder instead of default ./dist, and as a filename, I passed a variable, so the [name] of our bundle file will be as defined in the entry index.bundle.js.

Right now, we can try out our Webpack bundle, as we have a minimum config ready. Let’s open package.json file, and let’s adjust it to run Webpack from the console.

"scripts": {
  "test": "echo \"Error: no test specified\" && exit 1",
  "build": "webpack"
},

When it’s ready, open the project in the console and run the following command.

npm run build

When it’s done, you should see something like the image below in your console. Also, you’ll be able to see a new folder in your application with the bundle file. Nothing interesting happened there yet, but the build exists.

Duomly - How to setup Webpack?

Webpack - Html Plugin

Right now, I’ll show you how to add plugins to your Webpack config to make it more flexible. We want to add some HTML to our application, so we need it to be generated, and for this, we need a Webpack plugin. Let’s install html-webpack-plugin.

npm install html-webpack-plugin

When that’s installed, we can go back to our config file and configure the plugin.

const HtmlWebpackPlugin = require('html-webpack-plugin')

module.exports = {
  ...
  plugins: [
    new HTMLWebpackPlugin({
      template: path.resolve(__dirname, "./src/template.html")
    })
  ]
}

Before we move forward to update our build, we need actually to create our template.html file.

<!DOCTYPE html>
<html> 
  <head>
    <meta charset="UTF-8">
    <title>Hello world!</title>
  </head>
  <body>
    Hello world!
  </body>
</html>

Now, let’s build the bundle again and run the index.html file you will find in the build folder in the browser. You should be able to see the content of the HTML file we’ve created and the text we used in our index.js file.

Duomly - Webpack Tutorial

Webpack loaders

If we would like to add any other type of files like assets or CSS, we need to add loaders that allow Webpack to understand them, compile and use them. Let’s add some images to our code, and then we will need to set the assets loader.

Let’s create a new folder inside ./src, and let’s call it assets. Let’s add any image there. And next, try to import it in your index.js file, and let’s append it to our page.

import koala from './assets/koala.png'


...

const img = new Image();
img.src = koala;
page.appendChild(img);

Right now, to be able to update the bundle, you need to add a loader in your Webpack config.

module: {
  rules: [
  {
    test: /\.(?:ico|gif|png|jpg|jpeg)$/i,
    type: 'asset/resource',
  },
],
}

Let’s run the build once again, and let’s refresh the index.js file in the browser, and then you can see your image below the text.

Now, let’s learn how to use styles with Webpack.

Webpack CSS loader

Let’s create a simple styles file, and let’s import it into our index.js file.

// style.css
body {
  text-align: center;
  margin: 20px;
}
h3 {
  color: #333333;
  background: #f5f5f5;
  padding: 20px;
}
img {
  max-height: 500px;
}

//index.js 
import './style.css';

Now, to be able to see the changes, we need to install loaders for CSS: style-loader and css-loader.

npm install css-loader style-loader

When those are installed, let’s go back to our Webpack config, and let’s add the loader rule.

module: {
  rules: [
    {
      test: /\.css$/,
      use: ["style-loader", "css-loader"]
    },
    {
      test: /\.(?:ico|gif|png|jpg|jpeg)$/i,
      type: 'asset/resource',
    },
  ],
}

After that, we can rerun the bundle and see if our styles work.

It seems like it works!

Conclusion

Congratulations, you’ve just created a simple Webpack setup!

In this article, we went through what is Webpack and how Webpack works. Next, we went through the most important concepts of Webpack that you need to understand to build any Webpack configuration.

I hope this knowledge will help you next time when someone will start talking about Webpack. You can be as confident as never before and stop being scared of the big Webpack monster.

In my opinion, it’s not so difficult and confusing when you will know what’s the purpose of using it, how it works, and what the concepts are responsible for.

Thank you for reading,
Anna from Duomly

#webpack #javascript #web-development #html5 #html #css

What is GEEK

Buddha Community

How to setup Webpack config - tutorial for beginners

Matteo Gioioso

1607266841

Video is not displayed

Jeromy  Lowe

Jeromy Lowe

1599097440

Data Visualization in R with ggplot2: A Beginner Tutorial

A famous general is thought to have said, “A good sketch is better than a long speech.” That advice may have come from the battlefield, but it’s applicable in lots of other areas — including data science. “Sketching” out our data by visualizing it using ggplot2 in R is more impactful than simply describing the trends we find.

This is why we visualize data. We visualize data because it’s easier to learn from something that we can see rather than read. And thankfully for data analysts and data scientists who use R, there’s a tidyverse package called ggplot2 that makes data visualization a snap!

In this blog post, we’ll learn how to take some data and produce a visualization using R. To work through it, it’s best if you already have an understanding of R programming syntax, but you don’t need to be an expert or have any prior experience working with ggplot2

#data science tutorials #beginner #ggplot2 #r #r tutorial #r tutorials #rstats #tutorial #tutorials

Willie  Beier

Willie Beier

1596728880

Tutorial: Getting Started with R and RStudio

In this tutorial we’ll learn how to begin programming with R using RStudio. We’ll install R, and RStudio RStudio, an extremely popular development environment for R. We’ll learn the key RStudio features in order to start programming in R on our own.

If you already know how to use RStudio and want to learn some tips, tricks, and shortcuts, check out this Dataquest blog post.

Table of Contents

#data science tutorials #beginner #r tutorial #r tutorials #rstats #tutorial #tutorials

Tutorial: Loading and Cleaning Data with R and the tidyverse

1. Characteristics of Clean Data and Messy Data

What exactly is clean data? Clean data is accurate, complete, and in a format that is ready to analyze. Characteristics of clean data include data that are:

  • Free of duplicate rows/values
  • Error-free (e.g. free of misspellings)
  • Relevant (e.g. free of special characters)
  • The appropriate data type for analysis
  • Free of outliers (or only contain outliers have been identified/understood), and
  • Follows a “tidy data” structure

Common symptoms of messy data include data that contain:

  • Special characters (e.g. commas in numeric values)
  • Numeric values stored as text/character data types
  • Duplicate rows
  • Misspellings
  • Inaccuracies
  • White space
  • Missing data
  • Zeros instead of null values

2. Motivation

In this blog post, we will work with five property-sales datasets that are publicly available on the New York City Department of Finance Rolling Sales Data website. We encourage you to download the datasets and follow along! Each file contains one year of real estate sales data for one of New York City’s five boroughs. We will work with the following Microsoft Excel files:

  • rollingsales_bronx.xls
  • rollingsales_brooklyn.xls
  • rollingsales_manhattan.xls
  • rollingsales_queens.xls
  • rollingsales_statenisland.xls

As we work through this blog post, imagine that you are helping a friend launch their home-inspection business in New York City. You offer to help them by analyzing the data to better understand the real-estate market. But you realize that before you can analyze the data in R, you will need to diagnose and clean it first. And before you can diagnose the data, you will need to load it into R!

3. Load Data into R with readxl

Benefits of using tidyverse tools are often evident in the data-loading process. In many cases, the tidyverse package readxl will clean some data for you as Microsoft Excel data is loaded into R. If you are working with CSV data, the tidyverse readr package function read_csv() is the function to use (we’ll cover that later).

Let’s look at an example. Here’s how the Excel file for the Brooklyn borough looks:

The Brooklyn Excel file

Now let’s load the Brooklyn dataset into R from an Excel file. We’ll use the readxlpackage. We specify the function argument skip = 4 because the row that we want to use as the header (i.e. column names) is actually row 5. We can ignore the first four rows entirely and load the data into R beginning at row 5. Here’s the code:

library(readxl) # Load Excel files
brooklyn <- read_excel("rollingsales_brooklyn.xls", skip = 4)

Note we saved this dataset with the variable name brooklyn for future use.

4. View the Data with tidyr::glimpse()

The tidyverse offers a user-friendly way to view this data with the glimpse() function that is part of the tibble package. To use this package, we will need to load it for use in our current session. But rather than loading this package alone, we can load many of the tidyverse packages at one time. If you do not have the tidyverse collection of packages, install it on your machine using the following command in your R or R Studio session:

install.packages("tidyverse")

Once the package is installed, load it to memory:

library(tidyverse)

Now that tidyverse is loaded into memory, take a “glimpse” of the Brooklyn dataset:

glimpse(brooklyn)
## Observations: 20,185
## Variables: 21
## $ BOROUGH <chr> "3", "3", "3", "3", "3", "3", "…
## $ NEIGHBORHOOD <chr> "BATH BEACH", "BATH BEACH", "BA…
## $ `BUILDING CLASS CATEGORY` <chr> "01 ONE FAMILY DWELLINGS", "01 …
## $ `TAX CLASS AT PRESENT` <chr> "1", "1", "1", "1", "1", "1", "…
## $ BLOCK <dbl> 6359, 6360, 6364, 6367, 6371, 6…
## $ LOT <dbl> 70, 48, 74, 24, 19, 32, 65, 20,…
## $ `EASE-MENT` <lgl> NA, NA, NA, NA, NA, NA, NA, NA,…
## $ `BUILDING CLASS AT PRESENT` <chr> "S1", "A5", "A5", "A9", "A9", "…
## $ ADDRESS <chr> "8684 15TH AVENUE", "14 BAY 10T…
## $ `APARTMENT NUMBER` <chr> NA, NA, NA, NA, NA, NA, NA, NA,…
## $ `ZIP CODE` <dbl> 11228, 11228, 11214, 11214, 112…
## $ `RESIDENTIAL UNITS` <dbl> 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1…
## $ `COMMERCIAL UNITS` <dbl> 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `TOTAL UNITS` <dbl> 2, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1…
## $ `LAND SQUARE FEET` <dbl> 1933, 2513, 2492, 1571, 2320, 3…
## $ `GROSS SQUARE FEET` <dbl> 4080, 1428, 972, 1456, 1566, 22…
## $ `YEAR BUILT` <dbl> 1930, 1930, 1950, 1935, 1930, 1…
## $ `TAX CLASS AT TIME OF SALE` <chr> "1", "1", "1", "1", "1", "1", "…
## $ `BUILDING CLASS AT TIME OF SALE` <chr> "S1", "A5", "A5", "A9", "A9", "…
## $ `SALE PRICE` <dbl> 1300000, 849000, 0, 830000, 0, …
## $ `SALE DATE` <dttm> 2020-04-28, 2020-03-18, 2019-0…

The glimpse() function provides a user-friendly way to view the column names and data types for all columns, or variables, in the data frame. With this function, we are also able to view the first few observations in the data frame. This data frame has 20,185 observations, or property sales records. And there are 21 variables, or columns.

#data science tutorials #beginner #r #r tutorial #r tutorials #rstats #tidyverse #tutorial #tutorials

Asset Sync: Synchronises Assets Between Rails and S3

Asset Sync

Synchronises Assets between Rails and S3.

Asset Sync is built to run with the new Rails Asset Pipeline feature introduced in Rails 3.1. After you run bundle exec rake assets:precompile your assets will be synchronised to your S3 bucket, optionally deleting unused files and only uploading the files it needs to.

This was initially built and is intended to work on Heroku but can work on any platform.

Upgrading?

Upgraded from 1.x? Read UPGRADING.md

Installation

Since 2.x, Asset Sync depends on gem fog-core instead of fog.
This is due to fog is including many unused storage provider gems as its dependencies.

Asset Sync has no idea about what provider will be used,
so you are responsible for bundling the right gem for the provider to be used.

In your Gemfile:

gem "asset_sync"
gem "fog-aws"

Or, to use Azure Blob storage, configure as this.

gem "asset_sync"
gem "gitlab-fog-azure-rm"

# This gem seems unmaintianed
# gem "fog-azure-rm"

To use Backblaze B2, insert these.

gem "asset_sync"
gem "fog-backblaze"

Extended Installation (Faster sync with turbosprockets)

It's possible to improve asset:precompile time if you are using Rails 3.2.x the main source of which being compilation of non-digest assets.

turbo-sprockets-rails3 solves this by only compiling digest assets. Thus cutting compile time in half.

NOTE: It will be deprecated in Rails 4 as sprockets-rails has been extracted out of Rails and will only compile digest assets by default.

Configuration

Rails

Configure config/environments/production.rb to use Amazon S3 as the asset host and ensure precompiling is enabled.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com"

Or, to use Google Storage Cloud, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['FOG_DIRECTORY']}.storage.googleapis.com"

Or, to use Azure Blob storage, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['AZURE_STORAGE_ACCOUNT_NAME']}.blob.core.windows.net/#{ENV['FOG_DIRECTORY']}"

Or, to use Backblaze B2, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//f000.backblazeb2.com/file/#{ENV['FOG_DIRECTORY']}"

On HTTPS: the exclusion of any protocol in the asset host declaration above will allow browsers to choose the transport mechanism on the fly. So if your application is available under both HTTP and HTTPS the assets will be served to match.

The only caveat with this is that your S3 bucket name must not contain any periods so, mydomain.com.s3.amazonaws.com for example would not work under HTTPS as SSL certificates from Amazon would interpret our bucket name as not a subdomain of s3.amazonaws.com, but a multi level subdomain. To avoid this don't use a period in your subdomain or switch to the other style of S3 URL.

  config.action_controller.asset_host = "//s3.amazonaws.com/#{ENV['FOG_DIRECTORY']}"

Or, to use Google Storage Cloud, configure as this.

  config.action_controller.asset_host = "//storage.googleapis.com/#{ENV['FOG_DIRECTORY']}"

Or, to use Azure Blob storage, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['AZURE_STORAGE_ACCOUNT_NAME']}.blob.core.windows.net/#{ENV['FOG_DIRECTORY']}"

On non default S3 bucket region: If your bucket is set to a region that is not the default US Standard (us-east-1) you must use the first style of url //#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com or amazon will return a 301 permanently moved when assets are requested. Note the caveat above about bucket names and periods.

If you wish to have your assets sync to a sub-folder of your bucket instead of into the root add the following to your production.rb file

  # store assets in a 'folder' instead of bucket root
  config.assets.prefix = "/production/assets"

Also, ensure the following are defined (in production.rb or application.rb)

  • config.assets.digest is set to true.
  • config.assets.enabled is set to true.

Additionally, if you depend on any configuration that is setup in your initializers you will need to ensure that

  • config.assets.initialize_on_precompile is set to true

AssetSync

AssetSync supports the following methods of configuration.

Using the Built-in Initializer is the default method and is supposed to be used with environment variables. It's the recommended approach for deployments on Heroku.

If you need more control over configuration you will want to use a custom rails initializer.

Configuration using a YAML file (a common strategy for Capistrano deployments) is also supported.

The recommend way to configure asset_sync is by using environment variables however it's up to you, it will work fine if you hard code them too. The main reason why using environment variables is recommended is so your access keys are not checked into version control.

Built-in Initializer (Environment Variables)

The Built-in Initializer will configure AssetSync based on the contents of your environment variables.

Add your configuration details to heroku

heroku config:add AWS_ACCESS_KEY_ID=xxxx
heroku config:add AWS_SECRET_ACCESS_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx
heroku config:add FOG_PROVIDER=AWS
# and optionally:
heroku config:add FOG_REGION=eu-west-1
heroku config:add ASSET_SYNC_GZIP_COMPRESSION=true
heroku config:add ASSET_SYNC_MANIFEST=true
heroku config:add ASSET_SYNC_EXISTING_REMOTE_FILES=keep

Or add to a traditional unix system

export AWS_ACCESS_KEY_ID=xxxx
export AWS_SECRET_ACCESS_KEY=xxxx
export FOG_DIRECTORY=xxxx

Rackspace configuration is also supported

heroku config:add RACKSPACE_USERNAME=xxxx
heroku config:add RACKSPACE_API_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx
heroku config:add FOG_PROVIDER=Rackspace

Google Storage Cloud configuration is supported as well. The preferred option is using the GCS JSON API which requires that you create an appropriate service account, generate the signatures and make them accessible to asset sync at the prescribed location

heroku config:add FOG_PROVIDER=Google
heroku config:add GOOGLE_PROJECT=xxxx
heroku config:add GOOGLE_JSON_KEY_LOCATION=xxxx
heroku config:add FOG_DIRECTORY=xxxx

If using the S3 API the following config is required

heroku config:add FOG_PROVIDER=Google
heroku config:add GOOGLE_STORAGE_ACCESS_KEY_ID=xxxx
heroku config:add GOOGLE_STORAGE_SECRET_ACCESS_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx

The Built-in Initializer also sets the AssetSync default for existing_remote_files to keep.

Custom Rails Initializer (config/initializers/asset_sync.rb)

If you want to enable some of the advanced configuration options you will want to create your own initializer.

Run the included Rake task to generate a starting point.

rails g asset_sync:install --provider=Rackspace
rails g asset_sync:install --provider=AWS
rails g asset_sync:install --provider=AzureRM
rails g asset_sync:install --provider=Backblaze

The generator will create a Rails initializer at config/initializers/asset_sync.rb.

AssetSync.configure do |config|
  config.fog_provider = 'AWS'
  config.fog_directory = ENV['FOG_DIRECTORY']
  config.aws_access_key_id = ENV['AWS_ACCESS_KEY_ID']
  config.aws_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
  config.aws_session_token = ENV['AWS_SESSION_TOKEN'] if ENV.key?('AWS_SESSION_TOKEN')

  # Don't delete files from the store
  # config.existing_remote_files = 'keep'
  #
  # Increase upload performance by configuring your region
  # config.fog_region = 'eu-west-1'
  #
  # Set `public` option when uploading file depending on value,
  # Setting to "default" makes asset sync skip setting the option
  # Possible values: true, false, "default" (default: true)
  # config.fog_public = true
  #
  # Change AWS signature version. Default is 4
  # config.aws_signature_version = 4
  #
  # Change canned ACL of uploaded object. Default is unset. Will override fog_public if set.
  # Choose from: private | public-read | public-read-write | aws-exec-read |
  #              authenticated-read | bucket-owner-read | bucket-owner-full-control 
  # config.aws_acl = nil 
  #
  # Change host option in fog (only if you need to)
  # config.fog_host = 's3.amazonaws.com'
  #
  # Change port option in fog (only if you need to)
  # config.fog_port = "9000"
  #
  # Use http instead of https.
  # config.fog_scheme = 'http'
  #
  # Automatically replace files with their equivalent gzip compressed version
  # config.gzip_compression = true
  #
  # Use the Rails generated 'manifest.yml' file to produce the list of files to
  # upload instead of searching the assets directory.
  # config.manifest = true
  #
  # Upload the manifest file also.
  # config.include_manifest = false
  #
  # Upload files concurrently
  # config.concurrent_uploads = false
  #
  # Number of threads when concurrent_uploads is enabled
  # config.concurrent_uploads_max_threads = 10
  #
  # Path to cache file to skip scanning remote
  # config.remote_file_list_cache_file_path = './.asset_sync_remote_file_list_cache.json'
  #
  # Fail silently.  Useful for environments such as Heroku
  # config.fail_silently = true
  #
  # Log silently. Default is `true`. But you can set it to false if more logging message are preferred.
  # Logging messages are sent to `STDOUT` when `log_silently` is falsy
  # config.log_silently = true
  #
  # Allow custom assets to be cacheable. Note: The base filename will be matched
  # If you have an asset with name `app.0b1a4cd3.js`, only `app.0b1a4cd3` will need to be matched
  # only one of `cache_asset_regexp` or `cache_asset_regexps` is allowed.
  # config.cache_asset_regexp = /\.[a-f0-9]{8}$/i
  # config.cache_asset_regexps = [ /\.[a-f0-9]{8}$/i, /\.[a-f0-9]{20}$/i ]
end

YAML (config/asset_sync.yml)

Run the included Rake task to generate a starting point.

rails g asset_sync:install --use-yml --provider=Rackspace
rails g asset_sync:install --use-yml --provider=AWS
rails g asset_sync:install --use-yml --provider=AzureRM
rails g asset_sync:install --use-yml --provider=Backblaze

The generator will create a YAML file at config/asset_sync.yml.

defaults: &defaults
  fog_provider: "AWS"
  fog_directory: "rails-app-assets"
  aws_access_key_id: "<%= ENV['AWS_ACCESS_KEY_ID'] %>"
  aws_secret_access_key: "<%= ENV['AWS_SECRET_ACCESS_KEY'] %>"

  # To use AWS reduced redundancy storage.
  # aws_reduced_redundancy: true
  #
  # You may need to specify what region your storage bucket is in
  # fog_region: "eu-west-1"
  #
  # Change AWS signature version. Default is 4
  # aws_signature_version: 4
  #
  # Change canned ACL of uploaded object. Default is unset. Will override fog_public if set.
  # Choose from: private | public-read | public-read-write | aws-exec-read |
  #              authenticated-read | bucket-owner-read | bucket-owner-full-control 
  # aws_acl: null
  #
  # Change host option in fog (only if you need to)
  # fog_host: "s3.amazonaws.com"
  #
  # Use http instead of https. Default should be "https" (at least for fog-aws)
  # fog_scheme: "http"

  existing_remote_files: keep # Existing pre-compiled assets on S3 will be kept
  # To delete existing remote files.
  # existing_remote_files: delete
  # To ignore existing remote files and overwrite.
  # existing_remote_files: ignore
  # Automatically replace files with their equivalent gzip compressed version
  # gzip_compression: true
  # Fail silently.  Useful for environments such as Heroku
  # fail_silently: true
  # Always upload. Useful if you want to overwrite specific remote assets regardless of their existence
  #  eg: Static files in public often reference non-fingerprinted application.css
  #  note: You will still need to expire them from the CDN's edge cache locations
  # always_upload: ['application.js', 'application.css', !ruby/regexp '/application-/\d{32}\.css/']
  # Ignored files. Useful if there are some files that are created dynamically on the server and you don't want to upload on deploy.
  # ignored_files: ['ignore_me.js', !ruby/regexp '/ignore_some/\d{32}\.css/']
  # Allow custom assets to be cacheable. Note: The base filename will be matched
  # If you have an asset with name "app.0b1a4cd3.js", only "app.0b1a4cd3" will need to be matched
  # cache_asset_regexps: ['cache_me.js', !ruby/regexp '/cache_some\.\d{8}\.css/']

development:
  <<: *defaults

test:
  <<: *defaults

production:
  <<: *defaults

Available Configuration Options

Most AssetSync configuration can be modified directly using environment variables with the Built-in initializer. e.g.

AssetSync.config.fog_provider == ENV['FOG_PROVIDER']

Simply upcase the ruby attribute names to get the equivalent environment variable to set. The only exception to that rule are the internal AssetSync config variables, they must be prepended with ASSET_SYNC_* e.g.

AssetSync.config.gzip_compression == ENV['ASSET_SYNC_GZIP_COMPRESSION']

AssetSync (optional)

  • existing_remote_files: ('keep', 'delete', 'ignore') what to do with previously precompiled files. default: 'keep'
  • gzip_compression: (true, false) when enabled, will automatically replace files that have a gzip compressed equivalent with the compressed version. default: 'false'
  • manifest: (true, false) when enabled, will use the manifest.yml generated by Rails to get the list of local files to upload. experimental. default: 'false'
  • include_manifest: (true, false) when enabled, will upload the manifest.yml generated by Rails. default: 'false'
  • concurrent_uploads: (true, false) when enabled, will upload the files in different Threads, this greatly improves the upload speed. default: 'false'
  • concurrent_uploads_max_threads: when concurrent_uploads is enabled, this determines the number of threads that will be created. default: 10
  • remote_file_list_cache_file_path: if present, use this path to cache remote file list to skip scanning remote default: nil
  • enabled: (true, false) when false, will disable asset sync. default: 'true' (enabled)
  • ignored_files: an array of files to ignore e.g. ['ignore_me.js', %r(ignore_some/\d{32}\.css)] Useful if there are some files that are created dynamically on the server and you don't want to upload on deploy default: []
  • cache_asset_regexps: an array of files to add cache headers e.g. ['cache_me.js', %r(cache_some\.\d{8}\.css)] Useful if there are some files that are added to sprockets assets list and need to be set as 'Cacheable' on uploaded server. Only rails compiled regexp is matched internally default: []

Config Method add_local_file_paths

Adding local files by providing a block:

AssetSync.configure do |config|
  # The block should return an array of file paths
  config.add_local_file_paths do
    # Any code that returns paths of local asset files to be uploaded
    # Like Webpacker
    public_root = Rails.root.join("public")
    Dir.chdir(public_root) do
      packs_dir = Webpacker.config.public_output_path.relative_path_from(public_root)
      Dir[File.join(packs_dir, '/**/**')]
    end
  end
end

The blocks are run when local files are being scanned and uploaded

Config Method file_ext_to_mime_type_overrides

It's reported that mime-types 3.x returns application/ecmascript instead of application/javascript
Such change of mime type might cause some CDN to disable asset compression
So this gem has defined a default override for file ext js to be mapped to application/javascript by default

To customize the overrides:

AssetSync.configure do |config|
  # Clear the default overrides
  config.file_ext_to_mime_type_overrides.clear

  # Add/Edit overrides
  # Will call `#to_s` for inputs
  config.file_ext_to_mime_type_overrides.add(:js, :"application/x-javascript")
end

The blocks are run when local files are being scanned and uploaded

Fog (Required)

  • fog_provider: your storage provider AWS (S3) or Rackspace (Cloud Files) or Google (Google Storage) or AzureRM (Azure Blob) or Backblaze (Backblaze B2)
  • fog_directory: your bucket name

Fog (Optional)

  • fog_region: the region your storage bucket is in e.g. eu-west-1 (AWS), ord (Rackspace), japanwest (Azure Blob)
  • fog_path_style: To use buckets with dot in names, check fog/fog#2381 (comment)

AWS

  • aws_access_key_id: your Amazon S3 access key
  • aws_secret_access_key: your Amazon S3 access secret
  • aws_acl: set canned ACL of uploaded object, will override fog_public if set

Rackspace

  • rackspace_username: your Rackspace username
  • rackspace_api_key: your Rackspace API Key.

Google Storage

When using the JSON API

  • google_project: your Google Cloud Project name where the Google Cloud Storage bucket resides
  • google_json_key_location: path to the location of the service account key. The service account key must be a JSON type key

When using the S3 API

  • google_storage_access_key_id: your Google Storage access key
  • google_storage_secret_access_key: your Google Storage access secret

Azure Blob

  • azure_storage_account_name: your Azure Blob access key
  • azure_storage_access_key: your Azure Blob access secret

Backblaze B2

  • b2_key_id: Your Backblaze B2 key ID
  • b2_key_token: Your Backblaze B2 key token
  • b2_bucket_id: Your Backblaze B2 bucket ID

Rackspace (Optional)

  • rackspace_auth_url: Rackspace auth URL, for Rackspace London use: https://lon.identity.api.rackspacecloud.com/v2.0

Amazon S3 Multiple Region Support

If you are using anything other than the US buckets with S3 then you'll want to set the region. For example with an EU bucket you could set the following environment variable.

heroku config:add FOG_REGION=eu-west-1

Or via a custom initializer

AssetSync.configure do |config|
  # ...
  config.fog_region = 'eu-west-1'
end

Or via YAML

production:  # ...  fog_region: 'eu-west-1'

Amazon (AWS) IAM Users

Amazon has switched to the more secure IAM User security policy model. When generating a user & policy for asset_sync you must ensure the policy has the following permissions, or you'll see the error:

Expected(200) <=> Actual(403 Forbidden)

IAM User Policy Example with minimum require permissions (replace bucket_name with your bucket):

{
  "Statement": [
    {
      "Action": "s3:ListBucket",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::bucket_name"
    },
    {
      "Action": "s3:PutObject*",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::bucket_name/*"
    }
  ]
}

If you want to use IAM roles you must set config.aws_iam_roles = true in your initializers.

AssetSync.configure do |config|
  # ...
  config.aws_iam_roles = true
end

Automatic gzip compression

With the gzip_compression option enabled, when uploading your assets. If a file has a gzip compressed equivalent we will replace that asset with the compressed version and sets the correct headers for S3 to serve it. For example, if you have a file master.css and it was compressed to master.css.gz we will upload the .gz file to S3 in place of the uncompressed file.

If the compressed file is actually larger than the uncompressed file we will ignore this rule and upload the standard uncompressed version.

Fail Silently

With the fail_silently option enabled, when running rake assets:precompile AssetSync will never throw an error due to missing configuration variables.

With the new user_env_compile feature of Heroku (see above), this is no longer required or recommended. Yet was added for the following reasons:

With Rails 3.1 on the Heroku cedar stack, the deployment process automatically runs rake assets:precompile. If you are using ENV variable style configuration. Due to the methods with which Heroku compile slugs, there will be an error raised by asset_sync as the environment is not available. This causes heroku to install the rails31_enable_runtime_asset_compilation plugin which is not necessary when using asset_sync and also massively slows down the first incoming requests to your app.

To prevent this part of the deploy from failing (asset_sync raising a config error), but carry on as normal set fail_silently to true in your configuration and ensure to run heroku run rake assets:precompile after deploy.

Rake Task

A rake task is included within the asset_sync gem to perform the sync:

  namespace :assets do
    desc "Synchronize assets to S3"
    task :sync => :environment do
      AssetSync.sync
    end
  end

If AssetSync.config.run_on_precompile is true (default), then assets will be uploaded to S3 automatically after the assets:precompile rake task is invoked:

  if Rake::Task.task_defined?("assets:precompile:nondigest")
    Rake::Task["assets:precompile:nondigest"].enhance do
      Rake::Task["assets:sync"].invoke if defined?(AssetSync) && AssetSync.config.run_on_precompile
    end
  else
    Rake::Task["assets:precompile"].enhance do
      Rake::Task["assets:sync"].invoke if defined?(AssetSync) && AssetSync.config.run_on_precompile
    end
  end

You can disable this behavior by setting AssetSync.config.run_on_precompile = false.

Sinatra/Rack Support

You can use the gem with any Rack application, but you must specify two additional options; prefix and public_path.

AssetSync.configure do |config|
  config.fog_provider = 'AWS'
  config.fog_directory = ENV['FOG_DIRECTORY']
  config.aws_access_key_id = ENV['AWS_ACCESS_KEY_ID']
  config.aws_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
  config.prefix = 'assets'
  # Can be a `Pathname` or `String`
  # Will be converted into an `Pathname`
  # If relative, will be converted into an absolute path
  # via `::Rails.root` or `::Dir.pwd`
  config.public_path = Pathname('./public')
end

Then manually call AssetSync.sync at the end of your asset precompilation task.

namespace :assets do
  desc 'Precompile assets'
  task :precompile do
    target = Pathname('./public/assets')
    manifest = Sprockets::Manifest.new(sprockets, './public/assets/manifest.json')

    sprockets.each_logical_path do |logical_path|
      if (!File.extname(logical_path).in?(['.js', '.css']) || logical_path =~ /application\.(css|js)$/) && asset = sprockets.find_asset(logical_path)
        filename = target.join(logical_path)
        FileUtils.mkpath(filename.dirname)
        puts "Write asset: #{filename}"
        asset.write_to(filename)
        manifest.compile(logical_path)
      end
    end

    AssetSync.sync
  end
end

Webpacker (> 2.0) support

  1. Add webpacker files and disable run_on_precompile:
AssetSync.configure do |config|
  # Disable automatic run on precompile in order to attach to webpacker rake task
  config.run_on_precompile = false
  # The block should return an array of file paths
  config.add_local_file_paths do
    # Support webpacker assets
    public_root = Rails.root.join("public")
    Dir.chdir(public_root) do
      packs_dir = Webpacker.config.public_output_path.relative_path_from(public_root)
      Dir[File.join(packs_dir, '/**/**')]
    end
  end
end
  1. Add a asset_sync.rake in your lib/tasks directory that enhances the correct task, otherwise asset_sync runs before webpacker:compile does:
if defined?(AssetSync)
  Rake::Task['webpacker:compile'].enhance do
    Rake::Task["assets:sync"].invoke
  end
end

Caveat

By adding local files outside the normal Rails assets directory, the uploading part works, however checking that the asset was previously uploaded is not working because asset_sync is only fetching the files in the assets directory on the remote bucket. This will mean additional time used to upload the same assets again on every precompilation.

Running the specs

Make sure you have a .env file with these details:-

# for AWS provider
AWS_ACCESS_KEY_ID=<yourkeyid>
AWS_SECRET_ACCESS_KEY=<yoursecretkey>
FOG_DIRECTORY=<yourbucket>
FOG_REGION=<youbucketregion>

# for AzureRM provider
AZURE_STORAGE_ACCOUNT_NAME=<youraccountname>
AZURE_STORAGE_ACCESS_KEY=<youraccesskey>
FOG_DIRECTORY=<yourcontainer>
FOG_REGION=<yourcontainerregion>

Make sure the bucket has read/write permissions. Then to run the tests:-

foreman run rake

Todo

  1. Add some before and after filters for deleting and uploading
  2. Support more cloud storage providers
  3. Better test coverage
  4. Add rake tasks to clean old assets from a bucket

Credits

Inspired by:

License

MIT License. Copyright 2011-2013 Rumble Labs Ltd. rumblelabs.com


Author: AssetSync
Source code: https://github.com/AssetSync/asset_sync
License:

#ruby   #ruby-on-rails 

Marcus  Flatley

Marcus Flatley

1594399440

Getting Started with R Markdown — Guide and Cheatsheet

In this blog post, we’ll look at how to use R Markdown. By the end, you’ll have the skills you need to produce a document or presentation using R Mardown, from scratch!

We’ll show you how to convert the default R Markdown document into a useful reference guide of your own. We encourage you to follow along by building out your own R Markdown guide, but if you prefer to just read along, that works, too!

R Markdown is an open-source tool for producing reproducible reports in R. It enables you to keep all of your code, results, plots, and writing in one place. R Markdown is particularly useful when you are producing a document for an audience that is interested in the results from your analysis, but not your code.

R Markdown is powerful because it can be used for data analysis and data science, collaborating with others, and communicating results to decision makers. With R Markdown, you have the option to export your work to numerous formats including PDF, Microsoft Word, a slideshow, or an HTML document for use in a website.

r markdown tips, tricks, and shortcuts

Turn your data analysis into pretty documents with R Markdown.

We’ll use the RStudio integrated development environment (IDE) to produce our R Markdown reference guide. If you’d like to learn more about RStudio, check out our list of 23 awesome RStudio tips and tricks!

Here at Dataquest, we love using R Markdown for coding in R and authoring content. In fact, we wrote this blog post in R Markdown! Also, learners on the Dataquest platform use R Markdown for completing their R projects.

We included fully-reproducible code examples in this blog post. When you’ve mastered the content in this post, check out our other blog post on R Markdown tips, tricks, and shortcuts.

Okay, let’s get started with building our very own R Markdown reference document!

R Markdown Guide and Cheatsheet: Quick Navigation

1. Install R Markdown

R Markdown is a free, open source tool that is installed like any other R package. Use the following command to install R Markdown:

install.packages("rmarkdown")

Now that R Markdown is installed, open a new R Markdown file in RStudio by navigating to File > New File > R Markdown…. R Markdown files have the file extension “.Rmd”.

2. Default Output Format

When you open a new R Markdown file in RStudio, a pop-up window appears that prompts you to select output format to use for the document.

New Document

The default output format is HTML. With HTML, you can easily view it in a web browser.

We recommend selecting the default HTML setting for now — it can save you time! Why? Because compiling an HTML document is generally faster than generating a PDF or other format. When you near a finished product, you change the output to the format of your choosing and then make the final touches.

One final thing to note is that the title you give your document in the pop-up above is not the file name! Navigate to File > Save As.. to name, and save, the document.

#data science tutorials #beginner #r #r markdown #r tutorial #r tutorials #rstats #rstudio #tutorial #tutorials