野村  太郎

野村 太郎

1598055060

Config ルール・自動修復を使ってS3全バケットのデフォルト暗号化を自動的に有効化する

AWS Config は AWSリソースの構成を記録、管理するためのサービスです。 EC2インスタンスやセキュリティグループなど、AWSリソースが「いつ、どのような変更がされたか」 の履歴を記録してくれます。

また、 AWS Config ルール を利用して AWSリソースの構成の評価ができます。 AWSリソースのあるべき構成を条件としています。条件に違反しているリソースが検出されると 「非準拠」フラグが付けられます。

さらに、この「非準拠」リソースを 修復 することもできます。 “AWS Systems Manager オートメーション(SSM オートメーション)” という自動化を行うためのワークフローと連携して、 「非準拠」リソースが見つかれば SSMオートメーションを使ってあるべき構成へ自動修復する、便利な自動化が実現できます。

本記事では S3の全バケットのデフォルト暗号化を自動的に有効化する ための Configルール/自動修復を 構築してみます。

img

目次

  1. やってみる(ルールの作成)
  2. やってみる(IAMロールの作成)
  3. やってみる(自動修復設定)
  4. 確認
  5. CloudFormation テンプレート
  6. おわりに
  7. 参考

やってみる(ルールの作成)

まずは、「デフォルト暗号化が有効になっていないバケットを非準拠とする」Configルールを作成します。 ※Configを有効化していることは前提です。

「AWS Config > ルール」の画面から「ルールを追加」を選択します。

img

AWS管理のルール s3-bucket-server-side-encryption-enabled を選択します。

img

#aws #config #aws systems manager

What is GEEK

Buddha Community

Config ルール・自動修復を使ってS3全バケットのデフォルト暗号化を自動的に有効化する

Asset Sync: Synchronises Assets Between Rails and S3

Asset Sync

Synchronises Assets between Rails and S3.

Asset Sync is built to run with the new Rails Asset Pipeline feature introduced in Rails 3.1. After you run bundle exec rake assets:precompile your assets will be synchronised to your S3 bucket, optionally deleting unused files and only uploading the files it needs to.

This was initially built and is intended to work on Heroku but can work on any platform.

Upgrading?

Upgraded from 1.x? Read UPGRADING.md

Installation

Since 2.x, Asset Sync depends on gem fog-core instead of fog.
This is due to fog is including many unused storage provider gems as its dependencies.

Asset Sync has no idea about what provider will be used,
so you are responsible for bundling the right gem for the provider to be used.

In your Gemfile:

gem "asset_sync"
gem "fog-aws"

Or, to use Azure Blob storage, configure as this.

gem "asset_sync"
gem "gitlab-fog-azure-rm"

# This gem seems unmaintianed
# gem "fog-azure-rm"

To use Backblaze B2, insert these.

gem "asset_sync"
gem "fog-backblaze"

Extended Installation (Faster sync with turbosprockets)

It's possible to improve asset:precompile time if you are using Rails 3.2.x the main source of which being compilation of non-digest assets.

turbo-sprockets-rails3 solves this by only compiling digest assets. Thus cutting compile time in half.

NOTE: It will be deprecated in Rails 4 as sprockets-rails has been extracted out of Rails and will only compile digest assets by default.

Configuration

Rails

Configure config/environments/production.rb to use Amazon S3 as the asset host and ensure precompiling is enabled.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com"

Or, to use Google Storage Cloud, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['FOG_DIRECTORY']}.storage.googleapis.com"

Or, to use Azure Blob storage, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['AZURE_STORAGE_ACCOUNT_NAME']}.blob.core.windows.net/#{ENV['FOG_DIRECTORY']}"

Or, to use Backblaze B2, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//f000.backblazeb2.com/file/#{ENV['FOG_DIRECTORY']}"

On HTTPS: the exclusion of any protocol in the asset host declaration above will allow browsers to choose the transport mechanism on the fly. So if your application is available under both HTTP and HTTPS the assets will be served to match.

The only caveat with this is that your S3 bucket name must not contain any periods so, mydomain.com.s3.amazonaws.com for example would not work under HTTPS as SSL certificates from Amazon would interpret our bucket name as not a subdomain of s3.amazonaws.com, but a multi level subdomain. To avoid this don't use a period in your subdomain or switch to the other style of S3 URL.

  config.action_controller.asset_host = "//s3.amazonaws.com/#{ENV['FOG_DIRECTORY']}"

Or, to use Google Storage Cloud, configure as this.

  config.action_controller.asset_host = "//storage.googleapis.com/#{ENV['FOG_DIRECTORY']}"

Or, to use Azure Blob storage, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['AZURE_STORAGE_ACCOUNT_NAME']}.blob.core.windows.net/#{ENV['FOG_DIRECTORY']}"

On non default S3 bucket region: If your bucket is set to a region that is not the default US Standard (us-east-1) you must use the first style of url //#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com or amazon will return a 301 permanently moved when assets are requested. Note the caveat above about bucket names and periods.

If you wish to have your assets sync to a sub-folder of your bucket instead of into the root add the following to your production.rb file

  # store assets in a 'folder' instead of bucket root
  config.assets.prefix = "/production/assets"

Also, ensure the following are defined (in production.rb or application.rb)

  • config.assets.digest is set to true.
  • config.assets.enabled is set to true.

Additionally, if you depend on any configuration that is setup in your initializers you will need to ensure that

  • config.assets.initialize_on_precompile is set to true

AssetSync

AssetSync supports the following methods of configuration.

Using the Built-in Initializer is the default method and is supposed to be used with environment variables. It's the recommended approach for deployments on Heroku.

If you need more control over configuration you will want to use a custom rails initializer.

Configuration using a YAML file (a common strategy for Capistrano deployments) is also supported.

The recommend way to configure asset_sync is by using environment variables however it's up to you, it will work fine if you hard code them too. The main reason why using environment variables is recommended is so your access keys are not checked into version control.

Built-in Initializer (Environment Variables)

The Built-in Initializer will configure AssetSync based on the contents of your environment variables.

Add your configuration details to heroku

heroku config:add AWS_ACCESS_KEY_ID=xxxx
heroku config:add AWS_SECRET_ACCESS_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx
heroku config:add FOG_PROVIDER=AWS
# and optionally:
heroku config:add FOG_REGION=eu-west-1
heroku config:add ASSET_SYNC_GZIP_COMPRESSION=true
heroku config:add ASSET_SYNC_MANIFEST=true
heroku config:add ASSET_SYNC_EXISTING_REMOTE_FILES=keep

Or add to a traditional unix system

export AWS_ACCESS_KEY_ID=xxxx
export AWS_SECRET_ACCESS_KEY=xxxx
export FOG_DIRECTORY=xxxx

Rackspace configuration is also supported

heroku config:add RACKSPACE_USERNAME=xxxx
heroku config:add RACKSPACE_API_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx
heroku config:add FOG_PROVIDER=Rackspace

Google Storage Cloud configuration is supported as well. The preferred option is using the GCS JSON API which requires that you create an appropriate service account, generate the signatures and make them accessible to asset sync at the prescribed location

heroku config:add FOG_PROVIDER=Google
heroku config:add GOOGLE_PROJECT=xxxx
heroku config:add GOOGLE_JSON_KEY_LOCATION=xxxx
heroku config:add FOG_DIRECTORY=xxxx

If using the S3 API the following config is required

heroku config:add FOG_PROVIDER=Google
heroku config:add GOOGLE_STORAGE_ACCESS_KEY_ID=xxxx
heroku config:add GOOGLE_STORAGE_SECRET_ACCESS_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx

The Built-in Initializer also sets the AssetSync default for existing_remote_files to keep.

Custom Rails Initializer (config/initializers/asset_sync.rb)

If you want to enable some of the advanced configuration options you will want to create your own initializer.

Run the included Rake task to generate a starting point.

rails g asset_sync:install --provider=Rackspace
rails g asset_sync:install --provider=AWS
rails g asset_sync:install --provider=AzureRM
rails g asset_sync:install --provider=Backblaze

The generator will create a Rails initializer at config/initializers/asset_sync.rb.

AssetSync.configure do |config|
  config.fog_provider = 'AWS'
  config.fog_directory = ENV['FOG_DIRECTORY']
  config.aws_access_key_id = ENV['AWS_ACCESS_KEY_ID']
  config.aws_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
  config.aws_session_token = ENV['AWS_SESSION_TOKEN'] if ENV.key?('AWS_SESSION_TOKEN')

  # Don't delete files from the store
  # config.existing_remote_files = 'keep'
  #
  # Increase upload performance by configuring your region
  # config.fog_region = 'eu-west-1'
  #
  # Set `public` option when uploading file depending on value,
  # Setting to "default" makes asset sync skip setting the option
  # Possible values: true, false, "default" (default: true)
  # config.fog_public = true
  #
  # Change AWS signature version. Default is 4
  # config.aws_signature_version = 4
  #
  # Change canned ACL of uploaded object. Default is unset. Will override fog_public if set.
  # Choose from: private | public-read | public-read-write | aws-exec-read |
  #              authenticated-read | bucket-owner-read | bucket-owner-full-control 
  # config.aws_acl = nil 
  #
  # Change host option in fog (only if you need to)
  # config.fog_host = 's3.amazonaws.com'
  #
  # Change port option in fog (only if you need to)
  # config.fog_port = "9000"
  #
  # Use http instead of https.
  # config.fog_scheme = 'http'
  #
  # Automatically replace files with their equivalent gzip compressed version
  # config.gzip_compression = true
  #
  # Use the Rails generated 'manifest.yml' file to produce the list of files to
  # upload instead of searching the assets directory.
  # config.manifest = true
  #
  # Upload the manifest file also.
  # config.include_manifest = false
  #
  # Upload files concurrently
  # config.concurrent_uploads = false
  #
  # Number of threads when concurrent_uploads is enabled
  # config.concurrent_uploads_max_threads = 10
  #
  # Path to cache file to skip scanning remote
  # config.remote_file_list_cache_file_path = './.asset_sync_remote_file_list_cache.json'
  #
  # Fail silently.  Useful for environments such as Heroku
  # config.fail_silently = true
  #
  # Log silently. Default is `true`. But you can set it to false if more logging message are preferred.
  # Logging messages are sent to `STDOUT` when `log_silently` is falsy
  # config.log_silently = true
  #
  # Allow custom assets to be cacheable. Note: The base filename will be matched
  # If you have an asset with name `app.0b1a4cd3.js`, only `app.0b1a4cd3` will need to be matched
  # only one of `cache_asset_regexp` or `cache_asset_regexps` is allowed.
  # config.cache_asset_regexp = /\.[a-f0-9]{8}$/i
  # config.cache_asset_regexps = [ /\.[a-f0-9]{8}$/i, /\.[a-f0-9]{20}$/i ]
end

YAML (config/asset_sync.yml)

Run the included Rake task to generate a starting point.

rails g asset_sync:install --use-yml --provider=Rackspace
rails g asset_sync:install --use-yml --provider=AWS
rails g asset_sync:install --use-yml --provider=AzureRM
rails g asset_sync:install --use-yml --provider=Backblaze

The generator will create a YAML file at config/asset_sync.yml.

defaults: &defaults
  fog_provider: "AWS"
  fog_directory: "rails-app-assets"
  aws_access_key_id: "<%= ENV['AWS_ACCESS_KEY_ID'] %>"
  aws_secret_access_key: "<%= ENV['AWS_SECRET_ACCESS_KEY'] %>"

  # To use AWS reduced redundancy storage.
  # aws_reduced_redundancy: true
  #
  # You may need to specify what region your storage bucket is in
  # fog_region: "eu-west-1"
  #
  # Change AWS signature version. Default is 4
  # aws_signature_version: 4
  #
  # Change canned ACL of uploaded object. Default is unset. Will override fog_public if set.
  # Choose from: private | public-read | public-read-write | aws-exec-read |
  #              authenticated-read | bucket-owner-read | bucket-owner-full-control 
  # aws_acl: null
  #
  # Change host option in fog (only if you need to)
  # fog_host: "s3.amazonaws.com"
  #
  # Use http instead of https. Default should be "https" (at least for fog-aws)
  # fog_scheme: "http"

  existing_remote_files: keep # Existing pre-compiled assets on S3 will be kept
  # To delete existing remote files.
  # existing_remote_files: delete
  # To ignore existing remote files and overwrite.
  # existing_remote_files: ignore
  # Automatically replace files with their equivalent gzip compressed version
  # gzip_compression: true
  # Fail silently.  Useful for environments such as Heroku
  # fail_silently: true
  # Always upload. Useful if you want to overwrite specific remote assets regardless of their existence
  #  eg: Static files in public often reference non-fingerprinted application.css
  #  note: You will still need to expire them from the CDN's edge cache locations
  # always_upload: ['application.js', 'application.css', !ruby/regexp '/application-/\d{32}\.css/']
  # Ignored files. Useful if there are some files that are created dynamically on the server and you don't want to upload on deploy.
  # ignored_files: ['ignore_me.js', !ruby/regexp '/ignore_some/\d{32}\.css/']
  # Allow custom assets to be cacheable. Note: The base filename will be matched
  # If you have an asset with name "app.0b1a4cd3.js", only "app.0b1a4cd3" will need to be matched
  # cache_asset_regexps: ['cache_me.js', !ruby/regexp '/cache_some\.\d{8}\.css/']

development:
  <<: *defaults

test:
  <<: *defaults

production:
  <<: *defaults

Available Configuration Options

Most AssetSync configuration can be modified directly using environment variables with the Built-in initializer. e.g.

AssetSync.config.fog_provider == ENV['FOG_PROVIDER']

Simply upcase the ruby attribute names to get the equivalent environment variable to set. The only exception to that rule are the internal AssetSync config variables, they must be prepended with ASSET_SYNC_* e.g.

AssetSync.config.gzip_compression == ENV['ASSET_SYNC_GZIP_COMPRESSION']

AssetSync (optional)

  • existing_remote_files: ('keep', 'delete', 'ignore') what to do with previously precompiled files. default: 'keep'
  • gzip_compression: (true, false) when enabled, will automatically replace files that have a gzip compressed equivalent with the compressed version. default: 'false'
  • manifest: (true, false) when enabled, will use the manifest.yml generated by Rails to get the list of local files to upload. experimental. default: 'false'
  • include_manifest: (true, false) when enabled, will upload the manifest.yml generated by Rails. default: 'false'
  • concurrent_uploads: (true, false) when enabled, will upload the files in different Threads, this greatly improves the upload speed. default: 'false'
  • concurrent_uploads_max_threads: when concurrent_uploads is enabled, this determines the number of threads that will be created. default: 10
  • remote_file_list_cache_file_path: if present, use this path to cache remote file list to skip scanning remote default: nil
  • enabled: (true, false) when false, will disable asset sync. default: 'true' (enabled)
  • ignored_files: an array of files to ignore e.g. ['ignore_me.js', %r(ignore_some/\d{32}\.css)] Useful if there are some files that are created dynamically on the server and you don't want to upload on deploy default: []
  • cache_asset_regexps: an array of files to add cache headers e.g. ['cache_me.js', %r(cache_some\.\d{8}\.css)] Useful if there are some files that are added to sprockets assets list and need to be set as 'Cacheable' on uploaded server. Only rails compiled regexp is matched internally default: []

Config Method add_local_file_paths

Adding local files by providing a block:

AssetSync.configure do |config|
  # The block should return an array of file paths
  config.add_local_file_paths do
    # Any code that returns paths of local asset files to be uploaded
    # Like Webpacker
    public_root = Rails.root.join("public")
    Dir.chdir(public_root) do
      packs_dir = Webpacker.config.public_output_path.relative_path_from(public_root)
      Dir[File.join(packs_dir, '/**/**')]
    end
  end
end

The blocks are run when local files are being scanned and uploaded

Config Method file_ext_to_mime_type_overrides

It's reported that mime-types 3.x returns application/ecmascript instead of application/javascript
Such change of mime type might cause some CDN to disable asset compression
So this gem has defined a default override for file ext js to be mapped to application/javascript by default

To customize the overrides:

AssetSync.configure do |config|
  # Clear the default overrides
  config.file_ext_to_mime_type_overrides.clear

  # Add/Edit overrides
  # Will call `#to_s` for inputs
  config.file_ext_to_mime_type_overrides.add(:js, :"application/x-javascript")
end

The blocks are run when local files are being scanned and uploaded

Fog (Required)

  • fog_provider: your storage provider AWS (S3) or Rackspace (Cloud Files) or Google (Google Storage) or AzureRM (Azure Blob) or Backblaze (Backblaze B2)
  • fog_directory: your bucket name

Fog (Optional)

  • fog_region: the region your storage bucket is in e.g. eu-west-1 (AWS), ord (Rackspace), japanwest (Azure Blob)
  • fog_path_style: To use buckets with dot in names, check fog/fog#2381 (comment)

AWS

  • aws_access_key_id: your Amazon S3 access key
  • aws_secret_access_key: your Amazon S3 access secret
  • aws_acl: set canned ACL of uploaded object, will override fog_public if set

Rackspace

  • rackspace_username: your Rackspace username
  • rackspace_api_key: your Rackspace API Key.

Google Storage

When using the JSON API

  • google_project: your Google Cloud Project name where the Google Cloud Storage bucket resides
  • google_json_key_location: path to the location of the service account key. The service account key must be a JSON type key

When using the S3 API

  • google_storage_access_key_id: your Google Storage access key
  • google_storage_secret_access_key: your Google Storage access secret

Azure Blob

  • azure_storage_account_name: your Azure Blob access key
  • azure_storage_access_key: your Azure Blob access secret

Backblaze B2

  • b2_key_id: Your Backblaze B2 key ID
  • b2_key_token: Your Backblaze B2 key token
  • b2_bucket_id: Your Backblaze B2 bucket ID

Rackspace (Optional)

  • rackspace_auth_url: Rackspace auth URL, for Rackspace London use: https://lon.identity.api.rackspacecloud.com/v2.0

Amazon S3 Multiple Region Support

If you are using anything other than the US buckets with S3 then you'll want to set the region. For example with an EU bucket you could set the following environment variable.

heroku config:add FOG_REGION=eu-west-1

Or via a custom initializer

AssetSync.configure do |config|
  # ...
  config.fog_region = 'eu-west-1'
end

Or via YAML

production:  # ...  fog_region: 'eu-west-1'

Amazon (AWS) IAM Users

Amazon has switched to the more secure IAM User security policy model. When generating a user & policy for asset_sync you must ensure the policy has the following permissions, or you'll see the error:

Expected(200) <=> Actual(403 Forbidden)

IAM User Policy Example with minimum require permissions (replace bucket_name with your bucket):

{
  "Statement": [
    {
      "Action": "s3:ListBucket",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::bucket_name"
    },
    {
      "Action": "s3:PutObject*",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::bucket_name/*"
    }
  ]
}

If you want to use IAM roles you must set config.aws_iam_roles = true in your initializers.

AssetSync.configure do |config|
  # ...
  config.aws_iam_roles = true
end

Automatic gzip compression

With the gzip_compression option enabled, when uploading your assets. If a file has a gzip compressed equivalent we will replace that asset with the compressed version and sets the correct headers for S3 to serve it. For example, if you have a file master.css and it was compressed to master.css.gz we will upload the .gz file to S3 in place of the uncompressed file.

If the compressed file is actually larger than the uncompressed file we will ignore this rule and upload the standard uncompressed version.

Fail Silently

With the fail_silently option enabled, when running rake assets:precompile AssetSync will never throw an error due to missing configuration variables.

With the new user_env_compile feature of Heroku (see above), this is no longer required or recommended. Yet was added for the following reasons:

With Rails 3.1 on the Heroku cedar stack, the deployment process automatically runs rake assets:precompile. If you are using ENV variable style configuration. Due to the methods with which Heroku compile slugs, there will be an error raised by asset_sync as the environment is not available. This causes heroku to install the rails31_enable_runtime_asset_compilation plugin which is not necessary when using asset_sync and also massively slows down the first incoming requests to your app.

To prevent this part of the deploy from failing (asset_sync raising a config error), but carry on as normal set fail_silently to true in your configuration and ensure to run heroku run rake assets:precompile after deploy.

Rake Task

A rake task is included within the asset_sync gem to perform the sync:

  namespace :assets do
    desc "Synchronize assets to S3"
    task :sync => :environment do
      AssetSync.sync
    end
  end

If AssetSync.config.run_on_precompile is true (default), then assets will be uploaded to S3 automatically after the assets:precompile rake task is invoked:

  if Rake::Task.task_defined?("assets:precompile:nondigest")
    Rake::Task["assets:precompile:nondigest"].enhance do
      Rake::Task["assets:sync"].invoke if defined?(AssetSync) && AssetSync.config.run_on_precompile
    end
  else
    Rake::Task["assets:precompile"].enhance do
      Rake::Task["assets:sync"].invoke if defined?(AssetSync) && AssetSync.config.run_on_precompile
    end
  end

You can disable this behavior by setting AssetSync.config.run_on_precompile = false.

Sinatra/Rack Support

You can use the gem with any Rack application, but you must specify two additional options; prefix and public_path.

AssetSync.configure do |config|
  config.fog_provider = 'AWS'
  config.fog_directory = ENV['FOG_DIRECTORY']
  config.aws_access_key_id = ENV['AWS_ACCESS_KEY_ID']
  config.aws_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
  config.prefix = 'assets'
  # Can be a `Pathname` or `String`
  # Will be converted into an `Pathname`
  # If relative, will be converted into an absolute path
  # via `::Rails.root` or `::Dir.pwd`
  config.public_path = Pathname('./public')
end

Then manually call AssetSync.sync at the end of your asset precompilation task.

namespace :assets do
  desc 'Precompile assets'
  task :precompile do
    target = Pathname('./public/assets')
    manifest = Sprockets::Manifest.new(sprockets, './public/assets/manifest.json')

    sprockets.each_logical_path do |logical_path|
      if (!File.extname(logical_path).in?(['.js', '.css']) || logical_path =~ /application\.(css|js)$/) && asset = sprockets.find_asset(logical_path)
        filename = target.join(logical_path)
        FileUtils.mkpath(filename.dirname)
        puts "Write asset: #{filename}"
        asset.write_to(filename)
        manifest.compile(logical_path)
      end
    end

    AssetSync.sync
  end
end

Webpacker (> 2.0) support

  1. Add webpacker files and disable run_on_precompile:
AssetSync.configure do |config|
  # Disable automatic run on precompile in order to attach to webpacker rake task
  config.run_on_precompile = false
  # The block should return an array of file paths
  config.add_local_file_paths do
    # Support webpacker assets
    public_root = Rails.root.join("public")
    Dir.chdir(public_root) do
      packs_dir = Webpacker.config.public_output_path.relative_path_from(public_root)
      Dir[File.join(packs_dir, '/**/**')]
    end
  end
end
  1. Add a asset_sync.rake in your lib/tasks directory that enhances the correct task, otherwise asset_sync runs before webpacker:compile does:
if defined?(AssetSync)
  Rake::Task['webpacker:compile'].enhance do
    Rake::Task["assets:sync"].invoke
  end
end

Caveat

By adding local files outside the normal Rails assets directory, the uploading part works, however checking that the asset was previously uploaded is not working because asset_sync is only fetching the files in the assets directory on the remote bucket. This will mean additional time used to upload the same assets again on every precompilation.

Running the specs

Make sure you have a .env file with these details:-

# for AWS provider
AWS_ACCESS_KEY_ID=<yourkeyid>
AWS_SECRET_ACCESS_KEY=<yoursecretkey>
FOG_DIRECTORY=<yourbucket>
FOG_REGION=<youbucketregion>

# for AzureRM provider
AZURE_STORAGE_ACCOUNT_NAME=<youraccountname>
AZURE_STORAGE_ACCESS_KEY=<youraccesskey>
FOG_DIRECTORY=<yourcontainer>
FOG_REGION=<yourcontainerregion>

Make sure the bucket has read/write permissions. Then to run the tests:-

foreman run rake

Todo

  1. Add some before and after filters for deleting and uploading
  2. Support more cloud storage providers
  3. Better test coverage
  4. Add rake tasks to clean old assets from a bucket

Credits

Inspired by:

License

MIT License. Copyright 2011-2013 Rumble Labs Ltd. rumblelabs.com


Author: AssetSync
Source code: https://github.com/AssetSync/asset_sync
License:

#ruby   #ruby-on-rails 

野村  太郎

野村 太郎

1598055060

Config ルール・自動修復を使ってS3全バケットのデフォルト暗号化を自動的に有効化する

AWS Config は AWSリソースの構成を記録、管理するためのサービスです。 EC2インスタンスやセキュリティグループなど、AWSリソースが「いつ、どのような変更がされたか」 の履歴を記録してくれます。

また、 AWS Config ルール を利用して AWSリソースの構成の評価ができます。 AWSリソースのあるべき構成を条件としています。条件に違反しているリソースが検出されると 「非準拠」フラグが付けられます。

さらに、この「非準拠」リソースを 修復 することもできます。 “AWS Systems Manager オートメーション(SSM オートメーション)” という自動化を行うためのワークフローと連携して、 「非準拠」リソースが見つかれば SSMオートメーションを使ってあるべき構成へ自動修復する、便利な自動化が実現できます。

本記事では S3の全バケットのデフォルト暗号化を自動的に有効化する ための Configルール/自動修復を 構築してみます。

img

目次

  1. やってみる(ルールの作成)
  2. やってみる(IAMロールの作成)
  3. やってみる(自動修復設定)
  4. 確認
  5. CloudFormation テンプレート
  6. おわりに
  7. 参考

やってみる(ルールの作成)

まずは、「デフォルト暗号化が有効になっていないバケットを非準拠とする」Configルールを作成します。 ※Configを有効化していることは前提です。

「AWS Config > ルール」の画面から「ルールを追加」を選択します。

img

AWS管理のルール s3-bucket-server-side-encryption-enabled を選択します。

img

#aws #config #aws systems manager

Sasha  Roberts

Sasha Roberts

1659372869

Split: The Rack Based A/B Testing Framework with Ruby on Rails

Split

     📈 The Rack Based A/B testing framework https://libraries.io/rubygems/split

Split is a rack based A/B testing framework designed to work with Rails, Sinatra or any other rack based app.

Split is heavily inspired by the Abingo and Vanity Rails A/B testing plugins and Resque in its use of Redis.

Split is designed to be hacker friendly, allowing for maximum customisation and extensibility.

Install

Requirements

Split v4.0+ is currently tested with Ruby >= 2.5 and Rails >= 5.2.

If your project requires compatibility with Ruby 2.4.x or older Rails versions. You can try v3.0 or v0.8.0(for Ruby 1.9.3)

Split uses Redis as a datastore.

Split only supports Redis 4.0 or greater.

If you're on OS X, Homebrew is the simplest way to install Redis:

brew install redis
redis-server /usr/local/etc/redis.conf

You now have a Redis daemon running on port 6379.

Setup

gem install split

Rails

Adding gem 'split' to your Gemfile will autoload it when rails starts up, as long as you've configured Redis it will 'just work'.

Sinatra

To configure Sinatra with Split you need to enable sessions and mix in the helper methods. Add the following lines at the top of your Sinatra app:

require 'split'

class MySinatraApp < Sinatra::Base
  enable :sessions
  helpers Split::Helper

  get '/' do
  ...
end

Usage

To begin your A/B test use the ab_test method, naming your experiment with the first argument and then the different alternatives which you wish to test on as the other arguments.

ab_test returns one of the alternatives, if a user has already seen that test they will get the same alternative as before, which you can use to split your code on.

It can be used to render different templates, show different text or any other case based logic.

ab_finished is used to make a completion of an experiment, or conversion.

Example: View

<% ab_test(:login_button, "/images/button1.jpg", "/images/button2.jpg") do |button_file| %>
  <%= image_tag(button_file, alt: "Login!") %>
<% end %>

Example: Controller

def register_new_user
  # See what level of free points maximizes users' decision to buy replacement points.
  @starter_points = ab_test(:new_user_free_points, '100', '200', '300')
end

Example: Conversion tracking (in a controller!)

def buy_new_points
  # some business logic
  ab_finished(:new_user_free_points)
end

Example: Conversion tracking (in a view)

Thanks for signing up, dude! <% ab_finished(:signup_page_redesign) %>

You can find more examples, tutorials and guides on the wiki.

Statistical Validity

Split has two options for you to use to determine which alternative is the best.

The first option (default on the dashboard) uses a z test (n>30) for the difference between your control and alternative conversion rates to calculate statistical significance. This test will tell you whether an alternative is better or worse than your control, but it will not distinguish between which alternative is the best in an experiment with multiple alternatives. Split will only tell you if your experiment is 90%, 95%, or 99% significant, and this test only works if you have more than 30 participants and 5 conversions for each branch.

As per this blog post on the pitfalls of A/B testing, it is highly recommended that you determine your requisite sample size for each branch before running the experiment. Otherwise, you'll have an increased rate of false positives (experiments which show a significant effect where really there is none).

Here is a sample size calculator for your convenience.

The second option uses simulations from a beta distribution to determine the probability that the given alternative is the winner compared to all other alternatives. You can view these probabilities by clicking on the drop-down menu labeled "Confidence." This option should be used when the experiment has more than just 1 control and 1 alternative. It can also be used for a simple, 2-alternative A/B test.

Calculating the beta-distribution simulations for a large number of experiments can be slow, so the results are cached. You can specify how often they should be recalculated (the default is once per day).

Split.configure do |config|
  config.winning_alternative_recalculation_interval = 3600 # 1 hour
end

Extras

Weighted alternatives

Perhaps you only want to show an alternative to 10% of your visitors because it is very experimental or not yet fully load tested.

To do this you can pass a weight with each alternative in the following ways:

ab_test(:homepage_design, {'Old' => 18}, {'New' => 2})

ab_test(:homepage_design, 'Old', {'New' => 1.0/9})

ab_test(:homepage_design, {'Old' => 9}, 'New')

This will only show the new alternative to visitors 1 in 10 times, the default weight for an alternative is 1.

Overriding alternatives

For development and testing, you may wish to force your app to always return an alternative. You can do this by passing it as a parameter in the url.

If you have an experiment called button_color with alternatives called red and blue used on your homepage, a url such as:

http://myawesomesite.com?ab_test[button_color]=red

will always have red buttons. This won't be stored in your session or count towards to results, unless you set the store_override configuration option.

In the event you want to disable all tests without having to know the individual experiment names, add a SPLIT_DISABLE query parameter.

http://myawesomesite.com?SPLIT_DISABLE=true

It is not required to send SPLIT_DISABLE=false to activate Split.

Rspec Helper

To aid testing with RSpec, write spec/support/split_helper.rb and call use_ab_test(alternatives_by_experiment) in your specs as instructed below:

# Create a file with these contents at 'spec/support/split_helper.rb'
# and ensure it is `require`d in your rails_helper.rb or spec_helper.rb
module SplitHelper

  # Force a specific experiment alternative to always be returned:
  #   use_ab_test(signup_form: "single_page")
  #
  # Force alternatives for multiple experiments:
  #   use_ab_test(signup_form: "single_page", pricing: "show_enterprise_prices")
  #
  def use_ab_test(alternatives_by_experiment)
    allow_any_instance_of(Split::Helper).to receive(:ab_test) do |_receiver, experiment, &block|
      variant = alternatives_by_experiment.fetch(experiment) { |key| raise "Unknown experiment '#{key}'" }
      block.call(variant) unless block.nil?
      variant
    end
  end
end

# Make the `use_ab_test` method available to all specs:
RSpec.configure do |config|
  config.include SplitHelper
end

Now you can call use_ab_test(alternatives_by_experiment) in your specs, for example:

it "registers using experimental signup" do
  use_ab_test experiment_name: "alternative_name"
  post "/signups"
  ...
end

Starting experiments manually

By default new A/B tests will be active right after deployment. In case you would like to start new test a while after the deploy, you can do it by setting the start_manually configuration option to true.

After choosing this option tests won't be started right after deploy, but after pressing the Start button in Split admin dashboard. If a test is deleted from the Split dashboard, then it can only be started after pressing the Start button whenever being re-initialized.

Reset after completion

When a user completes a test their session is reset so that they may start the test again in the future.

To stop this behaviour you can pass the following option to the ab_finished method:

ab_finished(:experiment_name, reset: false)

The user will then always see the alternative they started with.

Any old unfinished experiment key will be deleted from the user's data storage if the experiment had been removed or is over and a winner had been chosen. This allows a user to enroll into any new experiment in cases when the allow_multiple_experiments config option is set to false.

Reset experiments manually

By default Split automatically resets the experiment whenever it detects the configuration for an experiment has changed (e.g. you call ab_test with different alternatives). You can prevent this by setting the option reset_manually to true.

You may want to do this when you want to change something, like the variants' names, the metadata about an experiment, etc. without resetting everything.

Multiple experiments at once

By default Split will avoid users participating in multiple experiments at once. This means you are less likely to skew results by adding in more variation to your tests.

To stop this behaviour and allow users to participate in multiple experiments at once set the allow_multiple_experiments config option to true like so:

Split.configure do |config|
  config.allow_multiple_experiments = true
end

This will allow the user to participate in any number of experiments and belong to any alternative in each experiment. This has the possible downside of a variation in one experiment influencing the outcome of another.

To address this, setting the allow_multiple_experiments config option to 'control' like so:

Split.configure do |config|
  config.allow_multiple_experiments = 'control'
end

For this to work, each and every experiment you define must have an alternative named 'control'. This will allow the user to participate in multiple experiments as long as the user belongs to the alternative 'control' in each experiment. As soon as the user belongs to an alternative named something other than 'control' the user may not participate in any more experiments. Calling ab_test() will always return the first alternative without adding the user to that experiment.

Experiment Persistence

Split comes with three built-in persistence adapters for storing users and the alternatives they've been given for each experiment.

By default Split will store the tests for each user in the session.

You can optionally configure Split to use a cookie, Redis, or any custom adapter of your choosing.

Cookies

Split.configure do |config|
  config.persistence = :cookie
end

When using the cookie persistence, Split stores data into an anonymous tracking cookie named 'split', which expires in 1 year. To change that, set the persistence_cookie_length in the configuration (unit of time in seconds).

Split.configure do |config|
  config.persistence = :cookie
  config.persistence_cookie_length = 2592000 # 30 days
end

The data stored consists of the experiment name and the variants the user is in. Example: { "experiment_name" => "variant_a" }

Note: Using cookies depends on ActionDispatch::Cookies or any identical API

Redis

Using Redis will allow ab_users to persist across sessions or machines.

Split.configure do |config|
  config.persistence = Split::Persistence::RedisAdapter.with_config(lookup_by: -> (context) { context.current_user_id })
  # Equivalent
  # config.persistence = Split::Persistence::RedisAdapter.with_config(lookup_by: :current_user_id)
end

Options:

  • lookup_by: method to invoke per request for uniquely identifying ab_users (mandatory configuration)
  • namespace: separate namespace to store these persisted values (default "persistence")
  • expire_seconds: sets TTL for user key. (if a user is in multiple experiments most recent update will reset TTL for all their assignments)

Dual Adapter

The Dual Adapter allows the use of different persistence adapters for logged-in and logged-out users. A common use case is to use Redis for logged-in users and Cookies for logged-out users.

cookie_adapter = Split::Persistence::CookieAdapter
redis_adapter = Split::Persistence::RedisAdapter.with_config(
    lookup_by: -> (context) { context.send(:current_user).try(:id) },
    expire_seconds: 2592000)

Split.configure do |config|
  config.persistence = Split::Persistence::DualAdapter.with_config(
      logged_in: -> (context) { !context.send(:current_user).try(:id).nil? },
      logged_in_adapter: redis_adapter,
      logged_out_adapter: cookie_adapter)
  config.persistence_cookie_length = 2592000 # 30 days
end

Custom Adapter

Your custom adapter needs to implement the same API as existing adapters. See Split::Persistence::CookieAdapter or Split::Persistence::SessionAdapter for a starting point.

Split.configure do |config|
  config.persistence = YourCustomAdapterClass
end

Trial Event Hooks

You can define methods that will be called at the same time as experiment alternative participation and goal completion.

For example:

Split.configure do |config|
  config.on_trial  = :log_trial # run on every trial
  config.on_trial_choose   = :log_trial_choose # run on trials with new users only
  config.on_trial_complete = :log_trial_complete
end

Set these attributes to a method name available in the same context as the ab_test method. These methods should accept one argument, a Trial instance.

def log_trial(trial)
  logger.info "experiment=%s alternative=%s user=%s" %
    [ trial.experiment.name, trial.alternative, current_user.id ]
end

def log_trial_choose(trial)
  logger.info "[new user] experiment=%s alternative=%s user=%s" %
    [ trial.experiment.name, trial.alternative, current_user.id ]
end

def log_trial_complete(trial)
  logger.info "experiment=%s alternative=%s user=%s complete=true" %
    [ trial.experiment.name, trial.alternative, current_user.id ]
end

Views

If you are running ab_test from a view, you must define your event hook callback as a helper_method in the controller:

helper_method :log_trial_choose

def log_trial_choose(trial)
  logger.info "experiment=%s alternative=%s user=%s" %
    [ trial.experiment.name, trial.alternative, current_user.id ]
end

Experiment Hooks

You can assign a proc that will be called when an experiment is reset or deleted. You can use these hooks to call methods within your application to keep data related to experiments in sync with Split.

For example:

Split.configure do |config|
  # after experiment reset or deleted
  config.on_experiment_reset  = -> (example) { # Do something on reset }
  config.on_experiment_delete = -> (experiment) { # Do something else on delete }
  # before experiment reset or deleted
  config.on_before_experiment_reset  = -> (example) { # Do something on reset }
  config.on_before_experiment_delete = -> (experiment) { # Do something else on delete }
  # after experiment winner had been set
  config.on_experiment_winner_choose = -> (experiment) { # Do something on winner choose }
end

Web Interface

Split comes with a Sinatra-based front end to get an overview of how your experiments are doing.

If you are running Rails 2: You can mount this inside your app using Rack::URLMap in your config.ru

require 'split/dashboard'

run Rack::URLMap.new \
  "/"       => Your::App.new,
  "/split" => Split::Dashboard.new

However, if you are using Rails 3 or higher: You can mount this inside your app routes by first adding this to the Gemfile:

gem 'split', require: 'split/dashboard'

Then adding this to config/routes.rb

mount Split::Dashboard, at: 'split'

You may want to password protect that page, you can do so with Rack::Auth::Basic (in your split initializer file)

# Rails apps or apps that already depend on activesupport
Split::Dashboard.use Rack::Auth::Basic do |username, password|
  # Protect against timing attacks:
  # - Use & (do not use &&) so that it doesn't short circuit.
  # - Use digests to stop length information leaking
  ActiveSupport::SecurityUtils.secure_compare(::Digest::SHA256.hexdigest(username), ::Digest::SHA256.hexdigest(ENV["SPLIT_USERNAME"])) &
    ActiveSupport::SecurityUtils.secure_compare(::Digest::SHA256.hexdigest(password), ::Digest::SHA256.hexdigest(ENV["SPLIT_PASSWORD"]))
end

# Apps without activesupport
Split::Dashboard.use Rack::Auth::Basic do |username, password|
  # Protect against timing attacks:
  # - Use & (do not use &&) so that it doesn't short circuit.
  # - Use digests to stop length information leaking
  Rack::Utils.secure_compare(::Digest::SHA256.hexdigest(username), ::Digest::SHA256.hexdigest(ENV["SPLIT_USERNAME"])) &
    Rack::Utils.secure_compare(::Digest::SHA256.hexdigest(password), ::Digest::SHA256.hexdigest(ENV["SPLIT_PASSWORD"]))
end

You can even use Devise or any other Warden-based authentication method to authorize users. Just replace mount Split::Dashboard, :at => 'split' in config/routes.rb with the following:

match "/split" => Split::Dashboard, anchor: false, via: [:get, :post, :delete], constraints: -> (request) do
  request.env['warden'].authenticated? # are we authenticated?
  request.env['warden'].authenticate! # authenticate if not already
  # or even check any other condition such as request.env['warden'].user.is_admin?
end

More information on this here

Screenshot

split_screenshot

Configuration

You can override the default configuration options of Split like so:

Split.configure do |config|
  config.db_failover = true # handle Redis errors gracefully
  config.db_failover_on_db_error = -> (error) { Rails.logger.error(error.message) }
  config.allow_multiple_experiments = true
  config.enabled = true
  config.persistence = Split::Persistence::SessionAdapter
  #config.start_manually = false ## new test will have to be started manually from the admin panel. default false
  #config.reset_manually = false ## if true, it never resets the experiment data, even if the configuration changes
  config.include_rails_helper = true
  config.redis = "redis://custom.redis.url:6380"
end

Split looks for the Redis host in the environment variable REDIS_URL then defaults to redis://localhost:6379 if not specified by configure block.

On platforms like Heroku, Split will use the value of REDIS_PROVIDER to determine which env variable key to use when retrieving the host config. This defaults to REDIS_URL.

Filtering

In most scenarios you don't want to have AB-Testing enabled for web spiders, robots or special groups of users. Split provides functionality to filter this based on a predefined, extensible list of bots, IP-lists or custom exclude logic.

Split.configure do |config|
  # bot config
  config.robot_regex = /my_custom_robot_regex/ # or
  config.bots['newbot'] = "Description for bot with 'newbot' user agent, which will be added to config.robot_regex for exclusion"

  # IP config
  config.ignore_ip_addresses << '81.19.48.130' # or regex: /81\.19\.48\.[0-9]+/

  # or provide your own filter functionality, the default is proc{ |request| is_robot? || is_ignored_ip_address? || is_preview? }
  config.ignore_filter = -> (request) { CustomExcludeLogic.excludes?(request) }
end

Experiment configuration

Instead of providing the experiment options inline, you can store them in a hash. This hash can control your experiment's alternatives, weights, algorithm and if the experiment resets once finished:

Split.configure do |config|
  config.experiments = {
    my_first_experiment: {
      alternatives: ["a", "b"],
      resettable: false
    },
    :my_second_experiment => {
      algorithm: 'Split::Algorithms::Whiplash',
      alternatives: [
        { name: "a", percent: 67 },
        { name: "b", percent: 33 }
      ]
    }
  }
end

You can also store your experiments in a YAML file:

Split.configure do |config|
  config.experiments = YAML.load_file "config/experiments.yml"
end

You can then define the YAML file like:

my_first_experiment:
  alternatives:
    - a
    - b
my_second_experiment:
  alternatives:
    - name: a
      percent: 67
    - name: b
      percent: 33
  resettable: false

This simplifies the calls from your code:

ab_test(:my_first_experiment)

and:

ab_finished(:my_first_experiment)

You can also add meta data for each experiment, which is very useful when you need more than an alternative name to change behaviour:

Split.configure do |config|
  config.experiments = {
    my_first_experiment: {
      alternatives: ["a", "b"],
      metadata: {
        "a" => {"text" => "Have a fantastic day"},
        "b" => {"text" => "Don't get hit by a bus"}
      }
    }
  }
end
my_first_experiment:
  alternatives:
    - a
    - b
  metadata:
    a:
      text: "Have a fantastic day"
    b:
      text: "Don't get hit by a bus"

This allows for some advanced experiment configuration using methods like:

trial.alternative.name # => "a"

trial.metadata['text'] # => "Have a fantastic day"

or in views:

<% ab_test("my_first_experiment") do |alternative, meta| %>
  <%= alternative %>
  <small><%= meta['text'] %></small>
<% end %>

The keys used in meta data should be Strings

Metrics

You might wish to track generic metrics, such as conversions, and use those to complete multiple different experiments without adding more to your code. You can use the configuration hash to do this, thanks to the :metric option.

Split.configure do |config|
  config.experiments = {
    my_first_experiment: {
      alternatives: ["a", "b"],
      metric: :my_metric
    }
  }
end

Your code may then track a completion using the metric instead of the experiment name:

ab_finished(:my_metric)

You can also create a new metric by instantiating and saving a new Metric object.

Split::Metric.new(:my_metric)
Split::Metric.save

Goals

You might wish to allow an experiment to have multiple, distinguishable goals. The API to define goals for an experiment is this:

ab_test({link_color: ["purchase", "refund"]}, "red", "blue")

or you can define them in a configuration file:

Split.configure do |config|
  config.experiments = {
    link_color: {
      alternatives: ["red", "blue"],
      goals: ["purchase", "refund"]
    }
  }
end

To complete a goal conversion, you do it like:

ab_finished(link_color: "purchase")

Note that if you pass additional options, that should be a separate hash:

ab_finished({ link_color: "purchase" }, reset: false)

NOTE: This does not mean that a single experiment can complete more than one goal.

Once you finish one of the goals, the test is considered to be completed, and finishing the other goal will no longer register. (Assuming the test runs with reset: false.)

Good Example: Test if listing Plan A first result in more conversions to Plan A (goal: "plana_conversion") or Plan B (goal: "planb_conversion").

Bad Example: Test if button color increases conversion rate through multiple steps of a funnel. THIS WILL NOT WORK.

Bad Example: Test both how button color affects signup and how it affects login, at the same time. THIS WILL NOT WORK.

Combined Experiments

If you want to test how button color affects signup and how it affects login at the same time, use combined experiments. Configure like so:

  Split.configuration.experiments = {
        :button_color_experiment => {
          :alternatives => ["blue", "green"],
          :combined_experiments => ["button_color_on_signup", "button_color_on_login"]
        }
      }

Starting the combined test starts all combined experiments

 ab_combined_test(:button_color_experiment)

Finish each combined test as normal

   ab_finished(:button_color_on_login)
   ab_finished(:button_color_on_signup)

Additional Configuration:

  • Be sure to enable allow_multiple_experiments
  • In Sinatra include the CombinedExperimentsHelper
  helpers Split::CombinedExperimentsHelper

DB failover solution

Due to the fact that Redis has no automatic failover mechanism, it's possible to switch on the db_failover config option, so that ab_test and ab_finished will not crash in case of a db failure. ab_test always delivers alternative A (the first one) in that case.

It's also possible to set a db_failover_on_db_error callback (proc) for example to log these errors via Rails.logger.

Redis

You may want to change the Redis host and port Split connects to, or set various other options at startup.

Split has a redis setter which can be given a string or a Redis object. This means if you're already using Redis in your app, Split can re-use the existing connection.

String: Split.redis = 'redis://localhost:6379'

Redis: Split.redis = $redis

For our rails app we have a config/initializers/split.rb file where we load config/split.yml by hand and set the Redis information appropriately.

Here's our config/split.yml:

development: redis://localhost:6379
test: redis://localhost:6379
staging: redis://redis1.example.com:6379
fi: redis://localhost:6379
production: redis://redis1.example.com:6379

And our initializer:

split_config = YAML.load_file(Rails.root.join('config', 'split.yml'))
Split.redis = split_config[Rails.env]

Redis Caching (v4.0+)

In some high-volume usage scenarios, Redis load can be incurred by repeated fetches for fairly static data. Enabling caching will reduce this load.

Split.configuration.cache = true

This currently caches:

  • Split::ExperimentCatalog.find
  • Split::Experiment.start_time
  • Split::Experiment.winner

Namespaces

If you're running multiple, separate instances of Split you may want to namespace the keyspaces so they do not overlap. This is not unlike the approach taken by many memcached clients.

This feature can be provided by the redis-namespace library. To configure Split to use Redis::Namespace, do the following:

  1. Add redis-namespace to your Gemfile:
gem 'redis-namespace'
  1. Configure Split.redis to use a Redis::Namespace instance (possible in an initializer):
redis = Redis.new(url: ENV['REDIS_URL']) # or whatever config you want
Split.redis = Redis::Namespace.new(:your_namespace, redis: redis)

Outside of a Web Session

Split provides the Helper module to facilitate running experiments inside web sessions.

Alternatively, you can access the underlying Metric, Trial, Experiment and Alternative objects to conduct experiments that are not tied to a web session.

# create a new experiment
experiment = Split::ExperimentCatalog.find_or_create('color', 'red', 'blue')
# create a new trial
trial = Split::Trial.new(:experiment => experiment)
# run trial
trial.choose!
# get the result, returns either red or blue
trial.alternative.name

# if the goal has been achieved, increment the successful completions for this alternative.
if goal_achieved?
  trial.complete!
end

Algorithms

By default, Split ships with Split::Algorithms::WeightedSample that randomly selects from possible alternatives for a traditional a/b test. It is possible to specify static weights to favor certain alternatives.

Split::Algorithms::Whiplash is an implementation of a multi-armed bandit algorithm. This algorithm will automatically weight the alternatives based on their relative performance, choosing the better-performing ones more often as trials are completed.

Split::Algorithms::BlockRandomization is an algorithm that ensures equal participation across all alternatives. This algorithm will choose the alternative with the fewest participants. In the event of multiple minimum participant alternatives (i.e. starting a new "Block") the algorithm will choose a random alternative from those minimum participant alternatives.

Users may also write their own algorithms. The default algorithm may be specified globally in the configuration file, or on a per experiment basis using the experiments hash of the configuration file.

To change the algorithm globally for all experiments, use the following in your initializer:

Split.configure do |config|
  config.algorithm = Split::Algorithms::Whiplash
end

Extensions

Screencast

Ryan bates has produced an excellent 10 minute screencast about split on the Railscasts site: A/B Testing with Split

Blogposts

Backers

Support us with a monthly donation and help us continue our activities. [Become a backer]

Sponsors

Become a sponsor and get your logo on our README on Github with a link to your site. [Become a sponsor]

Contribute

Please do! Over 70 different people have contributed to the project, you can see them all here: https://github.com/splitrb/split/graphs/contributors.

Development

The source code is hosted at GitHub.

Report issues and feature requests on GitHub Issues.

You can find a discussion form on Google Groups.

Tests

Run the tests like this:

# Start a Redis server in another tab.
redis-server

bundle
rake spec

A Note on Patches and Pull Requests

  • Fork the project.
  • Make your feature addition or bug fix.
  • Add tests for it. This is important so I don't break it in a future version unintentionally.
  • Add documentation if necessary.
  • Commit. Do not mess with the rakefile, version, or history. (If you want to have your own version, that is fine. But bump the version in a commit by itself, which I can ignore when I pull.)
  • Send a pull request. Bonus points for topic branches.

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.


Author: splitrb
Source code: https://github.com/splitrb/split
License: MIT license

#ruby  #ruby-on-rails #redis 

Config: A Lightweight Yet Powerful Config Package for Go Projects

Config

GoLobby Config is a lightweight yet powerful configuration manager for Go projects. It takes advantage of Dot-env (.env) files and OS environment variables alongside config files (JSON, YAML, and TOML) to meet all of your requirements.

Documentation

Required Go Version

It requires Go v1.11 or newer versions.

Installation

To install this package run the following command in the root of your project.

go get github.com/golobby/config/v3

Quick Start

The following example demonstrates how to use a JSON configuration file.

// The configuration struct
type MyConfig struct {
    App struct {
        Name string
        Port int
    }
    Debug      bool
    Production bool
    Pi         float64
}

// Create an instance of the configuration struct
myConfig := MyConfig{}

// Create a feeder that provides the configuration data from a JSON file
jsonFeeder := feeder.Json{Path: "config.json"}

// Create a Config instance and feed `myConfig` using `jsonFeeder`
c := config.New()
c.AddFeeder(jsonFeeder)
c.AddStruct(&myConfig)
err := c.Feed()

// Or use method chaining:
// err := config.New().AddFeeder(jsonFeeder).AddStruct(&myConfig).Feed()

// Use `myConfig`...

Feeders

Feeders provide the configuration data. The GoLobby Config package supports the following feeders out of the box.

  • Json: It feeds using a JSON file.
  • Yaml: It feeds using a YAML file.
  • Toml: It feeds using a TOML file.
  • DotEnv: It feeds using a dot env (.env) file.
  • Env: It feeds using OS environment variables.

You can also create your custom feeders by implementing the Feeder interface or use third-party feeders.

Json Feeder

The Json feeder uses Go built-in json package to load JSON files. The snippet below shows how to use the Json feeder.

jsonFeeder := feeder.Json{Path: "sample1.json"}
c := config.New().AddFeeder(jsonFeeder)

Yaml Feeder

The Yaml feeder uses the YAML package (v3) to load YAML files. The snippet below shows how to use the Yaml feeder.

yamlFeeder := feeder.Yaml{Path: "sample1.yaml"}
c := config.New().AddFeeder(yamlFeeder)

Toml Feeder

The Toml feeder uses the BurntSushi TOML package to load TOML files. The snippet below shows how to use the Toml feeder.

tomlFeeder := feeder.Toml{Path: "sample1.toml"}
c := config.New().AddFeeder(tomlFeeder)

DotEnv Feeder

The DotEnv feeder uses the GoLobby DotEnv package to load .env files. The example below shows how to use the DotEnv feeder.

The .env file: https://github.com/golobby/config/blob/v3/assets/.env.sample1

type MyConfig struct {
    App struct {
        Name string `env:"APP_NAME"`
        Port int    `env:"APP_PORT"`
    }
    Debug      bool    `env:"DEBUG"`
    Production bool    `env:"PRODUCTION"`
    Pi         float64 `env:"PI"`
    IDs        []int   `env:"IDS"`
}

myConfig := MyConfig{}
dotEnvFeeder := feeder.DotEnv{Path: ".env"}
err := config.New().AddFeeder(dotEnvFeeder).AddStruct(&myConfig).Feed()

You must add a env tag for each field that determines the related dot env variable. If there isn't any value for a field in the related file, it ignores the struct field. You can read more about this feeder in the GoLobby DotEnv package repository.

Env Feeder

The Env feeder is built on top of the GoLobby Env package. The example below shows how to use the Env feeder.

_ = os.Setenv("APP_NAME", "Shop")
_ = os.Setenv("APP_PORT", "8585")
_ = os.Setenv("DEBUG", "true")
_ = os.Setenv("PRODUCTION", "false")
_ = os.Setenv("PI", "3.14")
_ = os.Setenv("IPS", "192.168.0.1", "192.168.0.2")
_ = os.Setenv("IDS", "10, 11, 12, 13")

type MyConfig struct {
    App struct {
        Name string `env:"APP_NAME"`
        Port int    `env:"APP_PORT"`
    }
    Debug      bool     `env:"DEBUG"`
    Production bool     `env:"PRODUCTION"`
    Pi         float64  `env:"PI"`
    IPs        []string `env:"IPS"`
    IDs        []int16  `env:"IDS"`
}

myConfig := MyConfig{}
envFeeder := feeder.DotEnv{}
err := config.New().AddFeeder(envFeeder).AddStruct(&myConfig).Feed()

You must add a env tag for each field that determines the related OS environment variable name. If there isn't any value for a field in OS environment variables, it ignores the struct field. You can read more about this feeder in the GoLobby Env package repository.

Multiple Feeders

One of the key features in the GoLobby Config package is feeding using multiple feeders. Lately added feeders overrides early added ones.

The example below demonstrates how to use a JSON file as the main configuration feeder and override the configurations with dot env and os variables.

_ = os.Setenv("PRODUCTION", "true")
_ = os.Setenv("APP_PORT", "6969")
_ = os.Setenv("IDs", "6, 9")

type MyConfig struct {
    App struct {
        Name string `env:"APP_NAME"`
        Port int    `env:"APP_PORT"`
    }
    Debug      bool    `env:"DEBUG"`
    Production bool    `env:"PRODUCTION"`
    Pi         float64 `env:"PI"`
    IDs        []int32 `env:"IDS"`
}

myConfig := MyConfig{}

feeder1 := feeder.Json{Path: "sample1.json"}
feeder2 := feeder.DotEnv{Path: ".env.sample2"}
feeder3 := feeder.Env{}

err := config.New()
        .AddFeeder(feeder1)
        .AddFeeder(feeder2)
        .AddFeeder(feeder3)
        .AddStruct(&myConfig)
        .Feed()

fmt.Println(c.App.Name)   // Blog  [from DotEnv]
fmt.Println(c.App.Port)   // 6969  [from Env]
fmt.Println(c.Debug)      // false [from DotEnv]
fmt.Println(c.Production) // true  [from Env]
fmt.Println(c.Pi)         // 3.14  [from Json]
fmt.Println(c.IDs)        // 6, 9  [from Env]

What happened?

  • The Json feeder as the first feeder sets all the struct fields from the JSON file.
  • The DotEnv feeder as the second feeder overrides existing fields. The APP_NAME and DEBUG fields exist in the .env.sample2 file.
  • The Env feeder as the last feeder overrides existing fields, as well. The APP_PORT and PRODUCTION fields are defined in the OS environment.

Re-feed

You can re-feed the structs every time you need to. Just call the Feed() method again.

c := config.New().AddFeeder(feeder).AddStruct(&myConfig)
err := c.Feed()

// Is it time to re-feed?
err = c.Feed()

// Use `myConfig` with updated data!

Listener

One of the GoLobby Config features is the ability to update the configuration structs without redeployment. It takes advantage of OS signals to handle this requirement. Config instances listen to the "SIGHUP" operating system signal and refresh structs (call the Feed() method).

To enable the listener for a Config instance, you should call the SetupListener() method. It gets a fallback function and calls it when the Feed() method fails and returns an error.

c := config.New().AddFeeder(feeder).AddStruct(&myConfig)
c.SetupListener(func(err error) {
    fmt.Println(err)
})

err := c.Feed()

You can send the SIGHUP signal to your running application with the following shell command.

KILL -SIGHUP [YOUR-APP-PROCESS-ID]

You can get your application process ID using the ps command.

See Also

  • GoLobby/DotEnv: A lightweight package for loading dot env (.env) files into structs for Go projects
  • GoLobby/Env: A lightweight package for loading OS environment variables into structs for Go projects
  • GoLobby/Container: A lightweight yet powerful IoC dependency injection container for Go projects

Author: Golobby
Source Code: https://github.com/golobby/config  
License: MIT License

#go #golang #cli #config 

Mike  Kozey

Mike Kozey

1616437920

Setting Up A Spring Cloud Config Client

Learn how to create a configuration client which can communicate with a configuration server to access a git repository and fetch configuration files.

This post is a continuation of another post of mine,  “How to Setup the Spring Cloud Config Server With Git,” i.e., setting up of Spring cloud config server with a git repository. In my earlier post, successful communication was established between a cloud config server and a git repository. In this post, the focus will be on the development of a cloud config client and its interaction with a cloud config server.

Prerequisites

  • Java 1.8+
  • Maven
  • Eclipse with Spring Tools Suite (STS)
  • Git
  • Spring Cloud Config Server

#microservices #spring boot #spring config #spring config client