Gentoo Kernel Config | How I Configure My Gentoo Kernel

From popular demand, I created a video on Gentoo kernel configuration and how I configure my kernel. One thing I forgot to mention in the video is, you can use lspci -k on an existing system to see which kernel drivers are being used.

Find me elsewhere:
Instagram: https://www.instagram.com/kamiyaa91/
Github: https://github.com/kamiyaa/

Late Bloomers by Rich Karlgaard: https://amzn.to/2DcmCLo
Disclaimer, this is an affiliate link

Gentoo provides a variety of kernel sources to choose from.
The most popular being gentoo-sources. I personally prefer vanilla-sources which is the Linux kernel without any 3rd party patches.

Kernel source code is located under /usr/src in gentoo.
Kernel configuration is written to a file called .config (ie. /usr/src/linux-5.7.12/.config).

You can copy this config file around to keep your configuration. But to ensure the configuration is up-to-date, make sure to use make oldconfig whenever you are using that config on a newer kernel.
Certain kernel options may not be visible in menuconfig. This is usually because the prerequisites for the option has not been satisfied. So make sure all requirements are met by searching for that option and reading what it is SELECTED_BY.

If you have drivers that require firmware such as wireless drivers or graphics drivers, you have 2 options:

1. Compile them as built-in to the kernel. For this, you would need to go under Device Drivers) Generic Driver Options) Firmware loader) Build named firmware blobs into the kernel binary) and change its value to /lib/firmware. You can read more about it here: https://wiki.gentoo.org/wiki/Linux_firmware#Kernel
2. Compile them as modules, which is what I do.

Under Raid/LVM options is actually the options required for encryption software such as dm-crypt. So if you are thinking of encrypting your data, enable Device Mapper Support under Device Drivers) Multiple devices driver support (RAID and LVM))

X86 Platform Specific Device Drivers are the drivers that enable special keys on laptops and other keyboards like volume keys, brightness keys, etc.

The FUSE kernel option is required if you want to connect android phones to your system via MTP protocol. You can read more here: https://wiki.gentoo.org/wiki/MTP

Hope you guys enjoy the video!

Music:
Imagine by lukrembo
Michikusa by PeriTune (Licensed under https://creativecommons.org/licenses/by/4.0/)

#gentoo #gentoo kernel

What is GEEK

Buddha Community

Background Fetch for React Native Apps

react-native-background-fetch

Background Fetch is a very simple plugin which attempts to awaken an app in the background about every 15 minutes, providing a short period of background running-time. This plugin will execute your provided callbackFn whenever a background-fetch event occurs.

There is no way to increase the rate which a fetch-event occurs and this plugin sets the rate to the most frequent possible — you will never receive an event faster than 15 minutes. The operating-system will automatically throttle the rate the background-fetch events occur based upon usage patterns. Eg: if user hasn't turned on their phone for a long period of time, fetch events will occur less frequently or if an iOS user disables background refresh they may not happen at all.

:new: Background Fetch now provides a scheduleTask method for scheduling arbitrary "one-shot" or periodic tasks.

iOS

• There is no way to increase the rate which a fetch-event occurs and this plugin sets the rate to the most frequent possible — you will never receive an event faster than 15 minutes. The operating-system will automatically throttle the rate the background-fetch events occur based upon usage patterns. Eg: if user hasn't turned on their phone for a long period of time, fetch events will occur less frequently.
• scheduleTask seems only to fire when the device is plugged into power.
• ⚠️ When your app is terminated, iOS no longer fires events — There is no such thing as stopOnTerminate: false for iOS.
• iOS can take days before Apple's machine-learning algorithm settles in and begins regularly firing events. Do not sit staring at your logs waiting for an event to fire. If your simulated events work, that's all you need to know that everything is correctly configured.
• If the user doesn't open your iOS app for long periods of time, iOS will stop firing events.

Android

• The Android plugin provides a HeadlessJS implementation allowing you to continue handling events even after app-termination (see @config enableHeadless)

Installing the plugin

⚠️ If you have a previous version of react-native-background-fetch < 2.7.0 installed into react-native >= 0.60, you should first unlink your previous version as react-native link is no longer required.

$ react-native unlink react-native-background-fetch

With yarn

$ yarn add react-native-background-fetch

With npm

$ npm install --save react-native-background-fetch

Setup Guides

iOS Setup

react-native >= 0.60

• Auto-linking Setup

Android Setup

react-native >= 0.60

• Auto-linking Setup

Example

ℹ️ This repo contains its own Example App. See /example

import React from 'react';
import {
  SafeAreaView,
  StyleSheet,
  ScrollView,
  View,
  Text,
  FlatList,
  StatusBar,
} from 'react-native';

import {
  Header,
  Colors
} from 'react-native/Libraries/NewAppScreen';

import BackgroundFetch from "react-native-background-fetch";

class App extends React.Component {
  constructor(props) {
    super(props);
    this.state = {
      events: []
    };
  }

  componentDidMount() {
    // Initialize BackgroundFetch ONLY ONCE when component mounts.
    this.initBackgroundFetch();
  }

  async initBackgroundFetch() {
    // BackgroundFetch event handler.
    const onEvent = async (taskId) => {
      console.log('[BackgroundFetch] task: ', taskId);
      // Do your background work...
      await this.addEvent(taskId);
      // IMPORTANT:  You must signal to the OS that your task is complete.
      BackgroundFetch.finish(taskId);
    }

    // Timeout callback is executed when your Task has exceeded its allowed running-time.
    // You must stop what you're doing immediately BackgroundFetch.finish(taskId)
    const onTimeout = async (taskId) => {
      console.warn('[BackgroundFetch] TIMEOUT task: ', taskId);
      BackgroundFetch.finish(taskId);
    }

    // Initialize BackgroundFetch only once when component mounts.
    let status = await BackgroundFetch.configure({minimumFetchInterval: 15}, onEvent, onTimeout);

    console.log('[BackgroundFetch] configure status: ', status);
  }

  // Add a BackgroundFetch event to <FlatList>
  addEvent(taskId) {
    // Simulate a possibly long-running asynchronous task with a Promise.
    return new Promise((resolve, reject) => {
      this.setState(state => ({
        events: [...state.events, {
          taskId: taskId,
          timestamp: (new Date()).toString()
        }]
      }));
      resolve();
    });
  }

  render() {
    return (
      <>
        <StatusBar barStyle="dark-content" />
        <SafeAreaView>
          <ScrollView
            contentInsetAdjustmentBehavior="automatic"
            style={styles.scrollView}>
            <Header />

            <View style={styles.body}>
              <View style={styles.sectionContainer}>
                <Text style={styles.sectionTitle}>BackgroundFetch Demo</Text>
              </View>
            </View>
          </ScrollView>
          <View style={styles.sectionContainer}>
            <FlatList
              data={this.state.events}
              renderItem={({item}) => (<Text>[{item.taskId}]: {item.timestamp}</Text>)}
              keyExtractor={item => item.timestamp}
            />
          </View>
        </SafeAreaView>
      </>
    );
  }
}

const styles = StyleSheet.create({
  scrollView: {
    backgroundColor: Colors.lighter,
  },
  body: {
    backgroundColor: Colors.white,
  },
  sectionContainer: {
    marginTop: 32,
    paddingHorizontal: 24,
  },
  sectionTitle: {
    fontSize: 24,
    fontWeight: '600',
    color: Colors.black,
  },
  sectionDescription: {
    marginTop: 8,
    fontSize: 18,
    fontWeight: '400',
    color: Colors.dark,
  },
});

export default App;

Executing Custom Tasks

In addition to the default background-fetch task defined by BackgroundFetch.configure, you may also execute your own arbitrary "oneshot" or periodic tasks (iOS requires additional Setup Instructions). However, all events will be fired into the Callback provided to BackgroundFetch#configure:

⚠️ iOS:

• scheduleTask on iOS seems only to run when the device is plugged into power.
• scheduleTask on iOS are designed for low-priority tasks, such as purging cache files — they tend to be unreliable for mission-critical tasks. scheduleTask will never run as frequently as you want.
• The default fetch event is much more reliable and fires far more often.
• scheduleTask on iOS stop when the user terminates the app. There is no such thing as stopOnTerminate: false for iOS.

// Step 1:  Configure BackgroundFetch as usual.
let status = await BackgroundFetch.configure({
  minimumFetchInterval: 15
}, async (taskId) => {  // <-- Event callback
  // This is the fetch-event callback.
  console.log("[BackgroundFetch] taskId: ", taskId);

  // Use a switch statement to route task-handling.
  switch (taskId) {
    case 'com.foo.customtask':
      print("Received custom task");
      break;
    default:
      print("Default fetch task");
  }
  // Finish, providing received taskId.
  BackgroundFetch.finish(taskId);
}, async (taskId) => {  // <-- Task timeout callback
  // This task has exceeded its allowed running-time.
  // You must stop what you're doing and immediately .finish(taskId)
  BackgroundFetch.finish(taskId);
});

// Step 2:  Schedule a custom "oneshot" task "com.foo.customtask" to execute 5000ms from now.
BackgroundFetch.scheduleTask({
  taskId: "com.foo.customtask",
  forceAlarmManager: true,
  delay: 5000  // <-- milliseconds
});

API Documentation

Config

Common Options

@param {Integer} minimumFetchInterval [15]

The minimum interval in minutes to execute background fetch events. Defaults to 15 minutes. Note: Background-fetch events will never occur at a frequency higher than every 15 minutes. Apple uses a secret algorithm to adjust the frequency of fetch events, presumably based upon usage patterns of the app. Fetch events can occur less often than your configured minimumFetchInterval.

@param {Integer} delay (milliseconds)

ℹ️ Valid only for BackgroundFetch.scheduleTask. The minimum number of milliseconds in future that task should execute.

@param {Boolean} periodic [false]

ℹ️ Valid only for BackgroundFetch.scheduleTask. Defaults to false. Set true to execute the task repeatedly. When false, the task will execute just once.

Android Options

@config {Boolean} stopOnTerminate [true]

Set false to continue background-fetch events after user terminates the app. Default to true.

@config {Boolean} startOnBoot [false]

Set true to initiate background-fetch events when the device is rebooted. Defaults to false.

❗ NOTE: startOnBoot requires stopOnTerminate: false.

@config {Boolean} forceAlarmManager [false]

By default, the plugin will use Android's JobScheduler when possible. The JobScheduler API prioritizes for battery-life, throttling task-execution based upon device usage and battery level.

Configuring forceAlarmManager: true will bypass JobScheduler to use Android's older AlarmManager API, resulting in more accurate task-execution at the cost of higher battery usage.

let status = await BackgroundFetch.configure({
  minimumFetchInterval: 15,
  forceAlarmManager: true
}, async (taskId) => {  // <-- Event callback
  console.log("[BackgroundFetch] taskId: ", taskId);
  BackgroundFetch.finish(taskId);
}, async (taskId) => {  // <-- Task timeout callback
  // This task has exceeded its allowed running-time.
  // You must stop what you're doing and immediately .finish(taskId)
  BackgroundFetch.finish(taskId);
});
.
.
.
// And with with #scheduleTask
BackgroundFetch.scheduleTask({
  taskId: 'com.foo.customtask',
  delay: 5000,       // milliseconds
  forceAlarmManager: true,
  periodic: false
});

@config {Boolean} enableHeadless [false]

Set true to enable React Native's Headless JS mechanism, for handling fetch events after app termination.

• 📂 index.js (MUST BE IN index.js):

import BackgroundFetch from "react-native-background-fetch";

let MyHeadlessTask = async (event) => {
  // Get task id from event {}:
  let taskId = event.taskId;
  let isTimeout = event.timeout;  // <-- true when your background-time has expired.
  if (isTimeout) {
    // This task has exceeded its allowed running-time.
    // You must stop what you're doing immediately finish(taskId)
    console.log('[BackgroundFetch] Headless TIMEOUT:', taskId);
    BackgroundFetch.finish(taskId);
    return;
  }
  console.log('[BackgroundFetch HeadlessTask] start: ', taskId);

  // Perform an example HTTP request.
  // Important:  await asychronous tasks when using HeadlessJS.
  let response = await fetch('https://reactnative.dev/movies.json');
  let responseJson = await response.json();
  console.log('[BackgroundFetch HeadlessTask] response: ', responseJson);

  // Required:  Signal to native code that your task is complete.
  // If you don't do this, your app could be terminated and/or assigned
  // battery-blame for consuming too much time in background.
  BackgroundFetch.finish(taskId);
}

// Register your BackgroundFetch HeadlessTask
BackgroundFetch.registerHeadlessTask(MyHeadlessTask);

@config {integer} requiredNetworkType [BackgroundFetch.NETWORK_TYPE_NONE]

Set basic description of the kind of network your job requires.

If your job doesn't need a network connection, you don't need to use this option as the default value is BackgroundFetch.NETWORK_TYPE_NONE.

NetworkType	Description
BackgroundFetch.NETWORK_TYPE_NONE	This job doesn't care about network constraints, either any or none.
BackgroundFetch.NETWORK_TYPE_ANY	This job requires network connectivity.
BackgroundFetch.NETWORK_TYPE_CELLULAR	This job requires network connectivity that is a cellular network.
BackgroundFetch.NETWORK_TYPE_UNMETERED	This job requires network connectivity that is unmetered. Most WiFi networks are unmetered, as in "you can upload as much as you like".
BackgroundFetch.NETWORK_TYPE_NOT_ROAMING	This job requires network connectivity that is not roaming (being outside the country of origin)

@config {Boolean} requiresBatteryNotLow [false]

Specify that to run this job, the device's battery level must not be low.

This defaults to false. If true, the job will only run when the battery level is not low, which is generally the point where the user is given a "low battery" warning.

@config {Boolean} requiresStorageNotLow [false]

Specify that to run this job, the device's available storage must not be low.

This defaults to false. If true, the job will only run when the device is not in a low storage state, which is generally the point where the user is given a "low storage" warning.

@config {Boolean} requiresCharging [false]

Specify that to run this job, the device must be charging (or be a non-battery-powered device connected to permanent power, such as Android TV devices). This defaults to false.

@config {Boolean} requiresDeviceIdle [false]

When set true, ensure that this job will not run if the device is in active use.

The default state is false: that is, the for the job to be runnable even when someone is interacting with the device.

This state is a loose definition provided by the system. In general, it means that the device is not currently being used interactively, and has not been in use for some time. As such, it is a good time to perform resource heavy jobs. Bear in mind that battery usage will still be attributed to your application, and shown to the user in battery stats.

---

Methods

Method Name	Arguments	Returns	Notes
configure	{FetchConfig}, callbackFn, timeoutFn	Promise<BackgroundFetchStatus>	Configures the plugin's callbackFn and timeoutFn. This callback will fire each time a background-fetch event occurs in addition to events from #scheduleTask. The timeoutFn will be called when the OS reports your task is nearing the end of its allowed background-time.
scheduleTask	{TaskConfig}	Promise<boolean>	Executes a custom task. The task will be executed in the same Callback function provided to #configure.
status	callbackFn	Promise<BackgroundFetchStatus>	Your callback will be executed with the current status (Integer) 0: Restricted, 1: Denied, 2: Available. These constants are defined as BackgroundFetch.STATUS_RESTRICTED, BackgroundFetch.STATUS_DENIED, BackgroundFetch.STATUS_AVAILABLE (NOTE: Android will always return STATUS_AVAILABLE)
finish	String taskId	Void	You MUST call this method in your callbackFn provided to #configure in order to signal to the OS that your task is complete. iOS provides only 30s of background-time for a fetch-event -- if you exceed this 30s, iOS will kill your app.
start	none	Promise<BackgroundFetchStatus>	Start the background-fetch API. Your callbackFn provided to #configure will be executed each time a background-fetch event occurs. NOTE the #configure method automatically calls #start. You do not have to call this method after you #configure the plugin
stop	[taskId:String]	Promise<boolean>	Stop the background-fetch API and all #scheduleTask from firing events. Your callbackFn provided to #configure will no longer be executed. If you provide an optional taskId, only that #scheduleTask will be stopped.

Debugging

iOS

🆕 BGTaskScheduler API for iOS 13+

• ⚠️ At the time of writing, the new task simulator does not yet work in Simulator; Only real devices.
• See Apple docs Starting and Terminating Tasks During Development
• After running your app in XCode, Click the [||] button to initiate a Breakpoint.
• In the console (lldb), paste the following command (Note: use cursor up/down keys to cycle through previously run commands):

e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateLaunchForTaskWithIdentifier:@"com.transistorsoft.fetch"]

• Click the [ > ] button to continue. The task will execute and the Callback function provided to BackgroundFetch.configure will receive the event.

Simulating task-timeout events

• Only the new BGTaskScheduler api supports simulated task-timeout events. To simulate a task-timeout, your fetchCallback must not call BackgroundFetch.finish(taskId):

let status = await BackgroundFetch.configure({
  minimumFetchInterval: 15
}, async (taskId) => {  // <-- Event callback.
  // This is the task callback.
  console.log("[BackgroundFetch] taskId", taskId);
  //BackgroundFetch.finish(taskId); // <-- Disable .finish(taskId) when simulating an iOS task timeout
}, async (taskId) => {  // <-- Event timeout callback
  // This task has exceeded its allowed running-time.
  // You must stop what you're doing and immediately .finish(taskId)
  print("[BackgroundFetch] TIMEOUT taskId:", taskId);
  BackgroundFetch.finish(taskId);
});

• Now simulate an iOS task timeout as follows, in the same manner as simulating an event above:

e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateExpirationForTaskWithIdentifier:@"com.transistorsoft.fetch"]

Old BackgroundFetch API

• Simulate background fetch events in XCode using Debug->Simulate Background Fetch
• iOS can take some hours or even days to start a consistently scheduling background-fetch events since iOS schedules fetch events based upon the user's patterns of activity. If Simulate Background Fetch works, your can be sure that everything is working fine. You just need to wait.

Android

• Observe plugin logs in $ adb logcat:

$ adb logcat *:S ReactNative:V ReactNativeJS:V TSBackgroundFetch:V

• Simulate a background-fetch event on a device (insert <your.application.id>) (only works for sdk 21+:

$ adb shell cmd jobscheduler run -f <your.application.id> 999

• For devices with sdk <21, simulate a "Headless JS" event with (insert <your.application.id>)

$ adb shell am broadcast -a <your.application.id>.event.BACKGROUND_FETCH

Download Details:
Author: transistorsoft
Source Code: https://github.com/transistorsoft/react-native-background-fetch
License: MIT license

#react  #reactnative  #mobileapp  #javascript 

Gentoo Kernel Config | How I Configure My Gentoo Kernel

From popular demand, I created a video on Gentoo kernel configuration and how I configure my kernel. One thing I forgot to mention in the video is, you can use lspci -k on an existing system to see which kernel drivers are being used.

Find me elsewhere:
Instagram: https://www.instagram.com/kamiyaa91/
Github: https://github.com/kamiyaa/

Late Bloomers by Rich Karlgaard: https://amzn.to/2DcmCLo
Disclaimer, this is an affiliate link

Gentoo provides a variety of kernel sources to choose from.
The most popular being gentoo-sources. I personally prefer vanilla-sources which is the Linux kernel without any 3rd party patches.

Kernel source code is located under /usr/src in gentoo.
Kernel configuration is written to a file called .config (ie. /usr/src/linux-5.7.12/.config).

You can copy this config file around to keep your configuration. But to ensure the configuration is up-to-date, make sure to use make oldconfig whenever you are using that config on a newer kernel.
Certain kernel options may not be visible in menuconfig. This is usually because the prerequisites for the option has not been satisfied. So make sure all requirements are met by searching for that option and reading what it is SELECTED_BY.

If you have drivers that require firmware such as wireless drivers or graphics drivers, you have 2 options:

1. Compile them as built-in to the kernel. For this, you would need to go under Device Drivers) Generic Driver Options) Firmware loader) Build named firmware blobs into the kernel binary) and change its value to /lib/firmware. You can read more about it here: https://wiki.gentoo.org/wiki/Linux_firmware#Kernel
2. Compile them as modules, which is what I do.

Under Raid/LVM options is actually the options required for encryption software such as dm-crypt. So if you are thinking of encrypting your data, enable Device Mapper Support under Device Drivers) Multiple devices driver support (RAID and LVM))

X86 Platform Specific Device Drivers are the drivers that enable special keys on laptops and other keyboards like volume keys, brightness keys, etc.

The FUSE kernel option is required if you want to connect android phones to your system via MTP protocol. You can read more here: https://wiki.gentoo.org/wiki/MTP

Hope you guys enjoy the video!

Music:
Imagine by lukrembo
Michikusa by PeriTune (Licensed under https://creativecommons.org/licenses/by/4.0/)

#gentoo #gentoo kernel

Asset Sync: Synchronises Assets Between Rails and S3

Asset Sync

Synchronises Assets between Rails and S3.

Asset Sync is built to run with the new Rails Asset Pipeline feature introduced in Rails 3.1. After you run bundle exec rake assets:precompile your assets will be synchronised to your S3 bucket, optionally deleting unused files and only uploading the files it needs to.

This was initially built and is intended to work on Heroku but can work on any platform.

Upgrading?

Upgraded from 1.x? Read UPGRADING.md

Installation

Since 2.x, Asset Sync depends on gem fog-core instead of fog.
This is due to fog is including many unused storage provider gems as its dependencies.

Asset Sync has no idea about what provider will be used,
so you are responsible for bundling the right gem for the provider to be used.

In your Gemfile:

gem "asset_sync"
gem "fog-aws"

Or, to use Azure Blob storage, configure as this.

gem "asset_sync"
gem "gitlab-fog-azure-rm"

# This gem seems unmaintianed
# gem "fog-azure-rm"

To use Backblaze B2, insert these.

gem "asset_sync"
gem "fog-backblaze"

Extended Installation (Faster sync with turbosprockets)

It's possible to improve asset:precompile time if you are using Rails 3.2.x the main source of which being compilation of non-digest assets.

turbo-sprockets-rails3 solves this by only compiling digest assets. Thus cutting compile time in half.

NOTE: It will be deprecated in Rails 4 as sprockets-rails has been extracted out of Rails and will only compile digest assets by default.

Configuration

Rails

Configure config/environments/production.rb to use Amazon S3 as the asset host and ensure precompiling is enabled.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com"

Or, to use Google Storage Cloud, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['FOG_DIRECTORY']}.storage.googleapis.com"

Or, to use Azure Blob storage, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['AZURE_STORAGE_ACCOUNT_NAME']}.blob.core.windows.net/#{ENV['FOG_DIRECTORY']}"

Or, to use Backblaze B2, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//f000.backblazeb2.com/file/#{ENV['FOG_DIRECTORY']}"

On HTTPS: the exclusion of any protocol in the asset host declaration above will allow browsers to choose the transport mechanism on the fly. So if your application is available under both HTTP and HTTPS the assets will be served to match.

The only caveat with this is that your S3 bucket name must not contain any periods so, mydomain.com.s3.amazonaws.com for example would not work under HTTPS as SSL certificates from Amazon would interpret our bucket name as not a subdomain of s3.amazonaws.com, but a multi level subdomain. To avoid this don't use a period in your subdomain or switch to the other style of S3 URL.

  config.action_controller.asset_host = "//s3.amazonaws.com/#{ENV['FOG_DIRECTORY']}"

Or, to use Google Storage Cloud, configure as this.

  config.action_controller.asset_host = "//storage.googleapis.com/#{ENV['FOG_DIRECTORY']}"

Or, to use Azure Blob storage, configure as this.

  #config/environments/production.rb
  config.action_controller.asset_host = "//#{ENV['AZURE_STORAGE_ACCOUNT_NAME']}.blob.core.windows.net/#{ENV['FOG_DIRECTORY']}"

On non default S3 bucket region: If your bucket is set to a region that is not the default US Standard (us-east-1) you must use the first style of url //#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com or amazon will return a 301 permanently moved when assets are requested. Note the caveat above about bucket names and periods.

If you wish to have your assets sync to a sub-folder of your bucket instead of into the root add the following to your production.rb file

  # store assets in a 'folder' instead of bucket root
  config.assets.prefix = "/production/assets"

Also, ensure the following are defined (in production.rb or application.rb)

• config.assets.digest is set to true.
• config.assets.enabled is set to true.

Additionally, if you depend on any configuration that is setup in your initializers you will need to ensure that

• config.assets.initialize_on_precompile is set to true

AssetSync

AssetSync supports the following methods of configuration.

• Built-in Initializer (configured through environment variables)
• Rails Initializer
• A YAML config file

Using the Built-in Initializer is the default method and is supposed to be used with environment variables. It's the recommended approach for deployments on Heroku.

If you need more control over configuration you will want to use a custom rails initializer.

Configuration using a YAML file (a common strategy for Capistrano deployments) is also supported.

The recommend way to configure asset_sync is by using environment variables however it's up to you, it will work fine if you hard code them too. The main reason why using environment variables is recommended is so your access keys are not checked into version control.

Built-in Initializer (Environment Variables)

The Built-in Initializer will configure AssetSync based on the contents of your environment variables.

Add your configuration details to heroku

heroku config:add AWS_ACCESS_KEY_ID=xxxx
heroku config:add AWS_SECRET_ACCESS_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx
heroku config:add FOG_PROVIDER=AWS
# and optionally:
heroku config:add FOG_REGION=eu-west-1
heroku config:add ASSET_SYNC_GZIP_COMPRESSION=true
heroku config:add ASSET_SYNC_MANIFEST=true
heroku config:add ASSET_SYNC_EXISTING_REMOTE_FILES=keep

Or add to a traditional unix system

export AWS_ACCESS_KEY_ID=xxxx
export AWS_SECRET_ACCESS_KEY=xxxx
export FOG_DIRECTORY=xxxx

Rackspace configuration is also supported

heroku config:add RACKSPACE_USERNAME=xxxx
heroku config:add RACKSPACE_API_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx
heroku config:add FOG_PROVIDER=Rackspace

Google Storage Cloud configuration is supported as well. The preferred option is using the GCS JSON API which requires that you create an appropriate service account, generate the signatures and make them accessible to asset sync at the prescribed location

heroku config:add FOG_PROVIDER=Google
heroku config:add GOOGLE_PROJECT=xxxx
heroku config:add GOOGLE_JSON_KEY_LOCATION=xxxx
heroku config:add FOG_DIRECTORY=xxxx

If using the S3 API the following config is required

heroku config:add FOG_PROVIDER=Google
heroku config:add GOOGLE_STORAGE_ACCESS_KEY_ID=xxxx
heroku config:add GOOGLE_STORAGE_SECRET_ACCESS_KEY=xxxx
heroku config:add FOG_DIRECTORY=xxxx

The Built-in Initializer also sets the AssetSync default for existing_remote_files to keep.

Custom Rails Initializer (config/initializers/asset_sync.rb)

If you want to enable some of the advanced configuration options you will want to create your own initializer.

Run the included Rake task to generate a starting point.

rails g asset_sync:install --provider=Rackspace
rails g asset_sync:install --provider=AWS
rails g asset_sync:install --provider=AzureRM
rails g asset_sync:install --provider=Backblaze

The generator will create a Rails initializer at config/initializers/asset_sync.rb.

AssetSync.configure do |config|
  config.fog_provider = 'AWS'
  config.fog_directory = ENV['FOG_DIRECTORY']
  config.aws_access_key_id = ENV['AWS_ACCESS_KEY_ID']
  config.aws_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
  config.aws_session_token = ENV['AWS_SESSION_TOKEN'] if ENV.key?('AWS_SESSION_TOKEN')

  # Don't delete files from the store
  # config.existing_remote_files = 'keep'
  #
  # Increase upload performance by configuring your region
  # config.fog_region = 'eu-west-1'
  #
  # Set `public` option when uploading file depending on value,
  # Setting to "default" makes asset sync skip setting the option
  # Possible values: true, false, "default" (default: true)
  # config.fog_public = true
  #
  # Change AWS signature version. Default is 4
  # config.aws_signature_version = 4
  #
  # Change canned ACL of uploaded object. Default is unset. Will override fog_public if set.
  # Choose from: private | public-read | public-read-write | aws-exec-read |
  #              authenticated-read | bucket-owner-read | bucket-owner-full-control 
  # config.aws_acl = nil 
  #
  # Change host option in fog (only if you need to)
  # config.fog_host = 's3.amazonaws.com'
  #
  # Change port option in fog (only if you need to)
  # config.fog_port = "9000"
  #
  # Use http instead of https.
  # config.fog_scheme = 'http'
  #
  # Automatically replace files with their equivalent gzip compressed version
  # config.gzip_compression = true
  #
  # Use the Rails generated 'manifest.yml' file to produce the list of files to
  # upload instead of searching the assets directory.
  # config.manifest = true
  #
  # Upload the manifest file also.
  # config.include_manifest = false
  #
  # Upload files concurrently
  # config.concurrent_uploads = false
  #
  # Number of threads when concurrent_uploads is enabled
  # config.concurrent_uploads_max_threads = 10
  #
  # Path to cache file to skip scanning remote
  # config.remote_file_list_cache_file_path = './.asset_sync_remote_file_list_cache.json'
  #
  # Fail silently.  Useful for environments such as Heroku
  # config.fail_silently = true
  #
  # Log silently. Default is `true`. But you can set it to false if more logging message are preferred.
  # Logging messages are sent to `STDOUT` when `log_silently` is falsy
  # config.log_silently = true
  #
  # Allow custom assets to be cacheable. Note: The base filename will be matched
  # If you have an asset with name `app.0b1a4cd3.js`, only `app.0b1a4cd3` will need to be matched
  # only one of `cache_asset_regexp` or `cache_asset_regexps` is allowed.
  # config.cache_asset_regexp = /\.[a-f0-9]{8}$/i
  # config.cache_asset_regexps = [ /\.[a-f0-9]{8}$/i, /\.[a-f0-9]{20}$/i ]
end

YAML (config/asset_sync.yml)

Run the included Rake task to generate a starting point.

rails g asset_sync:install --use-yml --provider=Rackspace
rails g asset_sync:install --use-yml --provider=AWS
rails g asset_sync:install --use-yml --provider=AzureRM
rails g asset_sync:install --use-yml --provider=Backblaze

The generator will create a YAML file at config/asset_sync.yml.

defaults: &defaults
  fog_provider: "AWS"
  fog_directory: "rails-app-assets"
  aws_access_key_id: "<%= ENV['AWS_ACCESS_KEY_ID'] %>"
  aws_secret_access_key: "<%= ENV['AWS_SECRET_ACCESS_KEY'] %>"

  # To use AWS reduced redundancy storage.
  # aws_reduced_redundancy: true
  #
  # You may need to specify what region your storage bucket is in
  # fog_region: "eu-west-1"
  #
  # Change AWS signature version. Default is 4
  # aws_signature_version: 4
  #
  # Change canned ACL of uploaded object. Default is unset. Will override fog_public if set.
  # Choose from: private | public-read | public-read-write | aws-exec-read |
  #              authenticated-read | bucket-owner-read | bucket-owner-full-control 
  # aws_acl: null
  #
  # Change host option in fog (only if you need to)
  # fog_host: "s3.amazonaws.com"
  #
  # Use http instead of https. Default should be "https" (at least for fog-aws)
  # fog_scheme: "http"

  existing_remote_files: keep # Existing pre-compiled assets on S3 will be kept
  # To delete existing remote files.
  # existing_remote_files: delete
  # To ignore existing remote files and overwrite.
  # existing_remote_files: ignore
  # Automatically replace files with their equivalent gzip compressed version
  # gzip_compression: true
  # Fail silently.  Useful for environments such as Heroku
  # fail_silently: true
  # Always upload. Useful if you want to overwrite specific remote assets regardless of their existence
  #  eg: Static files in public often reference non-fingerprinted application.css
  #  note: You will still need to expire them from the CDN's edge cache locations
  # always_upload: ['application.js', 'application.css', !ruby/regexp '/application-/\d{32}\.css/']
  # Ignored files. Useful if there are some files that are created dynamically on the server and you don't want to upload on deploy.
  # ignored_files: ['ignore_me.js', !ruby/regexp '/ignore_some/\d{32}\.css/']
  # Allow custom assets to be cacheable. Note: The base filename will be matched
  # If you have an asset with name "app.0b1a4cd3.js", only "app.0b1a4cd3" will need to be matched
  # cache_asset_regexps: ['cache_me.js', !ruby/regexp '/cache_some\.\d{8}\.css/']

development:
  <<: *defaults

test:
  <<: *defaults

production:
  <<: *defaults

Available Configuration Options

Most AssetSync configuration can be modified directly using environment variables with the Built-in initializer. e.g.

AssetSync.config.fog_provider == ENV['FOG_PROVIDER']

Simply upcase the ruby attribute names to get the equivalent environment variable to set. The only exception to that rule are the internal AssetSync config variables, they must be prepended with ASSET_SYNC_* e.g.

AssetSync.config.gzip_compression == ENV['ASSET_SYNC_GZIP_COMPRESSION']

AssetSync (optional)

• existing_remote_files: ('keep', 'delete', 'ignore') what to do with previously precompiled files. default: 'keep'
• gzip_compression: (true, false) when enabled, will automatically replace files that have a gzip compressed equivalent with the compressed version. default: 'false'
• manifest: (true, false) when enabled, will use the manifest.yml generated by Rails to get the list of local files to upload. experimental. default: 'false'
• include_manifest: (true, false) when enabled, will upload the manifest.yml generated by Rails. default: 'false'
• concurrent_uploads: (true, false) when enabled, will upload the files in different Threads, this greatly improves the upload speed. default: 'false'
• concurrent_uploads_max_threads: when concurrent_uploads is enabled, this determines the number of threads that will be created. default: 10
• remote_file_list_cache_file_path: if present, use this path to cache remote file list to skip scanning remote default: nil
• enabled: (true, false) when false, will disable asset sync. default: 'true' (enabled)
• ignored_files: an array of files to ignore e.g. ['ignore_me.js', %r(ignore_some/\d{32}\.css)] Useful if there are some files that are created dynamically on the server and you don't want to upload on deploy default: []
• cache_asset_regexps: an array of files to add cache headers e.g. ['cache_me.js', %r(cache_some\.\d{8}\.css)] Useful if there are some files that are added to sprockets assets list and need to be set as 'Cacheable' on uploaded server. Only rails compiled regexp is matched internally default: []

Config Method add_local_file_paths

Adding local files by providing a block:

AssetSync.configure do |config|
  # The block should return an array of file paths
  config.add_local_file_paths do
    # Any code that returns paths of local asset files to be uploaded
    # Like Webpacker
    public_root = Rails.root.join("public")
    Dir.chdir(public_root) do
      packs_dir = Webpacker.config.public_output_path.relative_path_from(public_root)
      Dir[File.join(packs_dir, '/**/**')]
    end
  end
end

The blocks are run when local files are being scanned and uploaded

Config Method file_ext_to_mime_type_overrides

It's reported that mime-types 3.x returns application/ecmascript instead of application/javascript
Such change of mime type might cause some CDN to disable asset compression
So this gem has defined a default override for file ext js to be mapped to application/javascript by default

To customize the overrides:

AssetSync.configure do |config|
  # Clear the default overrides
  config.file_ext_to_mime_type_overrides.clear

  # Add/Edit overrides
  # Will call `#to_s` for inputs
  config.file_ext_to_mime_type_overrides.add(:js, :"application/x-javascript")
end

The blocks are run when local files are being scanned and uploaded

Fog (Required)

• fog_provider: your storage provider AWS (S3) or Rackspace (Cloud Files) or Google (Google Storage) or AzureRM (Azure Blob) or Backblaze (Backblaze B2)
• fog_directory: your bucket name

Fog (Optional)

• fog_region: the region your storage bucket is in e.g. eu-west-1 (AWS), ord (Rackspace), japanwest (Azure Blob)
• fog_path_style: To use buckets with dot in names, check fog/fog#2381 (comment)

AWS

• aws_access_key_id: your Amazon S3 access key
• aws_secret_access_key: your Amazon S3 access secret
• aws_acl: set canned ACL of uploaded object, will override fog_public if set

Rackspace

• rackspace_username: your Rackspace username
• rackspace_api_key: your Rackspace API Key.

Google Storage

When using the JSON API

• google_project: your Google Cloud Project name where the Google Cloud Storage bucket resides
• google_json_key_location: path to the location of the service account key. The service account key must be a JSON type key

When using the S3 API

• google_storage_access_key_id: your Google Storage access key
• google_storage_secret_access_key: your Google Storage access secret

Azure Blob

• azure_storage_account_name: your Azure Blob access key
• azure_storage_access_key: your Azure Blob access secret

Backblaze B2

• b2_key_id: Your Backblaze B2 key ID
• b2_key_token: Your Backblaze B2 key token
• b2_bucket_id: Your Backblaze B2 bucket ID

Rackspace (Optional)

• rackspace_auth_url: Rackspace auth URL, for Rackspace London use: https://lon.identity.api.rackspacecloud.com/v2.0

Amazon S3 Multiple Region Support

If you are using anything other than the US buckets with S3 then you'll want to set the region. For example with an EU bucket you could set the following environment variable.

heroku config:add FOG_REGION=eu-west-1

Or via a custom initializer

AssetSync.configure do |config|
  # ...
  config.fog_region = 'eu-west-1'
end

Or via YAML

production:  # ...  fog_region: 'eu-west-1'

Amazon (AWS) IAM Users

Amazon has switched to the more secure IAM User security policy model. When generating a user & policy for asset_sync you must ensure the policy has the following permissions, or you'll see the error:

Expected(200) <=> Actual(403 Forbidden)

IAM User Policy Example with minimum require permissions (replace bucket_name with your bucket):

{
  "Statement": [
    {
      "Action": "s3:ListBucket",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::bucket_name"
    },
    {
      "Action": "s3:PutObject*",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::bucket_name/*"
    }
  ]
}

If you want to use IAM roles you must set config.aws_iam_roles = true in your initializers.

AssetSync.configure do |config|
  # ...
  config.aws_iam_roles = true
end

Automatic gzip compression

With the gzip_compression option enabled, when uploading your assets. If a file has a gzip compressed equivalent we will replace that asset with the compressed version and sets the correct headers for S3 to serve it. For example, if you have a file master.css and it was compressed to master.css.gz we will upload the .gz file to S3 in place of the uncompressed file.

If the compressed file is actually larger than the uncompressed file we will ignore this rule and upload the standard uncompressed version.

Fail Silently

With the fail_silently option enabled, when running rake assets:precompile AssetSync will never throw an error due to missing configuration variables.

With the new user_env_compile feature of Heroku (see above), this is no longer required or recommended. Yet was added for the following reasons:

With Rails 3.1 on the Heroku cedar stack, the deployment process automatically runs rake assets:precompile. If you are using ENV variable style configuration. Due to the methods with which Heroku compile slugs, there will be an error raised by asset_sync as the environment is not available. This causes heroku to install the rails31_enable_runtime_asset_compilation plugin which is not necessary when using asset_sync and also massively slows down the first incoming requests to your app.

To prevent this part of the deploy from failing (asset_sync raising a config error), but carry on as normal set fail_silently to true in your configuration and ensure to run heroku run rake assets:precompile after deploy.

Rake Task

A rake task is included within the asset_sync gem to perform the sync:

  namespace :assets do
    desc "Synchronize assets to S3"
    task :sync => :environment do
      AssetSync.sync
    end
  end

If AssetSync.config.run_on_precompile is true (default), then assets will be uploaded to S3 automatically after the assets:precompile rake task is invoked:

  if Rake::Task.task_defined?("assets:precompile:nondigest")
    Rake::Task["assets:precompile:nondigest"].enhance do
      Rake::Task["assets:sync"].invoke if defined?(AssetSync) && AssetSync.config.run_on_precompile
    end
  else
    Rake::Task["assets:precompile"].enhance do
      Rake::Task["assets:sync"].invoke if defined?(AssetSync) && AssetSync.config.run_on_precompile
    end
  end

You can disable this behavior by setting AssetSync.config.run_on_precompile = false.

Sinatra/Rack Support

You can use the gem with any Rack application, but you must specify two additional options; prefix and public_path.

AssetSync.configure do |config|
  config.fog_provider = 'AWS'
  config.fog_directory = ENV['FOG_DIRECTORY']
  config.aws_access_key_id = ENV['AWS_ACCESS_KEY_ID']
  config.aws_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
  config.prefix = 'assets'
  # Can be a `Pathname` or `String`
  # Will be converted into an `Pathname`
  # If relative, will be converted into an absolute path
  # via `::Rails.root` or `::Dir.pwd`
  config.public_path = Pathname('./public')
end

Then manually call AssetSync.sync at the end of your asset precompilation task.

namespace :assets do
  desc 'Precompile assets'
  task :precompile do
    target = Pathname('./public/assets')
    manifest = Sprockets::Manifest.new(sprockets, './public/assets/manifest.json')

    sprockets.each_logical_path do |logical_path|
      if (!File.extname(logical_path).in?(['.js', '.css']) || logical_path =~ /application\.(css|js)$/) && asset = sprockets.find_asset(logical_path)
        filename = target.join(logical_path)
        FileUtils.mkpath(filename.dirname)
        puts "Write asset: #{filename}"
        asset.write_to(filename)
        manifest.compile(logical_path)
      end
    end

    AssetSync.sync
  end
end

Webpacker (> 2.0) support

1. Add webpacker files and disable run_on_precompile:

AssetSync.configure do |config|
  # Disable automatic run on precompile in order to attach to webpacker rake task
  config.run_on_precompile = false
  # The block should return an array of file paths
  config.add_local_file_paths do
    # Support webpacker assets
    public_root = Rails.root.join("public")
    Dir.chdir(public_root) do
      packs_dir = Webpacker.config.public_output_path.relative_path_from(public_root)
      Dir[File.join(packs_dir, '/**/**')]
    end
  end
end

1. Add a asset_sync.rake in your lib/tasks directory that enhances the correct task, otherwise asset_sync runs before webpacker:compile does:

if defined?(AssetSync)
  Rake::Task['webpacker:compile'].enhance do
    Rake::Task["assets:sync"].invoke
  end
end

Caveat

By adding local files outside the normal Rails assets directory, the uploading part works, however checking that the asset was previously uploaded is not working because asset_sync is only fetching the files in the assets directory on the remote bucket. This will mean additional time used to upload the same assets again on every precompilation.

Running the specs

Make sure you have a .env file with these details:-

# for AWS provider
AWS_ACCESS_KEY_ID=<yourkeyid>
AWS_SECRET_ACCESS_KEY=<yoursecretkey>
FOG_DIRECTORY=<yourbucket>
FOG_REGION=<youbucketregion>

# for AzureRM provider
AZURE_STORAGE_ACCOUNT_NAME=<youraccountname>
AZURE_STORAGE_ACCESS_KEY=<youraccesskey>
FOG_DIRECTORY=<yourcontainer>
FOG_REGION=<yourcontainerregion>

Make sure the bucket has read/write permissions. Then to run the tests:-

foreman run rake

Todo

1. Add some before and after filters for deleting and uploading
2. Support more cloud storage providers
3. Better test coverage
4. Add rake tasks to clean old assets from a bucket

Credits

Inspired by:

• https://github.com/moocode/asset_id
• https://gist.github.com/1053855

License

MIT License. Copyright 2011-2013 Rumble Labs Ltd. rumblelabs.com

---

Author: AssetSync
Source code: https://github.com/AssetSync/asset_sync
License:

#ruby   #ruby-on-rails 

Spring Cloud Config — Centralized Configuration in Microservices

Unlike a traditional monolithic application in which everything runs within a single instance, a microservice-based application consists of multiple instances of services running across multiple servers. Managing configuration settings for each of these service instances can be challenging as maintaining multiple copies of configuration settings across different servers, locations, and environments is error-prone and difficult to manage. This is especially true for the increasing number of services in microservices architecture and services deployed in the cloud with an auto-scaling feature where servers come and go in the cloud. As a result, need has grown for a better mechanism to manage configuration settings in microservice-based applications. This is where a centralized configuration server steps in to take these configuration settings into a centralized location that is externalized from the distributed services.

In this article, we will learn how to build a centralized configuration server using a Spring Cloud config server that uses Git repository as configuration storage. Will we also look at how to consume the remote configuration settings from a service build using Spring Boot.

Spring Cloud Config Server

A Spring Cloud config server is one of the more popular centralized configuration servers used in a microservice-based application. This is especially true with the increasing trend of Java developers building their application using Spring Boot; providing little to no effort of work to integrate their application to Spring Cloud config server. It uses a typical client and server approach for storing and serving configuration settings across these distributed services.

---

Prerequisites

Before getting ready to write some code, we need to create two Spring Boot projects: the server and the client project. We will be using Apache Maven in this article. You can use the Spring Initializr website to generate these projects with Spring Boot 2.x dependency. Alternatively you can download the sample server and client project from GitHub.

#spring-boot #centralized-configuration #programming #microservices #spring-cloud-config

How To Secure Boot with Gentoo Linux

Microsoft has announced Windows 11 and with it new system requirements. One of the most problematic requirement is enabling secure boot.
In this video, I will be going over how to get secure boot working on Linux.

Find me elsewhere:
Instagram: https://www.instagram.com/kamiyaa91/
Github: https://github.com/kamiyaa/

Theres 2 major ways of getting Secure Boot to work on Linux.

1. SHIM, this is a bootloader thats already signed and is used to boot the linux kernel
2. Manually signing, this is the method we will be using because it is much more flexible and work with many different bootloaders

Sakaki’s Secure Boot Guide: https://wiki.gentoo.org/wiki/User:Sakaki/Sakaki’s_EFI_Install_Guide/Configuring_Secure_Boot

Timestamps:
00:00 Intro
01:14 MBR/GPT/BIOS/UEFI
02:40 Secure Boot Architecture
03:45 Getting Secure Boot to Work
07:20 Reboot #1
09:14 Reboot #2

#gentoo #linux #gentoo linux