Michio JP

Michio JP

1631613488

Automated Network Configuration Backups using Github Actions

Network Config Scraper

This repository demonstrates the use of Github Actions and git-scraping to build an automated backup solution for network configuration files. Git already provides an efficient way to track and manage changes to textual data, and Github Actions provide automation that we can use to fetch and process configuration backups without reliance on any additional infrastructure. The solution in this repository uses both to retrieve configurations from network devices on a defined schedule and commits the detected changes back to the repository.

This approach is heavily inspired by Git scraping: track changes over time by scraping to a Git repository by Simon Willison. His post provides an excellent overview of git-scraping data from various sources on the Internet.

config scraper

Could I use this?

Because of the textual nature of network configurations, git-scraping offers a simple yet effective way to backup and version configuration data. It runs entirely on GitHub Actions, so there's no complex infrastructure or orchestrations to manage.

Because the data we retrieve data from devices in a private lab, the repository uses a self-hosted runner that makes a connection to Github and has access to the lab. You'd have to decide if this is an acceptable model for your environment.

However, the solution works wonderfully and can be extended to fit into your existing automation. For example, using the webhooks in Github can extend this solution by allowing you to build or set up integrations to external automation systems which can subscribe to certain events on the repository. The configured events can trigger an HTTP POST to a system like Ansible Tower to provide additional automation.

How does it work?

The configuration scraper is configured and scheduled in the .github/workflows/scrape.yml action workflow. It's a short and simple workflow that defines all the triggers and steps to run our automation.

trigger

The workflow can be triggered in 3 different ways: on a push event to the repo, manually using the workflow_dispatch, or, most importantly, on a cron schedule. For example, in the snippet below, you can see that we have the workflow triggered every 30 minutes.

on:
  push:
    branches:
      - main
  workflow_dispatch:
  schedule:
    - cron:  '*/30 * * * *'

self-hosted runner(s)

Because our use case is slightly different from fetching data from flat files, we need to account for how GitHub would access the devices that we are fetching data from. For example, the devices in the inventory.yml are in a lab that sits behind a firewall, so a public Github action runner would not have the required access. For that reason, the repo is configured to use a self-hosted Github action runner that has access to the lab environment. It took about 15 minutes to provision and to configure a runner in this environment, so it is a relatively easy and painless process. Once configured, the self-hosted runner creates a connection to Github and listens for job requests to execute your actions.

Within the action workflow, the only things to specify are the fact that we're using a self-hosted runner and we also provide the tags to identify which runner to use for the action. We use the runs-on directive in the action to do just that, as shown below.

jobs:
  scheduled:
    runs-on: [self-hosted, atc-runners]

data

For data gathering, the retrieve_configs.py script is used to retrieve and save configuration data from each of the devices listed in the inventory.yml file. The script uses an async SSH connection transport from the scrapli library to handle parallel sessions to devices. Once the files are saved, the action workflow uses git to stage any files in which changes have been detected. A commit is created with a timestamp before the changes are pushed into the repository.

  - name: Fetch latest configs
    run: |-
      python retrieve_configs.py
    env:
      SSH_AUTH_USERNAME: ${{ secrets.SSH_AUTH_USERNAME }}
      SSH_AUTH_PASSWORD: ${{ secrets.SSH_AUTH_PASSWORD }}

  - name: Commit and push if it changed
    run: |-
      git config user.name "Automated"
      git config user.email "actions@users.noreply.github.com"
      git add -A
      timestamp=$(date -u)
      git commit -m "Latest data: ${timestamp}" || exit 0
      git push

How do I track changes?

The neat thing about using Git to manage your configuration backups is that you get this commit log showing the history of commits that have been made to your configs.

Author(s)

Download Details:
 

Author: wwt
Download Link: Download The Source Code
Official Website: https://github.com/wwt/network-config-scraper 
License: Apache-2.0 License

What is GEEK

Buddha Community

Automated Network Configuration Backups using Github Actions
Chloe  Butler

Chloe Butler

1667425440

Pdf2gerb: Perl Script Converts PDF Files to Gerber format

pdf2gerb

Perl script converts PDF files to Gerber format

Pdf2Gerb generates Gerber 274X photoplotting and Excellon drill files from PDFs of a PCB. Up to three PDFs are used: the top copper layer, the bottom copper layer (for 2-sided PCBs), and an optional silk screen layer. The PDFs can be created directly from any PDF drawing software, or a PDF print driver can be used to capture the Print output if the drawing software does not directly support output to PDF.

The general workflow is as follows:

  1. Design the PCB using your favorite CAD or drawing software.
  2. Print the top and bottom copper and top silk screen layers to a PDF file.
  3. Run Pdf2Gerb on the PDFs to create Gerber and Excellon files.
  4. Use a Gerber viewer to double-check the output against the original PCB design.
  5. Make adjustments as needed.
  6. Submit the files to a PCB manufacturer.

Please note that Pdf2Gerb does NOT perform DRC (Design Rule Checks), as these will vary according to individual PCB manufacturer conventions and capabilities. Also note that Pdf2Gerb is not perfect, so the output files must always be checked before submitting them. As of version 1.6, Pdf2Gerb supports most PCB elements, such as round and square pads, round holes, traces, SMD pads, ground planes, no-fill areas, and panelization. However, because it interprets the graphical output of a Print function, there are limitations in what it can recognize (or there may be bugs).

See docs/Pdf2Gerb.pdf for install/setup, config, usage, and other info.


pdf2gerb_cfg.pm

#Pdf2Gerb config settings:
#Put this file in same folder/directory as pdf2gerb.pl itself (global settings),
#or copy to another folder/directory with PDFs if you want PCB-specific settings.
#There is only one user of this file, so we don't need a custom package or namespace.
#NOTE: all constants defined in here will be added to main namespace.
#package pdf2gerb_cfg;

use strict; #trap undef vars (easier debug)
use warnings; #other useful info (easier debug)


##############################################################################################
#configurable settings:
#change values here instead of in main pfg2gerb.pl file

use constant WANT_COLORS => ($^O !~ m/Win/); #ANSI colors no worky on Windows? this must be set < first DebugPrint() call

#just a little warning; set realistic expectations:
#DebugPrint("${\(CYAN)}Pdf2Gerb.pl ${\(VERSION)}, $^O O/S\n${\(YELLOW)}${\(BOLD)}${\(ITALIC)}This is EXPERIMENTAL software.  \nGerber files MAY CONTAIN ERRORS.  Please CHECK them before fabrication!${\(RESET)}", 0); #if WANT_DEBUG

use constant METRIC => FALSE; #set to TRUE for metric units (only affect final numbers in output files, not internal arithmetic)
use constant APERTURE_LIMIT => 0; #34; #max #apertures to use; generate warnings if too many apertures are used (0 to not check)
use constant DRILL_FMT => '2.4'; #'2.3'; #'2.4' is the default for PCB fab; change to '2.3' for CNC

use constant WANT_DEBUG => 0; #10; #level of debug wanted; higher == more, lower == less, 0 == none
use constant GERBER_DEBUG => 0; #level of debug to include in Gerber file; DON'T USE FOR FABRICATION
use constant WANT_STREAMS => FALSE; #TRUE; #save decompressed streams to files (for debug)
use constant WANT_ALLINPUT => FALSE; #TRUE; #save entire input stream (for debug ONLY)

#DebugPrint(sprintf("${\(CYAN)}DEBUG: stdout %d, gerber %d, want streams? %d, all input? %d, O/S: $^O, Perl: $]${\(RESET)}\n", WANT_DEBUG, GERBER_DEBUG, WANT_STREAMS, WANT_ALLINPUT), 1);
#DebugPrint(sprintf("max int = %d, min int = %d\n", MAXINT, MININT), 1); 

#define standard trace and pad sizes to reduce scaling or PDF rendering errors:
#This avoids weird aperture settings and replaces them with more standardized values.
#(I'm not sure how photoplotters handle strange sizes).
#Fewer choices here gives more accurate mapping in the final Gerber files.
#units are in inches
use constant TOOL_SIZES => #add more as desired
(
#round or square pads (> 0) and drills (< 0):
    .010, -.001,  #tiny pads for SMD; dummy drill size (too small for practical use, but needed so StandardTool will use this entry)
    .031, -.014,  #used for vias
    .041, -.020,  #smallest non-filled plated hole
    .051, -.025,
    .056, -.029,  #useful for IC pins
    .070, -.033,
    .075, -.040,  #heavier leads
#    .090, -.043,  #NOTE: 600 dpi is not high enough resolution to reliably distinguish between .043" and .046", so choose 1 of the 2 here
    .100, -.046,
    .115, -.052,
    .130, -.061,
    .140, -.067,
    .150, -.079,
    .175, -.088,
    .190, -.093,
    .200, -.100,
    .220, -.110,
    .160, -.125,  #useful for mounting holes
#some additional pad sizes without holes (repeat a previous hole size if you just want the pad size):
    .090, -.040,  #want a .090 pad option, but use dummy hole size
    .065, -.040, #.065 x .065 rect pad
    .035, -.040, #.035 x .065 rect pad
#traces:
    .001,  #too thin for real traces; use only for board outlines
    .006,  #minimum real trace width; mainly used for text
    .008,  #mainly used for mid-sized text, not traces
    .010,  #minimum recommended trace width for low-current signals
    .012,
    .015,  #moderate low-voltage current
    .020,  #heavier trace for power, ground (even if a lighter one is adequate)
    .025,
    .030,  #heavy-current traces; be careful with these ones!
    .040,
    .050,
    .060,
    .080,
    .100,
    .120,
);
#Areas larger than the values below will be filled with parallel lines:
#This cuts down on the number of aperture sizes used.
#Set to 0 to always use an aperture or drill, regardless of size.
use constant { MAX_APERTURE => max((TOOL_SIZES)) + .004, MAX_DRILL => -min((TOOL_SIZES)) + .004 }; #max aperture and drill sizes (plus a little tolerance)
#DebugPrint(sprintf("using %d standard tool sizes: %s, max aper %.3f, max drill %.3f\n", scalar((TOOL_SIZES)), join(", ", (TOOL_SIZES)), MAX_APERTURE, MAX_DRILL), 1);

#NOTE: Compare the PDF to the original CAD file to check the accuracy of the PDF rendering and parsing!
#for example, the CAD software I used generated the following circles for holes:
#CAD hole size:   parsed PDF diameter:      error:
#  .014                .016                +.002
#  .020                .02267              +.00267
#  .025                .026                +.001
#  .029                .03167              +.00267
#  .033                .036                +.003
#  .040                .04267              +.00267
#This was usually ~ .002" - .003" too big compared to the hole as displayed in the CAD software.
#To compensate for PDF rendering errors (either during CAD Print function or PDF parsing logic), adjust the values below as needed.
#units are pixels; for example, a value of 2.4 at 600 dpi = .0004 inch, 2 at 600 dpi = .0033"
use constant
{
    HOLE_ADJUST => -0.004 * 600, #-2.6, #holes seemed to be slightly oversized (by .002" - .004"), so shrink them a little
    RNDPAD_ADJUST => -0.003 * 600, #-2, #-2.4, #round pads seemed to be slightly oversized, so shrink them a little
    SQRPAD_ADJUST => +0.001 * 600, #+.5, #square pads are sometimes too small by .00067, so bump them up a little
    RECTPAD_ADJUST => 0, #(pixels) rectangular pads seem to be okay? (not tested much)
    TRACE_ADJUST => 0, #(pixels) traces seemed to be okay?
    REDUCE_TOLERANCE => .001, #(inches) allow this much variation when reducing circles and rects
};

#Also, my CAD's Print function or the PDF print driver I used was a little off for circles, so define some additional adjustment values here:
#Values are added to X/Y coordinates; units are pixels; for example, a value of 1 at 600 dpi would be ~= .002 inch
use constant
{
    CIRCLE_ADJUST_MINX => 0,
    CIRCLE_ADJUST_MINY => -0.001 * 600, #-1, #circles were a little too high, so nudge them a little lower
    CIRCLE_ADJUST_MAXX => +0.001 * 600, #+1, #circles were a little too far to the left, so nudge them a little to the right
    CIRCLE_ADJUST_MAXY => 0,
    SUBST_CIRCLE_CLIPRECT => FALSE, #generate circle and substitute for clip rects (to compensate for the way some CAD software draws circles)
    WANT_CLIPRECT => TRUE, #FALSE, #AI doesn't need clip rect at all? should be on normally?
    RECT_COMPLETION => FALSE, #TRUE, #fill in 4th side of rect when 3 sides found
};

#allow .012 clearance around pads for solder mask:
#This value effectively adjusts pad sizes in the TOOL_SIZES list above (only for solder mask layers).
use constant SOLDER_MARGIN => +.012; #units are inches

#line join/cap styles:
use constant
{
    CAP_NONE => 0, #butt (none); line is exact length
    CAP_ROUND => 1, #round cap/join; line overhangs by a semi-circle at either end
    CAP_SQUARE => 2, #square cap/join; line overhangs by a half square on either end
    CAP_OVERRIDE => FALSE, #cap style overrides drawing logic
};
    
#number of elements in each shape type:
use constant
{
    RECT_SHAPELEN => 6, #x0, y0, x1, y1, count, "rect" (start, end corners)
    LINE_SHAPELEN => 6, #x0, y0, x1, y1, count, "line" (line seg)
    CURVE_SHAPELEN => 10, #xstart, ystart, x0, y0, x1, y1, xend, yend, count, "curve" (bezier 2 points)
    CIRCLE_SHAPELEN => 5, #x, y, 5, count, "circle" (center + radius)
};
#const my %SHAPELEN =
#Readonly my %SHAPELEN =>
our %SHAPELEN =
(
    rect => RECT_SHAPELEN,
    line => LINE_SHAPELEN,
    curve => CURVE_SHAPELEN,
    circle => CIRCLE_SHAPELEN,
);

#panelization:
#This will repeat the entire body the number of times indicated along the X or Y axes (files grow accordingly).
#Display elements that overhang PCB boundary can be squashed or left as-is (typically text or other silk screen markings).
#Set "overhangs" TRUE to allow overhangs, FALSE to truncate them.
#xpad and ypad allow margins to be added around outer edge of panelized PCB.
use constant PANELIZE => {'x' => 1, 'y' => 1, 'xpad' => 0, 'ypad' => 0, 'overhangs' => TRUE}; #number of times to repeat in X and Y directions

# Set this to 1 if you need TurboCAD support.
#$turboCAD = FALSE; #is this still needed as an option?

#CIRCAD pad generation uses an appropriate aperture, then moves it (stroke) "a little" - we use this to find pads and distinguish them from PCB holes. 
use constant PAD_STROKE => 0.3; #0.0005 * 600; #units are pixels
#convert very short traces to pads or holes:
use constant TRACE_MINLEN => .001; #units are inches
#use constant ALWAYS_XY => TRUE; #FALSE; #force XY even if X or Y doesn't change; NOTE: needs to be TRUE for all pads to show in FlatCAM and ViewPlot
use constant REMOVE_POLARITY => FALSE; #TRUE; #set to remove subtractive (negative) polarity; NOTE: must be FALSE for ground planes

#PDF uses "points", each point = 1/72 inch
#combined with a PDF scale factor of .12, this gives 600 dpi resolution (1/72 * .12 = 600 dpi)
use constant INCHES_PER_POINT => 1/72; #0.0138888889; #multiply point-size by this to get inches

# The precision used when computing a bezier curve. Higher numbers are more precise but slower (and generate larger files).
#$bezierPrecision = 100;
use constant BEZIER_PRECISION => 36; #100; #use const; reduced for faster rendering (mainly used for silk screen and thermal pads)

# Ground planes and silk screen or larger copper rectangles or circles are filled line-by-line using this resolution.
use constant FILL_WIDTH => .01; #fill at most 0.01 inch at a time

# The max number of characters to read into memory
use constant MAX_BYTES => 10 * M; #bumped up to 10 MB, use const

use constant DUP_DRILL1 => TRUE; #FALSE; #kludge: ViewPlot doesn't load drill files that are too small so duplicate first tool

my $runtime = time(); #Time::HiRes::gettimeofday(); #measure my execution time

print STDERR "Loaded config settings from '${\(__FILE__)}'.\n";
1; #last value must be truthful to indicate successful load


#############################################################################################
#junk/experiment:

#use Package::Constants;
#use Exporter qw(import); #https://perldoc.perl.org/Exporter.html

#my $caller = "pdf2gerb::";

#sub cfg
#{
#    my $proto = shift;
#    my $class = ref($proto) || $proto;
#    my $settings =
#    {
#        $WANT_DEBUG => 990, #10; #level of debug wanted; higher == more, lower == less, 0 == none
#    };
#    bless($settings, $class);
#    return $settings;
#}

#use constant HELLO => "hi there2"; #"main::HELLO" => "hi there";
#use constant GOODBYE => 14; #"main::GOODBYE" => 12;

#print STDERR "read cfg file\n";

#our @EXPORT_OK = Package::Constants->list(__PACKAGE__); #https://www.perlmonks.org/?node_id=1072691; NOTE: "_OK" skips short/common names

#print STDERR scalar(@EXPORT_OK) . " consts exported:\n";
#foreach(@EXPORT_OK) { print STDERR "$_\n"; }
#my $val = main::thing("xyz");
#print STDERR "caller gave me $val\n";
#foreach my $arg (@ARGV) { print STDERR "arg $arg\n"; }

Download Details:

Author: swannman
Source Code: https://github.com/swannman/pdf2gerb

License: GPL-3.0 license

#perl 

Rahul Raathi

1607512027

Reasons why Network Automation is Important to learn for Network Engineers

Companies today are constantly pursuing greater agility and speed in their IT operations. By innovating with new technologies and adapting rapidly to changing consumer dynamics, they are seeking to seize market advantage.

In the quest for productivity, a big roadblock many companies face is that their business network is much more difficult to navigate than ever before. Distributed workloads and distributed IT resources have resulted in highly complex configurations and low environment-wide visibility.
What is network automation?

The process of automating the configuration, management, testing, implementation and operation of physical and virtual devices within a network is network automation. Network service availability increases with regular network operations and functions that are automatically monitored and handled by automated and repetitive processes.

Any type of network such as data centers, service providers, and enterprises can use network automation to increase efficiency, reduce man made errors, and lower operating expenses.

Why network automation is important?

The rise in IT costs for network operations is one of the main problems for network engineers. Data and system growth is beginning to outpace IT capacities, rendering manual approaches almost impossible.
However, up to 95% of network adjustments are made manually, resulting in maintenance costs that are 2 to 3 times higher than network costs. For companies to keep pace in the modern world, increased IT automation, centrally and remotely controlled, is important.

#1 Manual Tasks Removal: Automating the network substitutes predictable, repeatable network adjustments for manual tasks. That improves the competitiveness of IT, so you can concentrate on strategic opportunities that drive business growth.

#2 Accelerate delivery of service: A crucial advantage: delivering faster key data services, improving network performance, and speeding up the implementation of new services and applications.

#3 Templates standardize procedures: Use structured templates to drive performance and streamline network changes among network stakeholders.

#4 Enable fast improvements: Network modifications that you previously rarely performed because they were manual, time consuming, and resource intensive are more regularly implemented.

#5 Build a network that is secure: Using an infrastructure-as-code method to preserve the network’s state and configuration, from data centre to edge, in much the same way that source code is handled by developers.

#6 Gain visibility on the network: To detect performance problems, spiking resource consumption levels, and network failures, access wide-ranging performance monitoring capabilities.

#7 Analyze and fix problems: For insight into efficiency, usage, protection, and resource allocation, depend on network analytics that help solve problems much faster than through manual techniques.

Conclusion:
The number and type of devices, users and objects that are connected to the network has grown exponentially. And to satisfy this growing demand, networks are becoming more complicated and implementations are becoming huge. In addition to simplifying the Day one implementation process, Network Automation plays a very crucial role in simplifying day-to-day operations & maintenance.

These are the major reasons why enterprises are looking to hire IT professionals with network automation skills. That’s why; there is big demand of automation course for network engineers these days. You can quickly and easily plan, supply, and enforce policies across your network with network automation for your organization.

#network automation #why network automation #what is network automation

Rahul Raathi

1607512027

Reasons why Network Automation is Important to learn for Network Engineers

Companies today are constantly pursuing greater agility and speed in their IT operations. By innovating with new technologies and adapting rapidly to changing consumer dynamics, they are seeking to seize market advantage.

In the quest for productivity, a big roadblock many companies face is that their business network is much more difficult to navigate than ever before. Distributed workloads and distributed IT resources have resulted in highly complex configurations and low environment-wide visibility.
What is network automation?

The process of automating the configuration, management, testing, implementation and operation of physical and virtual devices within a network is network automation. Network service availability increases with regular network operations and functions that are automatically monitored and handled by automated and repetitive processes.

Any type of network such as data centers, service providers, and enterprises can use network automation to increase efficiency, reduce man made errors, and lower operating expenses.

Why network automation is important?

The rise in IT costs for network operations is one of the main problems for network engineers. Data and system growth is beginning to outpace IT capacities, rendering manual approaches almost impossible.
However, up to 95% of network adjustments are made manually, resulting in maintenance costs that are 2 to 3 times higher than network costs. For companies to keep pace in the modern world, increased IT automation, centrally and remotely controlled, is important.

#1 Manual Tasks Removal: Automating the network substitutes predictable, repeatable network adjustments for manual tasks. That improves the competitiveness of IT, so you can concentrate on strategic opportunities that drive business growth.

#2 Accelerate delivery of service: A crucial advantage: delivering faster key data services, improving network performance, and speeding up the implementation of new services and applications.

#3 Templates standardize procedures: Use structured templates to drive performance and streamline network changes among network stakeholders.

#4 Enable fast improvements: Network modifications that you previously rarely performed because they were manual, time consuming, and resource intensive are more regularly implemented.

#5 Build a network that is secure: Using an infrastructure-as-code method to preserve the network’s state and configuration, from data centre to edge, in much the same way that source code is handled by developers.

#6 Gain visibility on the network: To detect performance problems, spiking resource consumption levels, and network failures, access wide-ranging performance monitoring capabilities.

#7 Analyze and fix problems: For insight into efficiency, usage, protection, and resource allocation, depend on network analytics that help solve problems much faster than through manual techniques.

Conclusion:
The number and type of devices, users and objects that are connected to the network has grown exponentially. And to satisfy this growing demand, networks are becoming more complicated and implementations are becoming huge. In addition to simplifying the Day one implementation process, Network Automation plays a very crucial role in simplifying day-to-day operations & maintenance.

These are the major reasons why enterprises are looking to hire IT professionals with network automation skills. That’s why; there is big demand of automation course for network engineers these days. You can quickly and easily plan, supply, and enforce policies across your network with network automation for your organization.

#network automation #why network automation #what is network automation

Josefa  Corwin

Josefa Corwin

1659736920

Mailboxer: A Rails Gem to Send Messages inside A Web Application

Mailboxer

This project is based on the need for a private message system for ging / social_stream. Instead of creating our core message system heavily dependent on our development, we are trying to implement a generic and potent messaging gem.

After looking for a good gem to use we noticed the lack of messaging gems and functionality in them. Mailboxer tries to fill this void delivering a powerful and flexible message system. It supports the use of conversations with two or more participants, sending notifications to recipients (intended to be used as system notifications “Your picture has new comments”, “John Doe has updated his document”, etc.), and emailing the messageable model (if configured to do so). It has a complete implementation of a Mailbox object for each messageable with inbox, sentbox and trash.

The gem is constantly growing and improving its functionality. As it is used with our parallel development ging / social_stream we are finding and fixing bugs continously. If you want some functionality not supported yet or marked as TODO, you can create an issue to ask for it. It will be great feedback for us, and we will know what you may find useful in the gem.

Mailboxer was born from the great, but outdated, code from lpsergi / acts_as_messageable.

We are now working to make exhaustive documentation and some wiki pages in order to make it even easier to use the gem to its full potential. Please, give us some time if you find something missing or ask for it. You can also find us on the Gitter room for this repo. Join us there to talk.

Installation

Add to your Gemfile:

gem 'mailboxer'

Then run:

$ bundle install

Run install script:

$ rails g mailboxer:install

And don't forget to migrate your database:

$ rake db:migrate

You can also generate email views:

$ rails g mailboxer:views

Upgrading

If upgrading from 0.11.0 to 0.12.0, run the following generators:

$ rails generate mailboxer:namespacing_compatibility
$ rails generate mailboxer:install -s

Then, migrate your database:

$ rake db:migrate

Requirements & Settings

Emails

We are now adding support for sending emails when a Notification or a Message is sent to one or more recipients. You should modify the mailboxer initializer (/config/initializer/mailboxer.rb) to edit these settings:

Mailboxer.setup do |config|
  #Enables or disables email sending for Notifications and Messages
  config.uses_emails = true
  #Configures the default `from` address for the email sent for Messages and Notifications of Mailboxer
  config.default_from = "no-reply@dit.upm.es"
  ...
end

You can change the way in which emails are delivered by specifying a custom implementation of notification and message mailers:

Mailboxer.setup do |config|
  config.notification_mailer = CustomNotificationMailer
  config.message_mailer = CustomMessageMailer
  ...
end

If you have subclassed the Mailboxer::Notification class, you can specify the mailers using a member method:

class NewDocumentNotification < Mailboxer::Notification
  def mailer_class
    NewDocumentNotificationMailer
  end
end

class NewCommentNotification < Mailboxer::Notification
  def mailer_class
    NewDocumentNotificationMailer
  end
end

Otherwise, the mailer class will be determined by appending 'Mailer' to the mailable class name.

User identities

Users must have an identity defined by a name and an email. We must ensure that Messageable models have some specific methods. These methods are:

#Returning any kind of identification you want for the model
def name
  return "You should add method :name in your Messageable model"
end
#Returning the email address of the model if an email should be sent for this object (Message or Notification).
#If no mail has to be sent, return nil.
def mailboxer_email(object)
  #Check if an email should be sent for that object
  #if true
  return "define_email@on_your.model"
  #if false
  #return nil
end

These names are explicit enough to avoid colliding with other methods, but as long as you need to change them you can do it by using mailboxer initializer (/config/initializer/mailboxer.rb). Just add or uncomment the following lines:

Mailboxer.setup do |config|
  # ...
  #Configures the methods needed by mailboxer
  config.email_method = :mailboxer_email
  config.name_method = :name
  config.notify_method = :notify
  # ...
end

You may change whatever you want or need. For example:

config.email_method = :notification_email
config.name_method = :display_name
config.notify_method = :notify_mailboxer

Will use the method notification_email(object) instead of mailboxer_email(object), display_name for name and notify_mailboxer for notify.

Using default or custom method names, if your model doesn't implement them, Mailboxer will use dummy methods so as to notify you of missing methods rather than crashing.

Preparing your models

In your model:

class User < ActiveRecord::Base
  acts_as_messageable
end

You are not limited to the User model. You can use Mailboxer in any other model and use it in several different models. If you have ducks and cylons in your application and you want to exchange messages as if they were the same, just add acts_as_messageable to each one and you will be able to send duck-duck, duck-cylon, cylon-duck and cylon-cylon messages. Of course, you can extend it for as many classes as you need.

Example:

class Duck < ActiveRecord::Base
  acts_as_messageable
end
class Cylon < ActiveRecord::Base
  acts_as_messageable
end

Mailboxer API

Warning for version 0.8.0

Version 0.8.0 sees Messageable#read and Messageable#unread renamed to mark_as_(un)read, and Receipt#read and Receipt#unread to is_(un)read. This may break existing applications, but read is a reserved name for Active Record, and the best pratice in this case is simply avoid using it.

How can I send a message?

#alfa wants to send a message to beta
alfa.send_message(beta, "Body", "subject")

How can I read the messages of a conversation?

As a messageable, what you receive are receipts, which are associated with the message itself. You should retrieve your receipts for the conversation and get the message associated with them.

This is done this way because receipts save the information about the relation between messageable and the messages: is it read?, is it trashed?, etc.

#alfa gets the last conversation (chronologically, the first in the inbox)
conversation = alfa.mailbox.inbox.first

#alfa gets it receipts chronologically ordered.
receipts = conversation.receipts_for alfa

#using the receipts (i.e. in the view)
receipts.each do |receipt|
  ...
  message = receipt.message
  read = receipt.is_unread? #or message.is_unread?(alfa)
  ...
end

How can I reply to a message?

#alfa wants to reply to all in a conversation
#using a receipt
alfa.reply_to_all(receipt, "Reply body")

#using a conversation
alfa.reply_to_conversation(conversation, "Reply body")
#alfa wants to reply to the sender of a message (and ONLY the sender)
#using a receipt
alfa.reply_to_sender(receipt, "Reply body")

How can I delete a message from trash?

#delete conversations forever for one receipt (still in database)
receipt.mark_as_deleted

#you can mark conversation as deleted for one participant
conversation.mark_as_deleted participant

#Mark the object as deleted for messageable
#Object can be:
  #* A Receipt
  #* A Conversation
  #* A Notification
  #* A Message
  #* An array with any of them
alfa.mark_as_deleted conversation

# get available message for specific user
conversation.messages_for(alfa)

How can I retrieve my conversations?

#alfa wants to retrieve all his conversations
alfa.mailbox.conversations

#A wants to retrieve his inbox
alfa.mailbox.inbox

#A wants to retrieve his sent conversations
alfa.mailbox.sentbox

#alfa wants to retrieve his trashed conversations
alfa.mailbox.trash

How can I paginate conversations?

You can use Kaminari to paginate the conversations as normal. Please, make sure you use the last version as mailboxer uses select('DISTINCT conversations.*') which was not respected before Kaminari 0.12.4 according to its changelog. Working correctly on Kaminari 0.13.0.

#Paginating all conversations using :page parameter and 9 per page
conversations = alfa.mailbox.conversations.page(params[:page]).per(9)

#Paginating received conversations using :page parameter and 9 per page
conversations = alfa.mailbox.inbox.page(params[:page]).per(9)

#Paginating sent conversations using :page parameter and 9 per page
conversations = alfa.mailbox.sentbox.page(params[:page]).per(9)

#Paginating trashed conversations using :page parameter and 9 per page
conversations = alfa.mailbox.trash.page(params[:page]).per(9)

You can take a look at the full documentation for Mailboxer in rubydoc.info.

Do you want to test Mailboxer?

Thanks to Roman Kushnir (@RKushnir) you can test Mailboxer with this sample app.

I need a GUI!

If you need a GUI you should take a look at these links:

Contributors


Author: mailboxer
Source code: https://github.com/mailboxer/mailboxer
License: MIT license

#ruby  #ruby-on-rails 

Background Fetch for React Native Apps

react-native-background-fetch

Background Fetch is a very simple plugin which attempts to awaken an app in the background about every 15 minutes, providing a short period of background running-time. This plugin will execute your provided callbackFn whenever a background-fetch event occurs.

There is no way to increase the rate which a fetch-event occurs and this plugin sets the rate to the most frequent possible — you will never receive an event faster than 15 minutes. The operating-system will automatically throttle the rate the background-fetch events occur based upon usage patterns. Eg: if user hasn't turned on their phone for a long period of time, fetch events will occur less frequently or if an iOS user disables background refresh they may not happen at all.

:new: Background Fetch now provides a scheduleTask method for scheduling arbitrary "one-shot" or periodic tasks.

iOS

  • There is no way to increase the rate which a fetch-event occurs and this plugin sets the rate to the most frequent possible — you will never receive an event faster than 15 minutes. The operating-system will automatically throttle the rate the background-fetch events occur based upon usage patterns. Eg: if user hasn't turned on their phone for a long period of time, fetch events will occur less frequently.
  • scheduleTask seems only to fire when the device is plugged into power.
  • ⚠️ When your app is terminated, iOS no longer fires events — There is no such thing as stopOnTerminate: false for iOS.
  • iOS can take days before Apple's machine-learning algorithm settles in and begins regularly firing events. Do not sit staring at your logs waiting for an event to fire. If your simulated events work, that's all you need to know that everything is correctly configured.
  • If the user doesn't open your iOS app for long periods of time, iOS will stop firing events.

Android

Installing the plugin

⚠️ If you have a previous version of react-native-background-fetch < 2.7.0 installed into react-native >= 0.60, you should first unlink your previous version as react-native link is no longer required.

$ react-native unlink react-native-background-fetch

With yarn

$ yarn add react-native-background-fetch

With npm

$ npm install --save react-native-background-fetch

Setup Guides

iOS Setup

react-native >= 0.60

Android Setup

react-native >= 0.60

Example

ℹ️ This repo contains its own Example App. See /example

import React from 'react';
import {
  SafeAreaView,
  StyleSheet,
  ScrollView,
  View,
  Text,
  FlatList,
  StatusBar,
} from 'react-native';

import {
  Header,
  Colors
} from 'react-native/Libraries/NewAppScreen';

import BackgroundFetch from "react-native-background-fetch";

class App extends React.Component {
  constructor(props) {
    super(props);
    this.state = {
      events: []
    };
  }

  componentDidMount() {
    // Initialize BackgroundFetch ONLY ONCE when component mounts.
    this.initBackgroundFetch();
  }

  async initBackgroundFetch() {
    // BackgroundFetch event handler.
    const onEvent = async (taskId) => {
      console.log('[BackgroundFetch] task: ', taskId);
      // Do your background work...
      await this.addEvent(taskId);
      // IMPORTANT:  You must signal to the OS that your task is complete.
      BackgroundFetch.finish(taskId);
    }

    // Timeout callback is executed when your Task has exceeded its allowed running-time.
    // You must stop what you're doing immediately BackgroundFetch.finish(taskId)
    const onTimeout = async (taskId) => {
      console.warn('[BackgroundFetch] TIMEOUT task: ', taskId);
      BackgroundFetch.finish(taskId);
    }

    // Initialize BackgroundFetch only once when component mounts.
    let status = await BackgroundFetch.configure({minimumFetchInterval: 15}, onEvent, onTimeout);

    console.log('[BackgroundFetch] configure status: ', status);
  }

  // Add a BackgroundFetch event to <FlatList>
  addEvent(taskId) {
    // Simulate a possibly long-running asynchronous task with a Promise.
    return new Promise((resolve, reject) => {
      this.setState(state => ({
        events: [...state.events, {
          taskId: taskId,
          timestamp: (new Date()).toString()
        }]
      }));
      resolve();
    });
  }

  render() {
    return (
      <>
        <StatusBar barStyle="dark-content" />
        <SafeAreaView>
          <ScrollView
            contentInsetAdjustmentBehavior="automatic"
            style={styles.scrollView}>
            <Header />

            <View style={styles.body}>
              <View style={styles.sectionContainer}>
                <Text style={styles.sectionTitle}>BackgroundFetch Demo</Text>
              </View>
            </View>
          </ScrollView>
          <View style={styles.sectionContainer}>
            <FlatList
              data={this.state.events}
              renderItem={({item}) => (<Text>[{item.taskId}]: {item.timestamp}</Text>)}
              keyExtractor={item => item.timestamp}
            />
          </View>
        </SafeAreaView>
      </>
    );
  }
}

const styles = StyleSheet.create({
  scrollView: {
    backgroundColor: Colors.lighter,
  },
  body: {
    backgroundColor: Colors.white,
  },
  sectionContainer: {
    marginTop: 32,
    paddingHorizontal: 24,
  },
  sectionTitle: {
    fontSize: 24,
    fontWeight: '600',
    color: Colors.black,
  },
  sectionDescription: {
    marginTop: 8,
    fontSize: 18,
    fontWeight: '400',
    color: Colors.dark,
  },
});

export default App;

Executing Custom Tasks

In addition to the default background-fetch task defined by BackgroundFetch.configure, you may also execute your own arbitrary "oneshot" or periodic tasks (iOS requires additional Setup Instructions). However, all events will be fired into the Callback provided to BackgroundFetch#configure:

⚠️ iOS:

  • scheduleTask on iOS seems only to run when the device is plugged into power.
  • scheduleTask on iOS are designed for low-priority tasks, such as purging cache files — they tend to be unreliable for mission-critical tasks. scheduleTask will never run as frequently as you want.
  • The default fetch event is much more reliable and fires far more often.
  • scheduleTask on iOS stop when the user terminates the app. There is no such thing as stopOnTerminate: false for iOS.
// Step 1:  Configure BackgroundFetch as usual.
let status = await BackgroundFetch.configure({
  minimumFetchInterval: 15
}, async (taskId) => {  // <-- Event callback
  // This is the fetch-event callback.
  console.log("[BackgroundFetch] taskId: ", taskId);

  // Use a switch statement to route task-handling.
  switch (taskId) {
    case 'com.foo.customtask':
      print("Received custom task");
      break;
    default:
      print("Default fetch task");
  }
  // Finish, providing received taskId.
  BackgroundFetch.finish(taskId);
}, async (taskId) => {  // <-- Task timeout callback
  // This task has exceeded its allowed running-time.
  // You must stop what you're doing and immediately .finish(taskId)
  BackgroundFetch.finish(taskId);
});

// Step 2:  Schedule a custom "oneshot" task "com.foo.customtask" to execute 5000ms from now.
BackgroundFetch.scheduleTask({
  taskId: "com.foo.customtask",
  forceAlarmManager: true,
  delay: 5000  // <-- milliseconds
});

API Documentation

Config

Common Options

@param {Integer} minimumFetchInterval [15]

The minimum interval in minutes to execute background fetch events. Defaults to 15 minutes. Note: Background-fetch events will never occur at a frequency higher than every 15 minutes. Apple uses a secret algorithm to adjust the frequency of fetch events, presumably based upon usage patterns of the app. Fetch events can occur less often than your configured minimumFetchInterval.

@param {Integer} delay (milliseconds)

ℹ️ Valid only for BackgroundFetch.scheduleTask. The minimum number of milliseconds in future that task should execute.

@param {Boolean} periodic [false]

ℹ️ Valid only for BackgroundFetch.scheduleTask. Defaults to false. Set true to execute the task repeatedly. When false, the task will execute just once.

Android Options

@config {Boolean} stopOnTerminate [true]

Set false to continue background-fetch events after user terminates the app. Default to true.

@config {Boolean} startOnBoot [false]

Set true to initiate background-fetch events when the device is rebooted. Defaults to false.

NOTE: startOnBoot requires stopOnTerminate: false.

@config {Boolean} forceAlarmManager [false]

By default, the plugin will use Android's JobScheduler when possible. The JobScheduler API prioritizes for battery-life, throttling task-execution based upon device usage and battery level.

Configuring forceAlarmManager: true will bypass JobScheduler to use Android's older AlarmManager API, resulting in more accurate task-execution at the cost of higher battery usage.

let status = await BackgroundFetch.configure({
  minimumFetchInterval: 15,
  forceAlarmManager: true
}, async (taskId) => {  // <-- Event callback
  console.log("[BackgroundFetch] taskId: ", taskId);
  BackgroundFetch.finish(taskId);
}, async (taskId) => {  // <-- Task timeout callback
  // This task has exceeded its allowed running-time.
  // You must stop what you're doing and immediately .finish(taskId)
  BackgroundFetch.finish(taskId);
});
.
.
.
// And with with #scheduleTask
BackgroundFetch.scheduleTask({
  taskId: 'com.foo.customtask',
  delay: 5000,       // milliseconds
  forceAlarmManager: true,
  periodic: false
});

@config {Boolean} enableHeadless [false]

Set true to enable React Native's Headless JS mechanism, for handling fetch events after app termination.

  • 📂 index.js (MUST BE IN index.js):
import BackgroundFetch from "react-native-background-fetch";

let MyHeadlessTask = async (event) => {
  // Get task id from event {}:
  let taskId = event.taskId;
  let isTimeout = event.timeout;  // <-- true when your background-time has expired.
  if (isTimeout) {
    // This task has exceeded its allowed running-time.
    // You must stop what you're doing immediately finish(taskId)
    console.log('[BackgroundFetch] Headless TIMEOUT:', taskId);
    BackgroundFetch.finish(taskId);
    return;
  }
  console.log('[BackgroundFetch HeadlessTask] start: ', taskId);

  // Perform an example HTTP request.
  // Important:  await asychronous tasks when using HeadlessJS.
  let response = await fetch('https://reactnative.dev/movies.json');
  let responseJson = await response.json();
  console.log('[BackgroundFetch HeadlessTask] response: ', responseJson);

  // Required:  Signal to native code that your task is complete.
  // If you don't do this, your app could be terminated and/or assigned
  // battery-blame for consuming too much time in background.
  BackgroundFetch.finish(taskId);
}

// Register your BackgroundFetch HeadlessTask
BackgroundFetch.registerHeadlessTask(MyHeadlessTask);

@config {integer} requiredNetworkType [BackgroundFetch.NETWORK_TYPE_NONE]

Set basic description of the kind of network your job requires.

If your job doesn't need a network connection, you don't need to use this option as the default value is BackgroundFetch.NETWORK_TYPE_NONE.

NetworkTypeDescription
BackgroundFetch.NETWORK_TYPE_NONEThis job doesn't care about network constraints, either any or none.
BackgroundFetch.NETWORK_TYPE_ANYThis job requires network connectivity.
BackgroundFetch.NETWORK_TYPE_CELLULARThis job requires network connectivity that is a cellular network.
BackgroundFetch.NETWORK_TYPE_UNMETEREDThis job requires network connectivity that is unmetered. Most WiFi networks are unmetered, as in "you can upload as much as you like".
BackgroundFetch.NETWORK_TYPE_NOT_ROAMINGThis job requires network connectivity that is not roaming (being outside the country of origin)

@config {Boolean} requiresBatteryNotLow [false]

Specify that to run this job, the device's battery level must not be low.

This defaults to false. If true, the job will only run when the battery level is not low, which is generally the point where the user is given a "low battery" warning.

@config {Boolean} requiresStorageNotLow [false]

Specify that to run this job, the device's available storage must not be low.

This defaults to false. If true, the job will only run when the device is not in a low storage state, which is generally the point where the user is given a "low storage" warning.

@config {Boolean} requiresCharging [false]

Specify that to run this job, the device must be charging (or be a non-battery-powered device connected to permanent power, such as Android TV devices). This defaults to false.

@config {Boolean} requiresDeviceIdle [false]

When set true, ensure that this job will not run if the device is in active use.

The default state is false: that is, the for the job to be runnable even when someone is interacting with the device.

This state is a loose definition provided by the system. In general, it means that the device is not currently being used interactively, and has not been in use for some time. As such, it is a good time to perform resource heavy jobs. Bear in mind that battery usage will still be attributed to your application, and shown to the user in battery stats.


Methods

Method NameArgumentsReturnsNotes
configure{FetchConfig}, callbackFn, timeoutFnPromise<BackgroundFetchStatus>Configures the plugin's callbackFn and timeoutFn. This callback will fire each time a background-fetch event occurs in addition to events from #scheduleTask. The timeoutFn will be called when the OS reports your task is nearing the end of its allowed background-time.
scheduleTask{TaskConfig}Promise<boolean>Executes a custom task. The task will be executed in the same Callback function provided to #configure.
statuscallbackFnPromise<BackgroundFetchStatus>Your callback will be executed with the current status (Integer) 0: Restricted, 1: Denied, 2: Available. These constants are defined as BackgroundFetch.STATUS_RESTRICTED, BackgroundFetch.STATUS_DENIED, BackgroundFetch.STATUS_AVAILABLE (NOTE: Android will always return STATUS_AVAILABLE)
finishString taskIdVoidYou MUST call this method in your callbackFn provided to #configure in order to signal to the OS that your task is complete. iOS provides only 30s of background-time for a fetch-event -- if you exceed this 30s, iOS will kill your app.
startnonePromise<BackgroundFetchStatus>Start the background-fetch API. Your callbackFn provided to #configure will be executed each time a background-fetch event occurs. NOTE the #configure method automatically calls #start. You do not have to call this method after you #configure the plugin
stop[taskId:String]Promise<boolean>Stop the background-fetch API and all #scheduleTask from firing events. Your callbackFn provided to #configure will no longer be executed. If you provide an optional taskId, only that #scheduleTask will be stopped.

Debugging

iOS

🆕 BGTaskScheduler API for iOS 13+

  • ⚠️ At the time of writing, the new task simulator does not yet work in Simulator; Only real devices.
  • See Apple docs Starting and Terminating Tasks During Development
  • After running your app in XCode, Click the [||] button to initiate a Breakpoint.
  • In the console (lldb), paste the following command (Note: use cursor up/down keys to cycle through previously run commands):
e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateLaunchForTaskWithIdentifier:@"com.transistorsoft.fetch"]
  • Click the [ > ] button to continue. The task will execute and the Callback function provided to BackgroundFetch.configure will receive the event.

Simulating task-timeout events

  • Only the new BGTaskScheduler api supports simulated task-timeout events. To simulate a task-timeout, your fetchCallback must not call BackgroundFetch.finish(taskId):
let status = await BackgroundFetch.configure({
  minimumFetchInterval: 15
}, async (taskId) => {  // <-- Event callback.
  // This is the task callback.
  console.log("[BackgroundFetch] taskId", taskId);
  //BackgroundFetch.finish(taskId); // <-- Disable .finish(taskId) when simulating an iOS task timeout
}, async (taskId) => {  // <-- Event timeout callback
  // This task has exceeded its allowed running-time.
  // You must stop what you're doing and immediately .finish(taskId)
  print("[BackgroundFetch] TIMEOUT taskId:", taskId);
  BackgroundFetch.finish(taskId);
});
  • Now simulate an iOS task timeout as follows, in the same manner as simulating an event above:
e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateExpirationForTaskWithIdentifier:@"com.transistorsoft.fetch"]

Old BackgroundFetch API

  • Simulate background fetch events in XCode using Debug->Simulate Background Fetch
  • iOS can take some hours or even days to start a consistently scheduling background-fetch events since iOS schedules fetch events based upon the user's patterns of activity. If Simulate Background Fetch works, your can be sure that everything is working fine. You just need to wait.

Android

  • Observe plugin logs in $ adb logcat:
$ adb logcat *:S ReactNative:V ReactNativeJS:V TSBackgroundFetch:V
  • Simulate a background-fetch event on a device (insert <your.application.id>) (only works for sdk 21+:
$ adb shell cmd jobscheduler run -f <your.application.id> 999
  • For devices with sdk <21, simulate a "Headless JS" event with (insert <your.application.id>)
$ adb shell am broadcast -a <your.application.id>.event.BACKGROUND_FETCH

Download Details:
Author: transistorsoft
Source Code: https://github.com/transistorsoft/react-native-background-fetch
License: MIT license

#react  #reactnative  #mobileapp  #javascript