Level-replicate: Master-master Replication with Levelup

level-replicate

master-master replication with levelup.

Implements scuttlebutt style handshake and then syncs data, then replicates real time changes.

Example

Replicate from a between two processes. One process starts a server, and another connects.

//master1.js
var level   = require('level')
var SubLevel  = require('level-sublevel')
var net       = require('net')
var Replicate = require('level-replicate')

//setup the database.
var db = SubLevel(level('/tmp/example-master'))

//install Master plugin!
var master = Replicate(db, 'master', "MASTER-1")

//create a server, and stream data to who ever connects.
net.createServer(function (stream) {
  stream.pipe(master.createStream({tail: true})).pipe(stream)
}).listen(9999, function () {
  console.log('master db listening on 9999')
})

Then, the code for the client!

//master2.js
var levelup   = require('level')
var SubLevel  = require('level-sublevel')
var net       = require('net')
var Replicate = require('level-replicate')

var db = SubLevel(level('/tmp/example-slave'))
var master = Replicate(db, 'master', "MASTER-2")

var stream = net.connect(9999)

stream.pipe(master.createStream({tail: true})).pipe(stream)

Wow, that was simple.

Binary data.

by default, level-replicate supports buffers by converting them to base64 via stream-serializer and json-buffer. If you desire more efficiency, use level-replicate/msgpack

var Replicate = require('level-replicate/msgpack')

var db = SubLevel(level('/tmp/example-slave'))
var master = Replicate(db, 'master', "MASTER-2")

Replicating Sublevels Recursively

//install Master plugin with the `recursive` option set to `true`.
var master = Replicate(db, 'master', "MASTER-1", {recursive: true})

// changes made to all sublevels of `db` will replicate!
db.sublevel('documents').put('foo', {bar: 'baz'}, function() { ... })

note: this depends on having level-sublevel@>=5.2.0 with level-hooks@>=4.5.0

Author: Dominictarr
Source Code: https://github.com/dominictarr/level-replicate
License: MIT license

#javascript #node #data

Level-master: Master-slave Replication with Levelup

level-master

master-slave replication with levelup.

Master/Slave replication is when you have writes to one db, and reads from another. You can have either many slaves, and a single master, or many masters, and a single slave.

Either way, master-slave replication depends on centralization, but is very simple, if it fits your usecase.

Example

Replicate from a central server to clients.

//master.js
var levelup  = require('levelup')
var SubLevel = require('level-sublevel')
var net      = require('net')
var Master   = require('level-master')

//setup the database.
var db = SubLevel(levelup('/tmp/example-master'))

//install Master plugin!
var master = Master(db, 'master')

//create a server, and stream data to who ever connects.
net.createServer(function (stream) {
  stream.pipe(master.createStream({tail: true})).pipe(stream)
}).listen(9999, function () {
  console.log('master db listening on 9999')
})

Then, the code for the client!

//slave.js
var levelup  = require('levelup')
var SubLevel = require('level-sublevel')
var net      = require('net')
var Master   = require('level-master')

var db = SubLevel(levelup('/tmp/example-slave'))
var slave = Master.Slave(db, 'slave')

var stream = net.connect(9999)

stream.pipe(slave.createStream()).pipe(stream)

Wow, that was simple.

did someone say "webscale"?

master in the middle

writes go to the master, and are then copied to many slaves. requests are load balanced across the slaves...

slave in the middle

A large amount of data is written to many masters. Each master aggregates the data (probably with a module like map-reduce), and then the aggregation is replicated into the central slave.

The central slave the applies the same aggregation again, giving you global data.

(it's important here that the data from each node does not collide. keys from each node need a different prefix or to be stored in a separate sublevel)

Author: Dominictarr
Source Code: https://github.com/dominictarr/level-master
License: MIT license

#javascript #node #master

Chloe Butler

1667425440

Pdf2gerb: Perl Script Converts PDF Files to Gerber format

pdf2gerb

Perl script converts PDF files to Gerber format

Pdf2Gerb generates Gerber 274X photoplotting and Excellon drill files from PDFs of a PCB. Up to three PDFs are used: the top copper layer, the bottom copper layer (for 2-sided PCBs), and an optional silk screen layer. The PDFs can be created directly from any PDF drawing software, or a PDF print driver can be used to capture the Print output if the drawing software does not directly support output to PDF.

The general workflow is as follows:

Design the PCB using your favorite CAD or drawing software.
Print the top and bottom copper and top silk screen layers to a PDF file.
Run Pdf2Gerb on the PDFs to create Gerber and Excellon files.
Use a Gerber viewer to double-check the output against the original PCB design.
Make adjustments as needed.
Submit the files to a PCB manufacturer.

Please note that Pdf2Gerb does NOT perform DRC (Design Rule Checks), as these will vary according to individual PCB manufacturer conventions and capabilities. Also note that Pdf2Gerb is not perfect, so the output files must always be checked before submitting them. As of version 1.6, Pdf2Gerb supports most PCB elements, such as round and square pads, round holes, traces, SMD pads, ground planes, no-fill areas, and panelization. However, because it interprets the graphical output of a Print function, there are limitations in what it can recognize (or there may be bugs).

See docs/Pdf2Gerb.pdf for install/setup, config, usage, and other info.

pdf2gerb_cfg.pm

#Pdf2Gerb config settings:
#Put this file in same folder/directory as pdf2gerb.pl itself (global settings),
#or copy to another folder/directory with PDFs if you want PCB-specific settings.
#There is only one user of this file, so we don't need a custom package or namespace.
#NOTE: all constants defined in here will be added to main namespace.
#package pdf2gerb_cfg;

use strict; #trap undef vars (easier debug)
use warnings; #other useful info (easier debug)


##############################################################################################
#configurable settings:
#change values here instead of in main pfg2gerb.pl file

use constant WANT_COLORS => ($^O !~ m/Win/); #ANSI colors no worky on Windows? this must be set < first DebugPrint() call

#just a little warning; set realistic expectations:
#DebugPrint("${\(CYAN)}Pdf2Gerb.pl ${\(VERSION)}, $^O O/S\n${\(YELLOW)}${\(BOLD)}${\(ITALIC)}This is EXPERIMENTAL software.  \nGerber files MAY CONTAIN ERRORS.  Please CHECK them before fabrication!${\(RESET)}", 0); #if WANT_DEBUG

use constant METRIC => FALSE; #set to TRUE for metric units (only affect final numbers in output files, not internal arithmetic)
use constant APERTURE_LIMIT => 0; #34; #max #apertures to use; generate warnings if too many apertures are used (0 to not check)
use constant DRILL_FMT => '2.4'; #'2.3'; #'2.4' is the default for PCB fab; change to '2.3' for CNC

use constant WANT_DEBUG => 0; #10; #level of debug wanted; higher == more, lower == less, 0 == none
use constant GERBER_DEBUG => 0; #level of debug to include in Gerber file; DON'T USE FOR FABRICATION
use constant WANT_STREAMS => FALSE; #TRUE; #save decompressed streams to files (for debug)
use constant WANT_ALLINPUT => FALSE; #TRUE; #save entire input stream (for debug ONLY)

#DebugPrint(sprintf("${\(CYAN)}DEBUG: stdout %d, gerber %d, want streams? %d, all input? %d, O/S: $^O, Perl: $]${\(RESET)}\n", WANT_DEBUG, GERBER_DEBUG, WANT_STREAMS, WANT_ALLINPUT), 1);
#DebugPrint(sprintf("max int = %d, min int = %d\n", MAXINT, MININT), 1); 

#define standard trace and pad sizes to reduce scaling or PDF rendering errors:
#This avoids weird aperture settings and replaces them with more standardized values.
#(I'm not sure how photoplotters handle strange sizes).
#Fewer choices here gives more accurate mapping in the final Gerber files.
#units are in inches
use constant TOOL_SIZES => #add more as desired
(
#round or square pads (> 0) and drills (< 0):
    .010, -.001,  #tiny pads for SMD; dummy drill size (too small for practical use, but needed so StandardTool will use this entry)
    .031, -.014,  #used for vias
    .041, -.020,  #smallest non-filled plated hole
    .051, -.025,
    .056, -.029,  #useful for IC pins
    .070, -.033,
    .075, -.040,  #heavier leads
#    .090, -.043,  #NOTE: 600 dpi is not high enough resolution to reliably distinguish between .043" and .046", so choose 1 of the 2 here
    .100, -.046,
    .115, -.052,
    .130, -.061,
    .140, -.067,
    .150, -.079,
    .175, -.088,
    .190, -.093,
    .200, -.100,
    .220, -.110,
    .160, -.125,  #useful for mounting holes
#some additional pad sizes without holes (repeat a previous hole size if you just want the pad size):
    .090, -.040,  #want a .090 pad option, but use dummy hole size
    .065, -.040, #.065 x .065 rect pad
    .035, -.040, #.035 x .065 rect pad
#traces:
    .001,  #too thin for real traces; use only for board outlines
    .006,  #minimum real trace width; mainly used for text
    .008,  #mainly used for mid-sized text, not traces
    .010,  #minimum recommended trace width for low-current signals
    .012,
    .015,  #moderate low-voltage current
    .020,  #heavier trace for power, ground (even if a lighter one is adequate)
    .025,
    .030,  #heavy-current traces; be careful with these ones!
    .040,
    .050,
    .060,
    .080,
    .100,
    .120,
);
#Areas larger than the values below will be filled with parallel lines:
#This cuts down on the number of aperture sizes used.
#Set to 0 to always use an aperture or drill, regardless of size.
use constant { MAX_APERTURE => max((TOOL_SIZES)) + .004, MAX_DRILL => -min((TOOL_SIZES)) + .004 }; #max aperture and drill sizes (plus a little tolerance)
#DebugPrint(sprintf("using %d standard tool sizes: %s, max aper %.3f, max drill %.3f\n", scalar((TOOL_SIZES)), join(", ", (TOOL_SIZES)), MAX_APERTURE, MAX_DRILL), 1);

#NOTE: Compare the PDF to the original CAD file to check the accuracy of the PDF rendering and parsing!
#for example, the CAD software I used generated the following circles for holes:
#CAD hole size:   parsed PDF diameter:      error:
#  .014                .016                +.002
#  .020                .02267              +.00267
#  .025                .026                +.001
#  .029                .03167              +.00267
#  .033                .036                +.003
#  .040                .04267              +.00267
#This was usually ~ .002" - .003" too big compared to the hole as displayed in the CAD software.
#To compensate for PDF rendering errors (either during CAD Print function or PDF parsing logic), adjust the values below as needed.
#units are pixels; for example, a value of 2.4 at 600 dpi = .0004 inch, 2 at 600 dpi = .0033"
use constant
{
    HOLE_ADJUST => -0.004 * 600, #-2.6, #holes seemed to be slightly oversized (by .002" - .004"), so shrink them a little
    RNDPAD_ADJUST => -0.003 * 600, #-2, #-2.4, #round pads seemed to be slightly oversized, so shrink them a little
    SQRPAD_ADJUST => +0.001 * 600, #+.5, #square pads are sometimes too small by .00067, so bump them up a little
    RECTPAD_ADJUST => 0, #(pixels) rectangular pads seem to be okay? (not tested much)
    TRACE_ADJUST => 0, #(pixels) traces seemed to be okay?
    REDUCE_TOLERANCE => .001, #(inches) allow this much variation when reducing circles and rects
};

#Also, my CAD's Print function or the PDF print driver I used was a little off for circles, so define some additional adjustment values here:
#Values are added to X/Y coordinates; units are pixels; for example, a value of 1 at 600 dpi would be ~= .002 inch
use constant
{
    CIRCLE_ADJUST_MINX => 0,
    CIRCLE_ADJUST_MINY => -0.001 * 600, #-1, #circles were a little too high, so nudge them a little lower
    CIRCLE_ADJUST_MAXX => +0.001 * 600, #+1, #circles were a little too far to the left, so nudge them a little to the right
    CIRCLE_ADJUST_MAXY => 0,
    SUBST_CIRCLE_CLIPRECT => FALSE, #generate circle and substitute for clip rects (to compensate for the way some CAD software draws circles)
    WANT_CLIPRECT => TRUE, #FALSE, #AI doesn't need clip rect at all? should be on normally?
    RECT_COMPLETION => FALSE, #TRUE, #fill in 4th side of rect when 3 sides found
};

#allow .012 clearance around pads for solder mask:
#This value effectively adjusts pad sizes in the TOOL_SIZES list above (only for solder mask layers).
use constant SOLDER_MARGIN => +.012; #units are inches

#line join/cap styles:
use constant
{
    CAP_NONE => 0, #butt (none); line is exact length
    CAP_ROUND => 1, #round cap/join; line overhangs by a semi-circle at either end
    CAP_SQUARE => 2, #square cap/join; line overhangs by a half square on either end
    CAP_OVERRIDE => FALSE, #cap style overrides drawing logic
};
    
#number of elements in each shape type:
use constant
{
    RECT_SHAPELEN => 6, #x0, y0, x1, y1, count, "rect" (start, end corners)
    LINE_SHAPELEN => 6, #x0, y0, x1, y1, count, "line" (line seg)
    CURVE_SHAPELEN => 10, #xstart, ystart, x0, y0, x1, y1, xend, yend, count, "curve" (bezier 2 points)
    CIRCLE_SHAPELEN => 5, #x, y, 5, count, "circle" (center + radius)
};
#const my %SHAPELEN =
#Readonly my %SHAPELEN =>
our %SHAPELEN =
(
    rect => RECT_SHAPELEN,
    line => LINE_SHAPELEN,
    curve => CURVE_SHAPELEN,
    circle => CIRCLE_SHAPELEN,
);

#panelization:
#This will repeat the entire body the number of times indicated along the X or Y axes (files grow accordingly).
#Display elements that overhang PCB boundary can be squashed or left as-is (typically text or other silk screen markings).
#Set "overhangs" TRUE to allow overhangs, FALSE to truncate them.
#xpad and ypad allow margins to be added around outer edge of panelized PCB.
use constant PANELIZE => {'x' => 1, 'y' => 1, 'xpad' => 0, 'ypad' => 0, 'overhangs' => TRUE}; #number of times to repeat in X and Y directions

# Set this to 1 if you need TurboCAD support.
#$turboCAD = FALSE; #is this still needed as an option?

#CIRCAD pad generation uses an appropriate aperture, then moves it (stroke) "a little" - we use this to find pads and distinguish them from PCB holes. 
use constant PAD_STROKE => 0.3; #0.0005 * 600; #units are pixels
#convert very short traces to pads or holes:
use constant TRACE_MINLEN => .001; #units are inches
#use constant ALWAYS_XY => TRUE; #FALSE; #force XY even if X or Y doesn't change; NOTE: needs to be TRUE for all pads to show in FlatCAM and ViewPlot
use constant REMOVE_POLARITY => FALSE; #TRUE; #set to remove subtractive (negative) polarity; NOTE: must be FALSE for ground planes

#PDF uses "points", each point = 1/72 inch
#combined with a PDF scale factor of .12, this gives 600 dpi resolution (1/72 * .12 = 600 dpi)
use constant INCHES_PER_POINT => 1/72; #0.0138888889; #multiply point-size by this to get inches

# The precision used when computing a bezier curve. Higher numbers are more precise but slower (and generate larger files).
#$bezierPrecision = 100;
use constant BEZIER_PRECISION => 36; #100; #use const; reduced for faster rendering (mainly used for silk screen and thermal pads)

# Ground planes and silk screen or larger copper rectangles or circles are filled line-by-line using this resolution.
use constant FILL_WIDTH => .01; #fill at most 0.01 inch at a time

# The max number of characters to read into memory
use constant MAX_BYTES => 10 * M; #bumped up to 10 MB, use const

use constant DUP_DRILL1 => TRUE; #FALSE; #kludge: ViewPlot doesn't load drill files that are too small so duplicate first tool

my $runtime = time(); #Time::HiRes::gettimeofday(); #measure my execution time

print STDERR "Loaded config settings from '${\(__FILE__)}'.\n";
1; #last value must be truthful to indicate successful load


#############################################################################################
#junk/experiment:

#use Package::Constants;
#use Exporter qw(import); #https://perldoc.perl.org/Exporter.html

#my $caller = "pdf2gerb::";

#sub cfg
#{
#    my $proto = shift;
#    my $class = ref($proto) || $proto;
#    my $settings =
#    {
#        $WANT_DEBUG => 990, #10; #level of debug wanted; higher == more, lower == less, 0 == none
#    };
#    bless($settings, $class);
#    return $settings;
#}

#use constant HELLO => "hi there2"; #"main::HELLO" => "hi there";
#use constant GOODBYE => 14; #"main::GOODBYE" => 12;

#print STDERR "read cfg file\n";

#our @EXPORT_OK = Package::Constants->list(__PACKAGE__); #https://www.perlmonks.org/?node_id=1072691; NOTE: "_OK" skips short/common names

#print STDERR scalar(@EXPORT_OK) . " consts exported:\n";
#foreach(@EXPORT_OK) { print STDERR "$_\n"; }
#my $val = main::thing("xyz");
#print STDERR "caller gave me $val\n";
#foreach my $arg (@ARGV) { print STDERR "arg $arg\n"; }

Download Details:

Author: swannman
Source Code: https://github.com/swannman/pdf2gerb

License: GPL-3.0 license

#perl

Reid Rohan

1657866720

Level-replicate: Master-master Replication with Levelup

level-replicate

master-master replication with levelup.

Implements scuttlebutt style handshake and then syncs data, then replicates real time changes.

Example

Replicate from a between two processes. One process starts a server, and another connects.

//master1.js
var level   = require('level')
var SubLevel  = require('level-sublevel')
var net       = require('net')
var Replicate = require('level-replicate')

//setup the database.
var db = SubLevel(level('/tmp/example-master'))

//install Master plugin!
var master = Replicate(db, 'master', "MASTER-1")

//create a server, and stream data to who ever connects.
net.createServer(function (stream) {
  stream.pipe(master.createStream({tail: true})).pipe(stream)
}).listen(9999, function () {
  console.log('master db listening on 9999')
})

Then, the code for the client!

//master2.js
var levelup   = require('level')
var SubLevel  = require('level-sublevel')
var net       = require('net')
var Replicate = require('level-replicate')

var db = SubLevel(level('/tmp/example-slave'))
var master = Replicate(db, 'master', "MASTER-2")

var stream = net.connect(9999)

stream.pipe(master.createStream({tail: true})).pipe(stream)

Wow, that was simple.

Binary data.

by default, level-replicate supports buffers by converting them to base64 via stream-serializer and json-buffer. If you desire more efficiency, use level-replicate/msgpack

var Replicate = require('level-replicate/msgpack')

var db = SubLevel(level('/tmp/example-slave'))
var master = Replicate(db, 'master', "MASTER-2")

Replicating Sublevels Recursively

//install Master plugin with the `recursive` option set to `true`.
var master = Replicate(db, 'master', "MASTER-1", {recursive: true})

// changes made to all sublevels of `db` will replicate!
db.sublevel('documents').put('foo', {bar: 'baz'}, function() { ... })

note: this depends on having level-sublevel@>=5.2.0 with level-hooks@>=4.5.0

Author: Dominictarr
Source Code: https://github.com/dominictarr/level-replicate
License: MIT license

#javascript #node #data

Alayna Rippin

1598108400

35 Scrum Master Interview Questions and Answers [2020]

From the desk of a brilliant weirdo #1:

Are your hand shivering and pouring yourself with sweat right before your scrum master interview?

Do you feel quite uncomfortable, maybe unprepared, or maybe even lost in the darkness to what you can expect from the interviewer?

I’ve gone through some of the most common scrum interview questions and provided answers as well.

Big credits to Stefan Wolper, an agile coach, and his ebook about Scrum Masters.

Without further ado:

Scrum Master Role & Responsibilities (Overview)
It’s the Scrum Master’s responsibility to bring the development team on the same page by teaching them all the values and principles that fundamentally shape Scrum and, respectively, Agile.

The Scrum Master’s role revolves around helping and guiding the scrum team rather than managing it. He is also responsible for keeping the project-related stakeholders in the loop as well.

One of the Scrum Master’s primary responsibilities is to ensure a distraction-free environment for his team. The Scrum Master also participates in creating the product backlog helping the PO (product owner) prioritize tasks by importance.

And he is the one ultimately responsible for imbuing the organization with the Scrum values & principles by communicating with other scrum masters, stakeholders, etc.

So, if you are determined to follow a Scrum Master career path, earn the respective salary of a certified scrum master, and qualify for tons of scrum master interviews with the help of your brilliant scrum master resume, then this article can get you prepared for your first or next interview.

Now, before we dive in, here are a couple of…

Things (+ Tips) To Know Before The Scrum Master Interview
The questions you will encounter during a scrum master interview can vary widely based on the interviewer’s company and its values.

You might get asked kinds of questions about indifferent team members, keeping stakeholders in the loop, agile metrics, the barebones of scrum, and more. We’ll pretty much cover everything from exam-like, SDLC, and behavioral to more advanced technical and testing-related interview questions.

Of course, your experience will impact the interview process, as well. Senior scrum masters will get asked questions that an entry-level scrum master wouldn’t stumble upon until later in their career.

Also, regardless of your experience as a scrum master, you should always display curiosity and excitement in front of your interviewer. Tell the interviewer about your background and experience as a scrum master. Share what excites you (and what you think makes you a good fit) about the job position you are applying for. It will, without a doubt, boost your chances of getting hired.

Alrighty, let’s get you prepared for your scrum master interview…

Scrum Master Interview Questions and Answers
[Agile & Scrum-Related]
Explain to me the Agile mindset in your own terms?
If you can’t answer this question on your own then it’s best to go over the Agile Manifesto a couple of times.

How can you say that Agile is working in your organization?
When the products you deliver to the end-user result in higher customer retention rates, upsells, and more customer acquisition.
When the team genuinely feels happy, it can be considered that you are doing alright. There are no team members complaining about work environments. Nobody is leaving (or getting fired) the team.

Lots of submitted job applications can also be considered as a metric for a well-performing Agile organization.

If there aren’t piles of technical debt, bugs aren’t present, and there’s less time spent on maintenance, you can say that Agile is going well in your company.

What is the difference between Agile and Scrum (another newbie question)?
If you still struggle with explaining in simple terms, here’s a more straightforward definition that can make things easier for you: Agile is a philosophy. Scrum is a framework and subset of Agile.

Agile describes a set of ideals and values. In contrast, Scrum gives you principles through which you can make your project management easier.

What’s your criticism of Scrum?
This can be quite an interesting question as it’s quite unusual. It shows your point of view and rationalization process to a degree. That’s something you should personally answer without having someone else tell you the “correct” answer. Some developers consider the daily scrums to be a waste of time to a certain degree. Others believe that sprints can be managed better allocating more time to fixing the technical debt.

#scrum #scrum master #interview questions #interview answers #agile coach #scrum guide #scrum agile #scrum master certification #scrum leadership #scrum master role

August Larson

1624335780

How to Master Python for Data Science

Here’s the Essential Python you Need for Data Science

you’re embarking on your journey into data science and everyone recommends that you start with learning how to code. You decided on Python and are now paralyzed by the large piles of learning resources that are at your disposal. Perhaps you are overwhelmed and owing to analysis paralysis, you are procrastinating your first steps in learning how to code in Python.

In this article, I’ll be your guide and take you on a journey of exploring the essential bare minimal knowledge that you need in order to master Python for getting started in data science. I will assume that you have no prior coding experience or that you may come from a non-technical background. However, if you are coming from a technical or computer science background and have knowledge of a prior programming language and would like to transition to Python, you can use this article as a high-level overview to get acquainted with the gist of the Python language. Either way, it is the aim of this article to navigate you through the landscape of the Python language at their intersection with data science, which will help you get started in no time.

#python #data-science #programming #how to master python for data science #master python #master python for data science

Level-replicate: Master-master Replication with Levelup

level-replicate

Example

Binary data.

Replicating Sublevels Recursively

What is GEEK

Buddha Community

Level-master: Master-slave Replication with Levelup

level-master

Example

did someone say "webscale"?

master in the middle

slave in the middle

Pdf2gerb: Perl Script Converts PDF Files to Gerber format

pdf2gerb

pdf2gerb_cfg.pm

Download Details:

Level-replicate: Master-master Replication with Levelup

level-replicate

Example

Binary data.

Replicating Sublevels Recursively

35 Scrum Master Interview Questions and Answers [2020]

How to Master Python for Data Science

Here’s the Essential Python you Need for Data Science