Lawrence  Lesch

Lawrence Lesch

1657176540

Map-reduce: Async Map-reduce Functions for Nodejs

Map Reduce for leveldb (via levelup)

Incremental map-reduces and real-time results.

Waat?

An "incremental map reduce" means when you update one key, only a relevant portion of the data needs to be recalculated.

"real-time results" means that you can listen to the database, and recieve change notifications on the fly! a la level-live-stream

If you just want something very simple, like mapping the date a blog post is created to the blog, then level-index may be enough.

Example

create a simple map-reduce

var LevelUp   = require('levelup')
var SubLevel  = require('level-sublevel')
var MapReduce = require('map-reduce')

var db = SubLevel(LevelUp(file))

var mapDb = 
  MapReduce(
    db, //the parent db
    'example',  //name.
    function (key, value, emit) {
      //perform some mapping.
      var obj = JSON.parse(value)
      //emit(key, value)
      //key may be an array of strings. 
      //value must be a string or buffer.
      emit(['all', obj.group], ''+obj.lines.length)
    },
    function (acc, value, key) {
      //reduce little into big
      //must return a string or buffer.
      return ''+(Number(acc) + Number(value))
    },
    //pass in the initial value for the reduce.
    //*must* be a string or buffer.
    '0'
  })
})

map-reduce uses level-trigger to make map reduces durable.

querying results.

  //get all the results in a specific group
  //start:[...] implies end:.. to be the end of that group.
  mapDb.createReadStream({range: ['all', group]}) 

  //get all the results in under a group.
  mapDb.createReadStream({range: ['all', true]}) 

  //get all the top level 
  mapDb.createReadStream({range: [true]})

complex aggregations

map-reduce with multiple levels of aggregation.

suppose we are building a database of all the street-food in the world. the data looks like this:

{
  country: USA | Germany | Cambodia, etc...
  state:   CA | NY | '', etc...
  city: Oakland | New York | Berlin | Phnom Penh, etc...
  type: taco | chili-dog | doner | noodles, etc...
}

We will aggregate to counts per-region, that look like this:

//say: under the key USA
{
  'taco': 23497,
  'chili-dog': 5643,
  etc...
}

first we'll map the raw data to ([country, state, city],type) tuples. then we'll count up all the instances of a particular type in that region!


var LevelUp   = require('levelup')
var SubLevel  = require('level-sublevel')
var MapReduce = require('map-reduce')

var db = SubLevel(LevelUp(file))
var mapDb = 
  MapReduce(
    db,
    'streetfood',
    function (key, value, emit) {
      //perform some mapping.
      var obj = JSON.parse(value)
      //emit(key, value)
      //key may be an array of strings. 
      //value must be a string or buffer.
      emit(
        [obj.country, obj.state || '', obj.city],
        //notice that we are just returning a string.
        JSON.stringify(obj.type)
      )
    },
    function (acc, value) {
      acc = JSON.parse(acc)
      value = JSON.parse(value)
      //check if this is top level data, like 'taco' or 'noodle'
      if('string' === typeof value) {
        //increment by one (remember to set as a number if it was undefined)
        acc[value] = (acc[value] || 0) ++
        return JSON.stringify(acc)
      }
      //if we get to here, we are combining two aggregates.
      //say, all the cities in a state, or all the countries in the world.
      //value and acc will both be objects {taco: number, doner: number2, etc...}

      for(var type in value) {
        //add the counts for each type together...
        //remembering to check that it is set as a value...
        acc[type] = (acc[type] || 0) + value[type]
      }
      //stringify the object, so that it can be written to disk!
      return JSON.stringify(acc)
    },
    '{}')

then query it like this:

mapDb.createReadStream({range: ['USA', 'CA', true]})
  .pipe(...)

retrive a specific result

pass db.get an array, and you can retrive a specific value, by group.

var userMapping = require("map-reduce")(
    db,
    "userPoints",
    function(key, value, emit){
        value = JSON.parse(value);
        var date = new Date(value.created);
        emit([value.user, date.getYear(), date.getMonth()], value.amount);
    },
    function(acc, value){
        return (Number(acc) + Number(value)).toString();
    },
    0
);

function getTotalPointsForUser(user, year, month, cb){
    userMapping.get([user, year, month], cb);
}

Author: Ddominictarr
Source Code: https://github.com/dominictarr/map-reduce 
License: MIT license

#node #map #async #javascript 

What is GEEK

Buddha Community

Map-reduce: Async Map-reduce Functions for Nodejs
Lawrence  Lesch

Lawrence Lesch

1657176540

Map-reduce: Async Map-reduce Functions for Nodejs

Map Reduce for leveldb (via levelup)

Incremental map-reduces and real-time results.

Waat?

An "incremental map reduce" means when you update one key, only a relevant portion of the data needs to be recalculated.

"real-time results" means that you can listen to the database, and recieve change notifications on the fly! a la level-live-stream

If you just want something very simple, like mapping the date a blog post is created to the blog, then level-index may be enough.

Example

create a simple map-reduce

var LevelUp   = require('levelup')
var SubLevel  = require('level-sublevel')
var MapReduce = require('map-reduce')

var db = SubLevel(LevelUp(file))

var mapDb = 
  MapReduce(
    db, //the parent db
    'example',  //name.
    function (key, value, emit) {
      //perform some mapping.
      var obj = JSON.parse(value)
      //emit(key, value)
      //key may be an array of strings. 
      //value must be a string or buffer.
      emit(['all', obj.group], ''+obj.lines.length)
    },
    function (acc, value, key) {
      //reduce little into big
      //must return a string or buffer.
      return ''+(Number(acc) + Number(value))
    },
    //pass in the initial value for the reduce.
    //*must* be a string or buffer.
    '0'
  })
})

map-reduce uses level-trigger to make map reduces durable.

querying results.

  //get all the results in a specific group
  //start:[...] implies end:.. to be the end of that group.
  mapDb.createReadStream({range: ['all', group]}) 

  //get all the results in under a group.
  mapDb.createReadStream({range: ['all', true]}) 

  //get all the top level 
  mapDb.createReadStream({range: [true]})

complex aggregations

map-reduce with multiple levels of aggregation.

suppose we are building a database of all the street-food in the world. the data looks like this:

{
  country: USA | Germany | Cambodia, etc...
  state:   CA | NY | '', etc...
  city: Oakland | New York | Berlin | Phnom Penh, etc...
  type: taco | chili-dog | doner | noodles, etc...
}

We will aggregate to counts per-region, that look like this:

//say: under the key USA
{
  'taco': 23497,
  'chili-dog': 5643,
  etc...
}

first we'll map the raw data to ([country, state, city],type) tuples. then we'll count up all the instances of a particular type in that region!


var LevelUp   = require('levelup')
var SubLevel  = require('level-sublevel')
var MapReduce = require('map-reduce')

var db = SubLevel(LevelUp(file))
var mapDb = 
  MapReduce(
    db,
    'streetfood',
    function (key, value, emit) {
      //perform some mapping.
      var obj = JSON.parse(value)
      //emit(key, value)
      //key may be an array of strings. 
      //value must be a string or buffer.
      emit(
        [obj.country, obj.state || '', obj.city],
        //notice that we are just returning a string.
        JSON.stringify(obj.type)
      )
    },
    function (acc, value) {
      acc = JSON.parse(acc)
      value = JSON.parse(value)
      //check if this is top level data, like 'taco' or 'noodle'
      if('string' === typeof value) {
        //increment by one (remember to set as a number if it was undefined)
        acc[value] = (acc[value] || 0) ++
        return JSON.stringify(acc)
      }
      //if we get to here, we are combining two aggregates.
      //say, all the cities in a state, or all the countries in the world.
      //value and acc will both be objects {taco: number, doner: number2, etc...}

      for(var type in value) {
        //add the counts for each type together...
        //remembering to check that it is set as a value...
        acc[type] = (acc[type] || 0) + value[type]
      }
      //stringify the object, so that it can be written to disk!
      return JSON.stringify(acc)
    },
    '{}')

then query it like this:

mapDb.createReadStream({range: ['USA', 'CA', true]})
  .pipe(...)

retrive a specific result

pass db.get an array, and you can retrive a specific value, by group.

var userMapping = require("map-reduce")(
    db,
    "userPoints",
    function(key, value, emit){
        value = JSON.parse(value);
        var date = new Date(value.created);
        emit([value.user, date.getYear(), date.getMonth()], value.amount);
    },
    function(acc, value){
        return (Number(acc) + Number(value)).toString();
    },
    0
);

function getTotalPointsForUser(user, year, month, cb){
    userMapping.get([user, year, month], cb);
}

Author: Ddominictarr
Source Code: https://github.com/dominictarr/map-reduce 
License: MIT license

#node #map #async #javascript 

Hire NodeJs Developer

Looking to build dynamic, extensively featured, and full-fledged web applications?

Hire NodeJs Developer to create a real-time, faster, and scalable application to accelerate your business. At HourlyDeveloper.io, we have a team of expert Node.JS developers, who have experience in working with Bootstrap, HTML5, & CSS, and also hold the knowledge of the most advanced frameworks and platforms.

Contact our experts: https://bit.ly/3hUdppS

#hire nodejs developer #nodejs developer #nodejs development company #nodejs development services #nodejs development #nodejs

Vincent Lab

Vincent Lab

1605017502

The Difference Between Regular Functions and Arrow Functions in JavaScript

Other then the syntactical differences. The main difference is the way the this keyword behaves? In an arrow function, the this keyword remains the same throughout the life-cycle of the function and is always bound to the value of this in the closest non-arrow parent function. Arrow functions can never be constructor functions so they can never be invoked with the new keyword. And they can never have duplicate named parameters like a regular function not using strict mode.

Here are a few code examples to show you some of the differences
this.name = "Bob";

const person = {
name: “Jon”,

<span style="color: #008000">// Regular function</span>
func1: <span style="color: #0000ff">function</span> () {
    console.log(<span style="color: #0000ff">this</span>);
},

<span style="color: #008000">// Arrow function</span>
func2: () =&gt; {
    console.log(<span style="color: #0000ff">this</span>);
}

}

person.func1(); // Call the Regular function
// Output: {name:“Jon”, func1:[Function: func1], func2:[Function: func2]}

person.func2(); // Call the Arrow function
// Output: {name:“Bob”}

The new keyword with an arrow function
const person = (name) => console.log("Your name is " + name);
const bob = new person("Bob");
// Uncaught TypeError: person is not a constructor

If you want to see a visual presentation on the differences, then you can see the video below:

#arrow functions #javascript #regular functions #arrow functions vs normal functions #difference between functions and arrow functions

How to Install NodeJS on Ubuntu 19.04

Overview
In this tutorial, you will learn how to install Node onto Ubuntu 19.04 Disco Dingo. We will cover installation from the default repositories and, for those wanting more recent releases, how to install from the NodeSource repositories.

Installing from Ubuntu
The Ubuntu 19.04 Disco Dingo repository includes NodeJS version 10.15. Like most packages found here, it certainly is not the most recent release; however, if stability is more important than features, it will be your preferred choice.

#nodejs #nodejs 10.x #nodejs 11.x #nodejs 12.x #nodejs 8.x

Top NodeJS Mobile App Development Company in USA

AppClues Infotech is one of the leading NodeJS app development company in USA that offering excellent NodeJS development services for web app development. We provide customized and high-quality NodeJS app development services to clients for different industries with advanced technology and functionalities.

Our dedicated app developers have years of experience in NodeJS development and thus successfully deliver cost-effective and highly customized solutions using the robust JavaScript engine of NodeJS.

Why Choose AppClues Infotech for NodeJS Application Development?
• Fast App Development
• Real-Time Application
• JSON (JavaScript Object Notation) in your Database
• Single Codebase
• Lower Cost
• Built-in NPM Support
• Inexpensive Testing and Hosting

For more info:
Website: https://www.appcluesinfotech.com/
Email: info@appcluesinfotech.com
Call: +1-978-309-9910

#top nodejs app development company in usa #nodejs web app development #nodejs development agency in usa #hire nodejs app developers in usa #custom nodejs app development company #best nodejs app development service company