How I created a simple Long-Short Equity trading in JavaScript using Stock API

How I created a simple Long-Short Equity trading in JavaScript using Stock API

In this post, I will be sharing how I created a simple Long-Short Equity trading script in JavaScript (a Node.js app) using Alpaca’s free paper trading API.

Alpaca’s paper trading API simulates the live trading API in a paper environment so we don’t have to worry about trading with real money when testing a strategy.

To get access to the paper trading API, sign up here 17. Once you’ve signed up, you should now have access to the Alpaca paper trading dashboard, which allows you to monitor your paper trading and gives you the access keys to link your script to your account.

The Strategy

The example strategy I will be implementing in this script is the Long-Short equity strategy. The idea behind this strategy is to rank a group of stocks from better performing to worse performing stocks, then to long the top stocks in the list and short the bottom ones. By doing this, the algorithm takes maximizes potential gains by longing what it predicts are going to be the better performing stocks, and shorting the predicted worse performing stocks.

The driving force behind success, though, is how the stocks are ranked. The ranking can make or break your algorithm. If your ranking system is good, ideally you’ll be longing stocks that are green the following days and shorting ones that aren’t doing so well. However, a bad ranking system could see your algorithm longing poorly performing stocks and, perhaps more dangerously, shorting high performing stocks, which could cause your algorithm to quickly lose you money.

Getting Started

If you already know how to set up a Node.js environment and run Node.js apps, you can skip this section. If not, this section will detail how to set up a Node.js environment for running the Long-Short script.

First, you need to download Node.js from the internet and run the installer. You can download it here 1. Once Node.js is finished downloading, create a new folder and run following command while in the folder:

npm init && npm install

Once you run this command, you’ll be presented with options while in the terminal. You can just hit enter through all the options. After completing all of the options, your Node.js application will be initialized.

To run this particular script, you will also need to install the Alpaca node module, which can be found in the Alpaca JavaScript SDK 5. To install, you just run the command:

npm install --save @alpacahq/alpaca-trade-api

while in your Node.js directory, and the Alpaca node module should install.

Once you’ve completed all these steps, all you need to do is download the long-short.js file (found in the Alpaca JavaScript SDK, under the “examples” folder) and place it in the folder (making sure to input your API keys into the script). Now you just run the command node long-short while in the folder, and the script should start running. You can stop the script anytime using the command Ctrl+C in the command line.

Overview

Before we dive into the code, I’ll give a high-level overview of this script. Here’s an outline:

  1. The script checks if the market is open. If not, the script spins for a minute and does nothing. If open, the script begins to run.
  2. Rebalance the portfolio every minute by getting a list of stocks to long and short, and then executing those orders based on amount of equity allocated to each bucket.
  3. When the time is within 15 minutes of market close, the script clears all positions and sleeps until the market is closed.
  4. After the market is closed, the script awakes and begins checking if the market is open, effectively looping back to step

And that’s it! Now that we have a decent, high-level understanding, we can dive into the code so that you can fully understand what’s going on. This way, you will be able to make changes along the way so that you can customize the script to fit your needs.

PS: If you find yourself not understanding the code well enough to make changes after the walk-through, don’t worry. The last section of this article will explicitly show you some simple customizations you can make without needing to understand the code.

The Code

I’ll walk you through each major component of the script. First, I have input constants so that I can link my account to the script. To get your keys, hit the “Generate Keys” on your Alpaca paper trading dashboard and copy and paste them here:

const API_KEY = 'YOUR_API_KEY_HERE';
const API_SECRET = 'YOUR_API_SECRET_HERE';
const PAPER = true;

Next, I initialized a class which will contain all methods that the script will use. In the class initializer, the script initially establishes a connection to the Alpaca API, and initializes a bunch of class variables that will be used throughout the script. You can customize the stocks that you want to rank in the this.allStocks array, making sure to keep the formatting as a string array of stock symbols. The list of stocks below is an example and is for illustrative purposes only.

class LongShort {
 constructor(API_KEY, API_SECRET, PAPER){
   this.Alpaca = require('@alpacahq/alpaca-trade-api');
   this.alpaca = new this.Alpaca({
     keyId: API_KEY, 
     secretKey: API_SECRET, 
     paper: PAPER
   });
   this.allStocks = ['DOMO', 'TLRY', 'SQ', 'MRO', 'AAPL', 'GM', 'SNAP', 'SHOP', 'SPLK', 'BA', 'AMZN', 'SUI', 'SUN', 'TSLA', 'CGC', 'SPWR', 'NIO', 'CAT', 'MSFT', 'PANW', 'OKTA', 'TWTR', 'TM', 'RTN', 'ATVI', 'GS', 'BAC', 'MS', 'TWLO', 'QCOM'];
   // Format the allStocks variable for use in the class.
   var temp = [];
   this.allStocks.forEach((stockName) => {
     temp.push({name: stockName, pc: 0});
   });
   this.allStocks = temp.slice();

   this.long = [];
   this.short = [];
   this.qShort = null;
   this.qLong = null;
   this.adjustedQLong = null;
   this.adjustedQShort = null;
   this.blacklist = new Set();
   this.longAmount = 0;
   this.shortAmount = 0;
   this.timeToClose = null;
 }
}

// Run the LongShort class
var ls = new LongShort(API_KEY, API_SECRET, PAPER);
ls.run();

The function run() will be the main method of this algorithm. First, it clears all incomplete orders so that none of them fill while rebalancing the portfolio, which would interfere with the rebalancing. Next, it waits until the market opens by awaiting a promise from awaitMarketOpen(), which will only resolve when the market is open. Finally, there’s the trading portion of the script (handled mainly by the function rebalance()), which executes after every minute until 15 minutes before market close, where it then closes all positions and sleeps until the market is closed, where it then calls run() again and starts the cycle all over again.

async run(){
 // First, cancel any existing orders so they don't impact our buying power.
 var orders;
 await this.alpaca.getOrders({
   status: 'open',
   direction: 'desc'
 }).then((resp) => {
   orders = resp;
 }).catch((err) => {console.log(err.error);});
 var promOrders = [];
 orders.forEach((order) => {
   promOrders.push(new Promise(async (resolve, reject) => {
     await this.alpaca.cancelOrder(order.id).catch((err) => {console.log(err.error);});
     resolve();
   }));
 });
 await Promise.all(promOrders);

 // Wait for market to open.
 console.log("Waiting for market to open...");
 var promMarket = this.awaitMarketOpen();
 await promMarket;
 console.log("Market opened.");

 // Rebalance the portfolio every minute, making necessary trades.
 var spin = setInterval(async () => {

   // Figure out when the market will close so we can prepare to sell beforehand.
   await this.alpaca.getClock().then((resp) =>{
     var closingTime = new Date(resp.next_close.substring(0, resp.next_close.length - 6));
     var currTime = new Date(resp.timestamp.substring(0, resp.timestamp.length - 6));
     this.timeToClose = Math.abs(closingTime - currTime);
   }).catch((err) => {console.log(err.error);});

   if(this.timeToClose < (60000 * 15)) {
     // Close all positions when 15 minutes til market close.
     console.log("Market closing soon. Closing positions.");

     await this.alpaca.getPositions().then(async (resp) => {
       var promClose = [];
       resp.forEach((position) => {
         promClose.push(new Promise(async (resolve, reject) => {
           var orderSide;
           if(position.side == 'long') orderSide = 'sell';
           else orderSide = 'buy';
           var quantity = Math.abs(position.qty);
           await this.submitOrder(quantity, position.symbol, orderSide);
           resolve();
         }));
       });

       await Promise.all(promClose);
     }).catch((err) => {console.log(err.error);});
     clearInterval(spin);
     console.log("Sleeping until market close (15 minutes).");
     setTimeout(() => {
       // Run script again after market close for next trading day.
       this.run();
     }, 60000*15);
   }
   else {
     // Rebalance the portfolio.
     await this.rebalance();
   }
 }, 60000);
}

// Spin until the market is open
awaitMarketOpen(){
 var prom = new Promise(async (resolve, reject) => {
   var isOpen = false;
   await this.alpaca.getClock().then(async (resp) => {
     if(resp.is_open) {
       resolve();
     }
     else {
       var marketChecker = setInterval(async () => {
         await this.alpaca.getClock().then((resp) => {
           isOpen = resp.is_open;
           if(isOpen) {
             clearInterval(marketChecker);
             resolve();
           }
           else {
             var openTime = new Date(resp.next_open.substring(0, resp.next_close.length - 6));
             var currTime = new Date(resp.timestamp.substring(0, resp.timestamp.length - 6));
             this.timeToClose = Math.floor((openTime - currTime) / 1000 / 60);
             console.log(this.timeToClose + " minutes til next market open.")
           }
         }).catch((err) => {console.log(err.error);});
       }, 60000);
     }
   });
 });
 return prom;
}

// Submit an order if quantity is above 0.
async submitOrder(quantity,stock,side){
 var prom = new Promise(async (resolve,reject) => {
   if(quantity > 0){
     await this.alpaca.createOrder({
       symbol: stock,
       qty: quantity,
       side: side,
       type: 'market',
       time_in_force: 'day',
     }).then(() => {
       console.log("Market order of | " + quantity + " " + stock + " " + side + " | completed.");
       resolve(true);
     }).catch((err) => {
       console.log("Order of | " + quantity + " " + stock + " " + side + " | did not go through.");
       resolve(false);
     });
   }
   else {
     console.log("Quantity is <=0, order of | " + quantity + " " + stock + " " + side + " | not sent.");
     resolve(true);
   }
 });
 return prom;
}

The function rebalance() is where all the trading happens. It first calls the function rerank(), which ranks the stocks from last to first, and then assigns them to either the long or short (or nothing) bucket, and determines the quantity that should be shorted for each bucket. The ranking mechanism itself is handled by rank(). After calling rerank(), rebalance() will adjust any positions that I have with the new quantities (either ordering a corrective amount or clearing the position if the stock is no longer in either list). After clearing positions, the script then market orders all the new stocks with the respective quantities using sendBatchOrder(). sendBatchOrder() is special because it’ll return the stocks that succeeded to order, and stocks that failed to order. After receiving which stocks orders failed, rebalance() reorders more of the stocks that executed successfully using the leftover equity that would’ve been used for stock orders that failed. Once the stocks are reordered, the trading will end, and the script will wait a minute to trade again.

// Rebalance our position after an update.
async rebalance(){
 await this.rerank();

 // Clear existing orders again.
 var orders;
 await this.alpaca.getOrders({
   status: 'open',
   direction: 'desc'
 }).then((resp) => {
   orders = resp;
 }).catch((err) => {console.log(err.error);});
 var promOrders = [];
 orders.forEach((order) => {
   promOrders.push(new Promise(async (resolve, reject) => {
     await this.alpaca.cancelOrder(order.id).catch((err) => {console.log(err.error);});
     resolve();
   }));
 });
 await Promise.all(promOrders);

 console.log("We are taking a long position in: " + this.long.toString());
 console.log("We are taking a short position in: " + this.short.toString());
 // Remove positions that are no longer in the short or long list, and make a list of positions that do not need to change. Adjust position quantities if needed.
 var positions;
 await this.alpaca.getPositions().then((resp) => {
   positions = resp;
 }).catch((err) => {console.log(err.error);});
 var promPositions = [];
 var executed = {long:[], short:[]};
 var side;
 this.blacklist.clear();
 positions.forEach((position) => {
   promPositions.push(new Promise(async (resolve, reject) => {
     if(this.long.indexOf(position.symbol) < 0){
       // Position is not in long list.
       if(this.short.indexOf(position.symbol) < 0){
         // Position not in short list either. Clear position.
         if(position.side == "long") side = "sell";
         else side = "buy";
         var promCO = this.submitOrder(Math.abs(position.qty), position.symbol, side);
         await promCO.then(() => {
           resolve();
         });
       }
       else{
         // Position in short list.
         if(position.side == "long") {
           // Position changed from long to short. Clear long position and short instead
           var promCS = this.submitOrder(position.qty, position.symbol, "sell");
           await promCS.then(() => {
             resolve();
           });
         }
         else {
           if(Math.abs(position.qty) == this.qShort){
             // Position is where we want it. Pass for now.
           }
           else{
             // Need to adjust position amount
             var diff = Number(Math.abs(position.qty)) - Number(this.qShort);
             if(diff > 0){
               // Too many short positions. Buy some back to rebalance.
               side = "buy";
             }
             else{
               // Too little short positions. Sell some more.
               side = "sell";
             }
             var promRebalance = this.submitOrder(Math.abs(diff), position.symbol, side);
             await promRebalance;
           }
           executed.short.push(position.symbol);
           this.blacklist.add(position.symbol);
           resolve();
         }
       }
     }
     else{
       // Position in long list.
       if(position.side == "short"){
         // Position changed from short to long. Clear short position and long instead.
         var promCS = this.submitOrder(Math.abs(position.qty), position.symbol, "buy");
         await promCS.then(() => {
           resolve();
         });
       }
       else{
         if(position.qty == this.qLong){
           // Position is where we want it. Pass for now.
         }
         else{
           // Need to adjust position amount.
           var diff = Number(position.qty) - Number(this.qLong);
           if(diff > 0){
             // Too many long positions. Sell some to rebalance.
             side = "sell";
           }
           else{
             // Too little long positions. Buy some more.
             side = "buy";
           }
           var promRebalance = this.submitOrder(Math.abs(diff), position.symbol, side);
           await promRebalance;
         }
         executed.long.push(position.symbol);
         this.blacklist.add(position.symbol);
         resolve();
       }
     }
   }));
 });
 await Promise.all(promPositions);

 // Send orders to all remaining stocks in the long and short list.
 var promLong = this.sendBatchOrder(this.qLong, this.long, 'buy');
 var promShort = this.sendBatchOrder(this.qShort, this.short, 'sell');

 var promBatches = [];
 this.adjustedQLong = -1;
 this.adjustedQShort = -1;

 await Promise.all([promLong, promShort]).then(async (resp) => {
   // Handle rejected/incomplete orders.
   resp.forEach(async (arrays, i) => {
     promBatches.push(new Promise(async (resolve, reject) => {
       if(i == 0) {
         arrays[1] = arrays[1].concat(executed.long);
         executed.long = arrays[1].slice();
       }
       else {
         arrays[1] = arrays[1].concat(executed.short);
         executed.short = arrays[1].slice();
       }
       // Return orders that didn't complete, and determine new quantities to purchase.
       if(arrays[0].length > 0 && arrays[1].length > 0){
         var promPrices = this.getTotalPrice(arrays[1]);

         await Promise.all(promPrices).then((resp) => {
           var completeTotal = resp.reduce((a, b) => a + b, 0);
           if(completeTotal != 0){
             if(i == 0){
               this.adjustedQLong = Math.floor(this.longAmount / completeTotal);
             }
             else{
               this.adjustedQShort = Math.floor(this.shortAmount / completeTotal);
             }
           }
         });
       }
       resolve();
     }));
   });
   await Promise.all(promBatches);
 }).then(async () => {
   // Reorder stocks that didn't throw an error so that the equity quota is reached.
   var promReorder = new Promise(async (resolve, reject) => {
     var promLong = [];
     if(this.adjustedQLong >= 0){
       this.qLong = this.adjustedQLong - this.qLong;
       executed.long.forEach(async (stock) => {
         promLong.push(new Promise(async (resolve, reject) => {
           var promLong = this.submitOrder(this.qLong, stock, 'buy');
           await promLong;
           resolve();
         }));
       });
     }

     var promShort = [];
     if(this.adjustedQShort >= 0){
       this.qShort = this.adjustedQShort - this.qShort;
       executed.short.forEach(async(stock) => {
         promShort.push(new Promise(async (resolve, reject) => {
           var promShort = this.submitOrder(this.qShort, stock, 'sell');
           await promShort;
           resolve();
         }));
       });
     }
     var allProms = promLong.concat(promShort);
     if(allProms.length > 0){
       await Promise.all(allProms);
     }
     resolve();
   });
   await promReorder;
 });
}

// Re-rank all stocks to adjust longs and shorts.
async rerank(){
 await this.rank();

 // Grabs the top and bottom quarter of the sorted stock list to get the long and short lists.
 var longShortAmount = Math.floor(this.allStocks.length / 4);
 this.long = [];
 this.short = [];
 for(var i = 0; i < this.allStocks.length; i++){
   if(i < longShortAmount) this.short.push(this.allStocks[i].name);
   else if(i > (this.allStocks.length - 1 - longShortAmount)) this.long.push(this.allStocks[i].name);
   else continue;
 }
 // Determine amount to long/short based on total stock price of each bucket.
 var equity;
 await this.alpaca.getAccount().then((resp) => {
   equity = resp.equity;
 }).catch((err) => {console.log(err.error);});
 this.shortAmount = 0.30 * equity;
 this.longAmount = Number(this.shortAmount) + Number(equity);

 var promLong = await this.getTotalPrice(this.long);
 var promShort = await this.getTotalPrice(this.short);
 var longTotal;
 var shortTotal;
 await Promise.all(promLong).then((resp) => {
   longTotal = resp.reduce((a, b) => a + b, 0);
 });
 await Promise.all(promShort).then((resp) => {
   shortTotal = resp.reduce((a, b) => a + b, 0);
 });

 this.qLong = Math.floor(this.longAmount / longTotal);
 this.qShort = Math.floor(this.shortAmount / shortTotal);
}

// Mechanism used to rank the stocks, the basis of the Long-Short Equity Strategy.
async rank(){
 // Ranks all stocks by percent change over the past 10 minutes (higher is better).
 var promStocks = this.getPercentChanges(this.allStocks);
 await Promise.all(promStocks);

 // Sort the stocks in place by the percent change field (marked by pc).
 this.allStocks.sort((a, b) => {return a.pc - b.pc;});
}

// Get the total price of the array of input stocks.
getTotalPrice(stocks){
 var proms = [];
 stocks.forEach(async (stock) => {
   proms.push(new Promise(async (resolve, reject) => {
     await this.alpaca.getBars('minute', stock, {limit: 1}).then((resp) => {
       resolve(resp[stock][0].c);
     }).catch((err) => {console.log(err.error);});
   }));
 });
 return proms;
}

// Submit a batch order that returns completed and uncompleted orders.
async sendBatchOrder(quantity, stocks, side){
 var prom = new Promise(async (resolve, reject) => {
   var incomplete = [];
   var executed = [];
   var promOrders = [];
   stocks.forEach(async (stock) => {
     promOrders.push(new Promise(async (resolve, reject) => {
       if(!this.blacklist.has(stock)) {
         var promSO = this.submitOrder(quantity, stock, side);
         await promSO.then((resp) => {
           if(resp) executed.push(stock);
           else incomplete.push(stock);
           resolve();
         });
       }
       else resolve();
     }));
   });
   await Promise.all(promOrders).then(() => {
     resolve([incomplete, executed]);
   });
 });
 return prom;
}

Here’s a link to the entire script on the GitHub. 8

What You Can Do

Within the classification of a Long-Short equity strategy, there are many variables that you can control to fine-tune your ranking system. I mentioned earlier that you can change the universe of stocks by altering the this.allStocks array at the top of the script. The amount of stocks you will long and short is another customization you can make. The script takes the top and bottom 25% to long and short, but you can change this to whatever you prefer. Just edit the “4” in the snippet below (4 representing ¼) and you can change the script to select a larger or smaller amount of stocks for each bucket.

var longShortAmount = Math.floor(this.allStocks.length / 4);

Another variable you can control is the percentage of equity to use for longing and shorting. This algorithm uses a 130/30 split which is popular amongst hedge funds. What this means is 130% of equity is used for taking long positions, and 30% of equity is used for short positions. You can edit that in the snippet below, choosing either percentage of equity or hard-coding amounts for your strategy.

// Determine amount to long/short based on total stock price of each bucket.
var equity;
await this.alpaca.getAccount().then((resp) => {
 equity = resp.equity;
}).catch((err) => {console.log(err.error);});
this.shortAmount = 0.30 * equity;
this.longAmount = Number(this.shortAmount) + Number(equity);

You can also change the frequency of portfolio rebalancing. At the moment, the script rebalances after every 1 minute. This is represented by the “60000” in the code below, representing 60000 milliseconds. You can change this to be more or less frequent (for example, if you only wanted rebalancing every 5 minutes, you would change 60000 to 5 * 60000 = 300000). However, be sure that you are not making your frequency too high or low. If you set it to be too frequent, you might reach the 200 requests per minute quota and the script would start throwing errors. If you set it to be too infrequent, the script may not be able to clear all positions in time for market close (you can adjust for this by increasing the 15 minute tolerance to market close, located in the run() function).

async run(){
 ...
   else {
     // Rebalance the portfolio.
     await this.rebalance();
   }
 }, 60000);
}

The final variable that you can easily manipulate is the ranking system. In this script, the stocks are ranked in terms of percent change in stock price over the past 10 minutes. This means that the algorithm is betting stocks that are doing well will continue to do well, and vice versa. You can change the ranking system by changing the function rank(). In this function, you can rank the stocks in whatever manner you please, just make sure you input the rank in the field “pc” for each stock in the allStocks array, where higher numbers is better (0 is the worst stock, 5 is a better stock). This way, the function rerank() will order the allStocks array least to greatest by the “pc” field, which then allows the script to split up the stocks into buckets.

// Get percent changes of the stock prices over the past 10 minutes.
getPercentChanges(allStocks){
 var length = 10;
 var promStocks = [];
 allStocks.forEach((stock) => {
   promStocks.push(new Promise(async (resolve, reject) => {
     await this.alpaca.getBars('minute', stock.name, {limit: length}).then((resp) => {
       stock.pc = (resp[stock.name][length - 1].c - resp[stock.name][0].o) / resp[stock.name][0].o;
     }).catch((err) => {console.log(err.error);});
     resolve();
   }));
 });
 return promStocks;
}

// Mechanism used to rank the stocks, the basis of the Long-Short Equity Strategy.
async rank(){
 // Ranks all stocks by percent change over the past 10 minutes (higher is better).
 var promStocks = this.getPercentChanges(this.allStocks);
 await Promise.all(promStocks);

 // Sort the stocks in place by the percent change field (marked by pc).
 this.allStocks.sort((a, b) => {return a.pc - b.pc;});
}

There are other customizations you can make to this script (for example, buying more shares of the highest ranked stock in the long bucket and buying less shares of the lowest ranked stock in the long bucket, and then doing the same for the short bucket), but those changes require more intricate code changes than simply changing one number/area of code.

Thanks for reading

If you liked this post, please do share/like it with all of your programming buddies!

Follow us on Facebook | Twitter

Further reading about JavaScript

The Complete JavaScript Course 2019: Build Real Projects!

Vue JS 2 - The Complete Guide (incl. Vue Router & Vuex)

JavaScript Bootcamp - Build Real World Applications

The Web Developer Bootcamp

JavaScript Programming Tutorial - Full JavaScript Course for Beginners

New ES2019 Features Every JavaScript Developer Should Know

Best JavaScript Frameworks, Libraries and Tools to Use in 2019

React vs Angular vs Vue.js by Example

Microfrontends — Connecting JavaScript frameworks together (React, Angular, Vue etc)

Creating Web Animations with Anime.js

Ember.js vs Vue.js - Which is JavaScript Framework Works Better for You

Do we still need JavaScript frameworks?


Top 7 Most Popular Node.js Frameworks You Should Know

Top 7 Most Popular Node.js Frameworks You Should Know

Node.js is an open-source, cross-platform, runtime environment that allows developers to run JavaScript outside of a browser. In this post, you'll see top 7 of the most popular Node frameworks at this point in time (ranked from high to low by GitHub stars).

Node.js is an open-source, cross-platform, runtime environment that allows developers to run JavaScript outside of a browser.

One of the main advantages of Node is that it enables developers to use JavaScript on both the front-end and the back-end of an application. This not only makes the source code of any app cleaner and more consistent, but it significantly speeds up app development too, as developers only need to use one language.

Node is fast, scalable, and easy to get started with. Its default package manager is npm, which means it also sports the largest ecosystem of open-source libraries. Node is used by companies such as NASA, Uber, Netflix, and Walmart.

But Node doesn't come alone. It comes with a plethora of frameworks. A Node framework can be pictured as the external scaffolding that you can build your app in. These frameworks are built on top of Node and extend the technology's functionality, mostly by making apps easier to prototype and develop, while also making them faster and more scalable.

Below are 7of the most popular Node frameworks at this point in time (ranked from high to low by GitHub stars).

Express

With over 43,000 GitHub stars, Express is the most popular Node framework. It brands itself as a fast, unopinionated, and minimalist framework. Express acts as middleware: it helps set up and configure routes to send and receive requests between the front-end and the database of an app.

Express provides lightweight, powerful tools for HTTP servers. It's a great framework for single-page apps, websites, hybrids, or public HTTP APIs. It supports over fourteen different template engines, so developers aren't forced into any specific ORM.

Meteor

Meteor is a full-stack JavaScript platform. It allows developers to build real-time web apps, i.e. apps where code changes are pushed to all browsers and devices in real-time. Additionally, servers send data over the wire, instead of HTML. The client renders the data.

The project has over 41,000 GitHub stars and is built to power large projects. Meteor is used by companies such as Mazda, Honeywell, Qualcomm, and IKEA. It has excellent documentation and a strong community behind it.

Koa

Koa is built by the same team that built Express. It uses ES6 methods that allow developers to work without callbacks. Developers also have more control over error-handling. Koa has no middleware within its core, which means that developers have more control over configuration, but which means that traditional Node middleware (e.g. req, res, next) won't work with Koa.

Koa already has over 26,000 GitHub stars. The Express developers built Koa because they wanted a lighter framework that was more expressive and more robust than Express. You can find out more about the differences between Koa and Express here.

Sails

Sails is a real-time, MVC framework for Node that's built on Express. It supports auto-generated REST APIs and comes with an easy WebSocket integration.

The project has over 20,000 stars on GitHub and is compatible with almost all databases (MySQL, MongoDB, PostgreSQL, Redis). It's also compatible with most front-end technologies (Angular, iOS, Android, React, and even Windows Phone).

Nest

Nest has over 15,000 GitHub stars. It uses progressive JavaScript and is built with TypeScript, which means it comes with strong typing. It combines elements of object-oriented programming, functional programming, and functional reactive programming.

Nest is packaged in such a way it serves as a complete development kit for writing enterprise-level apps. The framework uses Express, but is compatible with a wide range of other libraries.

LoopBack

LoopBack is a framework that allows developers to quickly create REST APIs. It has an easy-to-use CLI wizard and allows developers to create models either on their schema or dynamically. It also has a built-in API explorer.

LoopBack has over 12,000 GitHub stars and is used by companies such as GoDaddy, Symantec, and the Bank of America. It's compatible with many REST services and a wide variety of databases (MongoDB, Oracle, MySQL, PostgreSQL).

Hapi

Similar to Express, hapi serves data by intermediating between server-side and client-side. As such, it's can serve as a substitute for Express. Hapi allows developers to focus on writing reusable app logic in a modular and prescriptive fashion.

The project has over 11,000 GitHub stars. It has built-in support for input validation, caching, authentication, and more. Hapi was originally developed to handle all of Walmart's mobile traffic during Black Friday.

Node.js for Beginners - Learn Node.js from Scratch (Step by Step)

Node.js for Beginners - Learn Node.js from Scratch (Step by Step)

Node.js for Beginners - Learn Node.js from Scratch (Step by Step) - Learn the basics of Node.js. This Node.js tutorial will guide you step by step so that you will learn basics and theory of every part. Learn to use Node.js like a professional. You’ll learn: Basic Of Node, Modules, NPM In Node, Event, Email, Uploading File, Advance Of Node.

Node.js for Beginners

Learn Node.js from Scratch (Step by Step)

Welcome to my course "Node.js for Beginners - Learn Node.js from Scratch". This course will guide you step by step so that you will learn basics and theory of every part. This course contain hands on example so that you can understand coding in Node.js better. If you have no previous knowledge or experience in Node.js, you will like that the course begins with Node.js basics. otherwise if you have few experience in programming in Node.js, this course can help you learn some new information . This course contain hands on practical examples without neglecting theory and basics. Learn to use Node.js like a professional. This comprehensive course will allow to work on the real world as an expert!
What you’ll learn:

  • Basic Of Node
  • Modules
  • NPM In Node
  • Event
  • Email
  • Uploading File
  • Advance Of Node

How to Use Express.js, Node.js and MongoDB.js

How to Use Express.js, Node.js and MongoDB.js

In this post, I will show you how to use Express.js, Node.js and MongoDB.js. We will be creating a very simple Node application, that will allow users to input data that they want to store in a MongoDB database. It will also show all items that have been entered into the database.

In this post, I will show you how to use Express.js, Node.js and MongoDB.js. We will be creating a very simple Node application, that will allow users to input data that they want to store in a MongoDB database. It will also show all items that have been entered into the database.

Creating a Node Application

To get started I would recommend creating a new database that will contain our application. For this demo I am creating a directory called node-demo. After creating the directory you will need to change into that directory.

mkdir node-demo
cd node-demo

Once we are in the directory we will need to create an application and we can do this by running the command
npm init

This will ask you a series of questions. Here are the answers I gave to the prompts.

The first step is to create a file that will contain our code for our Node.js server.

touch app.js

In our app.js we are going to add the following code to build a very simple Node.js Application.

var express = require("express");
var app = express();
var port = 3000;
 
app.get("/", (req, res) => {
&nbsp;&nbsp;res.send("Hello World");
});
 
app.listen(port, () => {
  console.log("Server listening on port " + port);
});

What the code does is require the express.js application. It then creates app by calling express. We define our port to be 3000.

The app.use line will listen to requests from the browser and will return the text “Hello World” back to the browser.

The last line actually starts the server and tells it to listen on port 3000.

Installing Express

Our app.js required the Express.js module. We need to install express in order for this to work properly. Go to your terminal and enter this command.

npm install express --save

This command will install the express module into our package.json. The module is installed as a dependency in our package.json as shown below.

To test our application you can go to the terminal and enter the command

node app.js

Open up a browser and navigate to the url http://localhost:3000

You will see the following in your browser

Creating Website to Save Data to MongoDB Database

Instead of showing the text “Hello World” when people view your application, what we want to do is to show a place for user to save data to the database.

We are going to allow users to enter a first name and a last name that we will be saving in the database.

To do this we will need to create a basic HTML file. In your terminal enter the following command to create an index.html file.

touch index.html

In our index.html file we will be creating an input filed where users can input data that they want to have stored in the database. We will also need a button for users to click on that will add the data to the database.

Here is what our index.html file looks like.

<!DOCTYPE html>
<html>
  <head>
    <title>Intro to Node and MongoDB<title>
  <head>

  <body>
    <h1>Into to Node and MongoDB<&#47;h1>
    <form method="post" action="/addname">
      <label>Enter Your Name<&#47;label><br>
      <input type="text" name="firstName" placeholder="Enter first name..." required>
      <input type="text" name="lastName" placeholder="Enter last name..." required>
      <input type="submit" value="Add Name">
    </form>
  <body>
<html>

If you are familiar with HTML, you will not find anything unusual in our code for our index.html file. We are creating a form where users can input their first name and last name and then click an “Add Name” button.

The form will do a post call to the /addname endpoint. We will be talking about endpoints and post later in this tutorial.

Displaying our Website to Users

We were previously displaying the text “Hello World” to users when they visited our website. Now we want to display our html file that we created. To do this we will need to change the app.use line our our app.js file.

We will be using the sendFile command to show the index.html file. We will need to tell the server exactly where to find the index.html file. We can do that by using a node global call __dirname. The __dirname will provide the current directly where the command was run. We will then append the path to our index.html file.

The app.use lines will need to be changed to
app.use("/", (req, res) => {   res.sendFile(__dirname + "/index.html"); });

Once you have saved your app.js file, we can test it by going to terminal and running node app.js

Open your browser and navigate to “http://localhost:3000”. You will see the following

Connecting to the Database

Now we need to add our database to the application. We will be connecting to a MongoDB database. I am assuming that you already have MongoDB installed and running on your computer.

To connect to the MongoDB database we are going to use a module called Mongoose. We will need to install mongoose module just like we did with express. Go to your terminal and enter the following command.
npm install mongoose --save

This will install the mongoose model and add it as a dependency in our package.json.

Connecting to the Database

Now that we have the mongoose module installed, we need to connect to the database in our app.js file. MongoDB, by default, runs on port 27017. You connect to the database by telling it the location of the database and the name of the database.

In our app.js file after the line for the port and before the app.use line, enter the following two lines to get access to mongoose and to connect to the database. For the database, I am going to use “node-demo”.

var mongoose = require("mongoose"); mongoose.Promise = global.Promise; mongoose.connect("mongodb://localhost:27017/node-demo");

Creating a Database Schema

Once the user enters data in the input field and clicks the add button, we want the contents of the input field to be stored in the database. In order to know the format of the data in the database, we need to have a Schema.

For this tutorial, we will need a very simple Schema that has only two fields. I am going to call the field firstName and lastName. The data stored in both fields will be a String.

After connecting to the database in our app.js we need to define our Schema. Here are the lines you need to add to the app.js.
var nameSchema = new mongoose.Schema({   firstName: String,   lastNameName: String });

Once we have built our Schema, we need to create a model from it. I am going to call my model “DataInput”. Here is the line you will add next to create our mode.
var User = mongoose.model("User", nameSchema);

Creating RESTful API

Now that we have a connection to our database, we need to create the mechanism by which data will be added to the database. This is done through our REST API. We will need to create an endpoint that will be used to send data to our server. Once the server receives this data then it will store the data in the database.

An endpoint is a route that our server will be listening to to get data from the browser. We already have one route that we have created already in the application and that is the route that is listening at the endpoint “/” which is the homepage of our application.

HTTP Verbs in a REST API

The communication between the client(the browser) and the server is done through an HTTP verb. The most common HTTP verbs are
GET, PUT, POST, and DELETE.

The following table explains what each HTTP verb does.

HTTP Verb Operation
GET Read
POST Create
PUT Update
DELETE Delete

As you can see from these verbs, they form the basis of CRUD operations that I talked about previously.

Building a CRUD endpoint

If you remember, the form in our index.html file used a post method to call this endpoint. We will now create this endpoint.

In our previous endpoint we used a “GET” http verb to display the index.html file. We are going to do something very similar but instead of using “GET”, we are going to use “POST”. To get started this is what the framework of our endpoint will look like.

app.post("/addname", (req, res) => {
 
});
Express Middleware

To fill out the contents of our endpoint, we want to store the firstName and lastName entered by the user into the database. The values for firstName and lastName are in the body of the request that we send to the server. We want to capture that data, convert it to JSON and store it into the database.

Express.js version 4 removed all middleware. To parse the data in the body we will need to add middleware into our application to provide this functionality. We will be using the body-parser module. We need to install it, so in your terminal window enter the following command.

npm install body-parser --save

Once it is installed, we will need to require this module and configure it. The configuration will allow us to pass the data for firstName and lastName in the body to the server. It can also convert that data into JSON format. This will be handy because we can take this formatted data and save it directly into our database.

To add the body-parser middleware to our application and configure it, we can add the following lines directly after the line that sets our port.

var bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
Saving data to database

Mongoose provides a save function that will take a JSON object and store it in the database. Our body-parser middleware, will convert the user’s input into the JSON format for us.

To save the data into the database, we need to create a new instance of our model that we created early. We will pass into this instance the user’s input. Once we have it then we just need to enter the command “save”.

Mongoose will return a promise on a save to the database. A promise is what is returned when the save to the database completes. This save will either finish successfully or it will fail. A promise provides two methods that will handle both of these scenarios.

If this save to the database was successful it will return to the .then segment of the promise. In this case we want to send text back the user to let them know the data was saved to the database.

If it fails it will return to the .catch segment of the promise. In this case, we want to send text back to the user telling them the data was not saved to the database. It is best practice to also change the statusCode that is returned from the default 200 to a 400. A 400 statusCode signifies that the operation failed.

Now putting all of this together here is what our final endpoint will look like.

app.post("/addname", (req, res) => {
  var myData = new User(req.body);
  myData.save()
    .then(item => {
      res.send("item saved to database");
    })
    .catch(err => {
      res.status(400).send("unable to save to database");
    });
});
Testing our code

Save your code. Go to your terminal and enter the command node app.js to start our server. Open up your browser and navigate to the URL “http://localhost:3000”. You will see our index.html file displayed to you.

Make sure you have mongo running.

Enter your first name and last name in the input fields and then click the “Add Name” button. You should get back text that says the name has been saved to the database like below.

Access to Code

The final version of the code is available in my Github repo. To access the code click here. Thank you for reading !