Which Query Builder/ORM should you use for Nodejs

Which Query Builder/ORM should you use for Nodejs

<strong>What is an ORM ? ORM solutions are useful to facilitate data-driven API development. Users have concrete needs which drive the data model of an application. In legacy development, this data architecture is typically implemented and version controlled using database scripts such as SQL scripts. A separate library is then used for the server application to execute CRUD actions on the database.</strong>

What is an ORM ? ORM solutions are useful to facilitate data-driven API development. Users have concrete needs which drive the data model of an application. In legacy development, this data architecture is typically implemented and version controlled using database scripts such as SQL scripts. A separate library is then used for the server application to execute CRUD actions on the database.

Why is it so Important?

ORMs work as a high-level API to execute CRUD, and these days quality ORMs also allow us to initialize the data through code. Complex data manipulation, cleaning and so on, is often easier in code. While dedicated Extract, Transform and Load (ETL) tools exist, the same ETL tasks can be easily implemented in ORM.

Implementing extract, transform, and load with code allows a system to more easily integrate data from very different sources. SQL databases of multiple flavors, NoSQL data, file system data, and third party data can all be integrated under a single language with a JavaScript ORM.

Finally, code-oriented data control also allows a system to implement data usage at run time or in the build process, and flexibly adapt usage during the development process as needed.

To restate, ORMs help to abstract data mappings between your code and the database, easing data querying and manipulation. It can also help to easily change the underlying database engine without (mostly) changing any code. The following are a few ORMs you can use with nodejs.


Should I use ORM like sequelize for PostgreSQL/Node.js?

To begin with it depends on the scale of your project and team working on that project. If you have a small scaled project which may need less customizations later on you can simply go for query based execution. But if you have a large project with customizations coming at a later stage try to develop yourself or your team to use an ORM like sequelize.

So the overall process needed to move to ORM is:

1. You should have great understanding of the queries and relations of the database schema.

2. You should understand the language your ORM supports. So for Node.js one should have a skilled understanding of basic Javascript, Callbacks and Promises.

3. Now if you know the queries well and can work with ORM too. You will speed your development process manyfolds.

Pros:

  • Standardization – ORMs usually have a single schema definition in the code. This makes it very clear what the schema is, and very simple to change it.
  • No need to learn SQL – queries are written in plain JavaScript.
  • Portable – with an ORM, it is easy to migrate between databases (we actually did that already, moving some older PostgreSQL databases to Amazon Aurora with the MySQL standardization). Because all your code uses the ORM, you only need to change / reimplement the ORM to replace the DB. Many ORMs (like Sequalize) support multiple popular databases out of the box.

Cons:

  • **It is hard to implement complex queries **– while ORMs simplify querying – especially if you lack experience with SQL – they can prove more difficult and non-flexible when writing complex queries (with aggregations, sub-queries, joins, etc…). (I know it’s supported, but it often feels very roundabout compared to SQL).
  • Can be inefficient – due to the previous point, ORMs can lead to inefficient querying (you’re writing a query that fits the ORM, not a simple / efficient query). Because you’re not using SQL, the query execution also becomes obscure (can’t run SQL explain and immediately understand what the query plan is).
  • New language – if you’re familiar with SQL, an ORM can feel like re-learning a new query language.

_Bottom line _– for smaller projects / projects that involve very complex queries, I’d recommend sticking with SQL. For bigger projects with bigger teams touching the codebase and making schema changes, a good ORM can be a life saver.

Following are some popular ORM :

1. Sequelize

Sequelize is another ORM for Node.js and io.js (which are finally merging together). It supports PostgreSQL, MySQL, MariaDB, SQLite and MSSQL and features solid transaction support, relations, read replication, and

more. You can install it by running the following commands:

# Install Sequelize
$ npm install --save sequelize
Install the DB driver

$ npm install --save pg pg-hstore

For both mysql and mariadb dialects

$ npm install --save mysql
$ npm install --save sqlite3

MSSQL

$ npm install --save tedious

Now you are ready to use it as shown in the example below:

var Sequelize = require('sequelize');
var sequelize = new Sequelize('database', 'username', 'password', {
host: 'localhost',
dialect: 'mysql'|'mariadb'|'sqlite'|'postgres'|'mssql', // use one of these

pool: {
max: 5,
min: 0,
idle: 10000
},

// SQLite only
storage: 'path/to/database.sqlite'
});

// Or you can simply use a connection uri
var sequelize = new Sequelize('postgres://user:[email protected]:5432/dbname');

Just like with Bookshelf.js, you need only one connection to the database. After that, you can create a model like:

var User = sequelize.define('user', {
firstName: {
type: Sequelize.STRING,
field: 'first_name' // first_name column matches User.firstName
},
lastName: {
type: Sequelize.STRING
}
}, {
freezeTableName: true // Model tableName (user) will be the same as the model name
});

The above Sequelize.STRING matches a VARCHAR in SQL. Other data types are Sequelize.INTEGER for INTEGERSequelize.BLOB for BLOB (or bytea in Postgres). You can read the full list here.

Sequelize allows you to write relations between the tables. For example, if you have a model called Project and another one called Developer and want to assign more than one developer to one project, you can do it like this:

Project.hasMany(Developer, {as: 'devs'})

This will make sure to add the necessary fields in each model (project_id to the Developer model in this case). Or if you feel you can’t profit from the Sequelize API, you can run raw SQL queries.

Sequelize can be found on GitHub as well

2. Bookshelf

Bookshelf is a JavaScript ORM for Node.js, built on the Knex SQL query builder. Featuring both promise based and traditional callback interfaces, providing transaction support, eager/nested-eager relation loading, polymorphic associations, and support for one-to-one, one-to-many, and many-to-many relations. It is designed to work well with PostgreSQL, MySQL, and SQLite3.

While Bookshelf primarily targets Node.js, all dependencies are browser compatible, and it could be adapted to work with other javascript environments supporting a sqlite3 database, by providing a custom Knex adapter.

|

var knex = require('knex')({
client: 'mysql',
connection: {
host : '127.0.0.1',
user : 'your_database_user',
password : 'your_database_password',
database : 'myapp_test',
charset : 'utf8'
}
});

var bookshelf = require('bookshelf')(knex);

var User = bookshelf.Model.extend({
tableName: 'users'
});

|

3. Objection.js

Objection.js is an ORM for Node.js that aims to stay out of your way and make it as easy as possible to use the full power of SQL and the underlying database engine.

Objection.js, like Bookshelf, is built on the wonderful SQL query builder knex. All databases supported by knex are supported by objection.js. SQLite3, Postgres and MySQL are thoroughly tested.

4. Lovefiled

Lovefield is not a real ORM. It’s actually a relational database for web apps, built upon IndexedDB, developed by Google and written entirely in JavaScript. It doesn’t support raw SQL queries, but it comes with an API that tries to mimic the SQL syntax.

Besides downloading directly from GitHub repository, Lovefield supports npm and bower package management systems and can be found using

npm info lovefield
bower info lovefield

Adding Lovefield as the dependency and executing npm update or bower update will automatically pull down the designated release.

Defining Schema

The concept of Lovefield is to define a database schema, then operate on the instance implementing that schema. In the example, schema definition is carried out through a set of synchronous APIs:

// SQL equivalent: CREATE DATABASE IF NOT EXISTS todo
// This schema definition (or data definition commands in SQL, DDL) is not
// executed immediately. Lovefield uses builder pattern to build the schema
// first, then performs necessary database open/creation later.
var schemaBuilder = lf.schema.create('todo', 1);

// SQL equivalent:
// CREATE TABLE IF NOT EXISTS Item (
// id AS INTEGER,
// description AS INTEGER,
// deadline as DATE_TIME,
// done as BOOLEAN,
// PRIMARY KEY ON ('id')
// );
// ALTER TABLE Item ADD INDEX idxDeadLine(Item.deadline DESC);
schemaBuilder.createTable('Item').
addColumn('id', lf.Type.INTEGER).
addColumn('description', lf.Type.STRING).
addColumn('deadline', lf.Type.DATE_TIME).
addColumn('done', lf.Type.BOOLEAN).
addPrimaryKey(['id']).
addIndex('idxDeadline', ['deadline'], false, lf.Order.DESC);

The code above has pseudo SQL commands to demonstrate their equivalent concept in SQL. Once the schema is defined, Lovefield needs to be instructed to create or connect to the corresponding instance:

// Promise-based API to get the instance.
schemaBuilder.connect().then(function(db) {
// ...
});

From this point on, the schema cannot be altered. Both the connect() and Lovefield offered query APIs are asynchronous Promise-based APIs. This design is to prevent Lovefield from blocking main thread since the queries can be long running and demanding quite some CPU and I/O cycles.

If the database is brand new, Lovefield will create it using the schema. If the database already exists, Lovefield will attempt to identify the instance using database name specified in the schema, and connect to it.

Lovefield also uses Promise chaining pattern extensively:

// Start of the Promise chaining
schemaBuilder.connect().then(function(db) {
// Asynchronous call connect() returned object: db
todoDb = db;

// Get the schema representation of table Item.
// All schema-related APIs are synchronous.
item = db.getSchema().table('Item');

// Creates a row. Lovefield does not accept plain objects as row.
// Use the createRow() API provided in table schema to create a row.
var row = item.createRow({
'id': 1,
'description': 'Get a cup of coffee',
'deadline': new Date(),
'done': false
});

// INSERT OR REPLACE INTO Item VALUES row;
// The exec() method returns a Promise.
return db.insertOrReplace().into(item).values([row]).exec();

}).then(function() {
// When reached here, Lovefield guarantees previous INSERT OR REPLACE
// has been committed with its implicit transaction.

// SELECT * FROM Item WHERE Item.done = false;
// Return another Promise by calling this SELECT query's exec() method.
return todoDb.select().from(item).where(item.done.eq(false)).exec();

}).then(function(results) {
// The SELECT query's Promise will return array of rows selected.
// If there were no rows, the array will be empty.

results.forEach(function(row) {
// Use column name to directly dereference the columns from a row.
console.log(row['description'], 'before', row['deadline']);
});
});

5. waterline

Waterline is a new kind of storage and retrieval engine.

It provides a uniform API for accessing stuff from different kinds of databases, protocols, and 3rd party APIs. That means you write the same code to get and store things like users, whether they live in Redis, mySQL, LDAP, MongoDB, or Postgres.

Waterline strives to inherit the best parts of ORMs like ActiveRecord, Hibernate, and Mongoose, but with a fresh perspective and emphasis on modularity, testability, and consistency across adapters.

Originally published at socialdribbler.com

Learn more

☞ MERN Stack Front To Back: Full Stack React, Redux & Node.js

☞ The Complete Node.js Developer Course (3rd Edition)

☞ Learn and Understand NodeJS

☞ NodeJS - The Complete Guide (incl. MVC, REST APIs, GraphQL)

☞ React Node FullStack - Social Network from Scratch to Deploy

☞ Node.js Absolute Beginners Guide - Learn Node From Scratch

How to Use Express.js, Node.js and MongoDB.js

How to Use Express.js, Node.js and MongoDB.js

In this post, I will show you how to use Express.js, Node.js and MongoDB.js. We will be creating a very simple Node application, that will allow users to input data that they want to store in a MongoDB database. It will also show all items that have been entered into the database.

In this post, I will show you how to use Express.js, Node.js and MongoDB.js. We will be creating a very simple Node application, that will allow users to input data that they want to store in a MongoDB database. It will also show all items that have been entered into the database.

Creating a Node Application

To get started I would recommend creating a new database that will contain our application. For this demo I am creating a directory called node-demo. After creating the directory you will need to change into that directory.

mkdir node-demo
cd node-demo

Once we are in the directory we will need to create an application and we can do this by running the command
npm init

This will ask you a series of questions. Here are the answers I gave to the prompts.

The first step is to create a file that will contain our code for our Node.js server.

touch app.js

In our app.js we are going to add the following code to build a very simple Node.js Application.

var express = require("express");
var app = express();
var port = 3000;
 
app.get("/", (req, res) => {
&nbsp;&nbsp;res.send("Hello World");
});
 
app.listen(port, () => {
  console.log("Server listening on port " + port);
});

What the code does is require the express.js application. It then creates app by calling express. We define our port to be 3000.

The app.use line will listen to requests from the browser and will return the text “Hello World” back to the browser.

The last line actually starts the server and tells it to listen on port 3000.

Installing Express

Our app.js required the Express.js module. We need to install express in order for this to work properly. Go to your terminal and enter this command.

npm install express --save

This command will install the express module into our package.json. The module is installed as a dependency in our package.json as shown below.

To test our application you can go to the terminal and enter the command

node app.js

Open up a browser and navigate to the url http://localhost:3000

You will see the following in your browser

Creating Website to Save Data to MongoDB Database

Instead of showing the text “Hello World” when people view your application, what we want to do is to show a place for user to save data to the database.

We are going to allow users to enter a first name and a last name that we will be saving in the database.

To do this we will need to create a basic HTML file. In your terminal enter the following command to create an index.html file.

touch index.html

In our index.html file we will be creating an input filed where users can input data that they want to have stored in the database. We will also need a button for users to click on that will add the data to the database.

Here is what our index.html file looks like.

<!DOCTYPE html>
<html>
  <head>
    <title>Intro to Node and MongoDB<title>
  <head>

  <body>
    <h1>Into to Node and MongoDB<&#47;h1>
    <form method="post" action="/addname">
      <label>Enter Your Name<&#47;label><br>
      <input type="text" name="firstName" placeholder="Enter first name..." required>
      <input type="text" name="lastName" placeholder="Enter last name..." required>
      <input type="submit" value="Add Name">
    </form>
  <body>
<html>

If you are familiar with HTML, you will not find anything unusual in our code for our index.html file. We are creating a form where users can input their first name and last name and then click an “Add Name” button.

The form will do a post call to the /addname endpoint. We will be talking about endpoints and post later in this tutorial.

Displaying our Website to Users

We were previously displaying the text “Hello World” to users when they visited our website. Now we want to display our html file that we created. To do this we will need to change the app.use line our our app.js file.

We will be using the sendFile command to show the index.html file. We will need to tell the server exactly where to find the index.html file. We can do that by using a node global call __dirname. The __dirname will provide the current directly where the command was run. We will then append the path to our index.html file.

The app.use lines will need to be changed to
app.use("/", (req, res) => {   res.sendFile(__dirname + "/index.html"); });

Once you have saved your app.js file, we can test it by going to terminal and running node app.js

Open your browser and navigate to “http://localhost:3000”. You will see the following

Connecting to the Database

Now we need to add our database to the application. We will be connecting to a MongoDB database. I am assuming that you already have MongoDB installed and running on your computer.

To connect to the MongoDB database we are going to use a module called Mongoose. We will need to install mongoose module just like we did with express. Go to your terminal and enter the following command.
npm install mongoose --save

This will install the mongoose model and add it as a dependency in our package.json.

Connecting to the Database

Now that we have the mongoose module installed, we need to connect to the database in our app.js file. MongoDB, by default, runs on port 27017. You connect to the database by telling it the location of the database and the name of the database.

In our app.js file after the line for the port and before the app.use line, enter the following two lines to get access to mongoose and to connect to the database. For the database, I am going to use “node-demo”.

var mongoose = require("mongoose"); mongoose.Promise = global.Promise; mongoose.connect("mongodb://localhost:27017/node-demo");

Creating a Database Schema

Once the user enters data in the input field and clicks the add button, we want the contents of the input field to be stored in the database. In order to know the format of the data in the database, we need to have a Schema.

For this tutorial, we will need a very simple Schema that has only two fields. I am going to call the field firstName and lastName. The data stored in both fields will be a String.

After connecting to the database in our app.js we need to define our Schema. Here are the lines you need to add to the app.js.
var nameSchema = new mongoose.Schema({   firstName: String,   lastNameName: String });

Once we have built our Schema, we need to create a model from it. I am going to call my model “DataInput”. Here is the line you will add next to create our mode.
var User = mongoose.model("User", nameSchema);

Creating RESTful API

Now that we have a connection to our database, we need to create the mechanism by which data will be added to the database. This is done through our REST API. We will need to create an endpoint that will be used to send data to our server. Once the server receives this data then it will store the data in the database.

An endpoint is a route that our server will be listening to to get data from the browser. We already have one route that we have created already in the application and that is the route that is listening at the endpoint “/” which is the homepage of our application.

HTTP Verbs in a REST API

The communication between the client(the browser) and the server is done through an HTTP verb. The most common HTTP verbs are
GET, PUT, POST, and DELETE.

The following table explains what each HTTP verb does.

HTTP Verb Operation
GET Read
POST Create
PUT Update
DELETE Delete

As you can see from these verbs, they form the basis of CRUD operations that I talked about previously.

Building a CRUD endpoint

If you remember, the form in our index.html file used a post method to call this endpoint. We will now create this endpoint.

In our previous endpoint we used a “GET” http verb to display the index.html file. We are going to do something very similar but instead of using “GET”, we are going to use “POST”. To get started this is what the framework of our endpoint will look like.

app.post("/addname", (req, res) => {
 
});
Express Middleware

To fill out the contents of our endpoint, we want to store the firstName and lastName entered by the user into the database. The values for firstName and lastName are in the body of the request that we send to the server. We want to capture that data, convert it to JSON and store it into the database.

Express.js version 4 removed all middleware. To parse the data in the body we will need to add middleware into our application to provide this functionality. We will be using the body-parser module. We need to install it, so in your terminal window enter the following command.

npm install body-parser --save

Once it is installed, we will need to require this module and configure it. The configuration will allow us to pass the data for firstName and lastName in the body to the server. It can also convert that data into JSON format. This will be handy because we can take this formatted data and save it directly into our database.

To add the body-parser middleware to our application and configure it, we can add the following lines directly after the line that sets our port.

var bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
Saving data to database

Mongoose provides a save function that will take a JSON object and store it in the database. Our body-parser middleware, will convert the user’s input into the JSON format for us.

To save the data into the database, we need to create a new instance of our model that we created early. We will pass into this instance the user’s input. Once we have it then we just need to enter the command “save”.

Mongoose will return a promise on a save to the database. A promise is what is returned when the save to the database completes. This save will either finish successfully or it will fail. A promise provides two methods that will handle both of these scenarios.

If this save to the database was successful it will return to the .then segment of the promise. In this case we want to send text back the user to let them know the data was saved to the database.

If it fails it will return to the .catch segment of the promise. In this case, we want to send text back to the user telling them the data was not saved to the database. It is best practice to also change the statusCode that is returned from the default 200 to a 400. A 400 statusCode signifies that the operation failed.

Now putting all of this together here is what our final endpoint will look like.

app.post("/addname", (req, res) => {
  var myData = new User(req.body);
  myData.save()
    .then(item => {
      res.send("item saved to database");
    })
    .catch(err => {
      res.status(400).send("unable to save to database");
    });
});
Testing our code

Save your code. Go to your terminal and enter the command node app.js to start our server. Open up your browser and navigate to the URL “http://localhost:3000”. You will see our index.html file displayed to you.

Make sure you have mongo running.

Enter your first name and last name in the input fields and then click the “Add Name” button. You should get back text that says the name has been saved to the database like below.

Access to Code

The final version of the code is available in my Github repo. To access the code click here. Thank you for reading !

Node.js for Beginners - Learn Node.js from Scratch (Step by Step)

Node.js for Beginners - Learn Node.js from Scratch (Step by Step)

Node.js for Beginners - Learn Node.js from Scratch (Step by Step) - Learn the basics of Node.js. This Node.js tutorial will guide you step by step so that you will learn basics and theory of every part. Learn to use Node.js like a professional. You’ll learn: Basic Of Node, Modules, NPM In Node, Event, Email, Uploading File, Advance Of Node.

Node.js for Beginners

Learn Node.js from Scratch (Step by Step)

Welcome to my course "Node.js for Beginners - Learn Node.js from Scratch". This course will guide you step by step so that you will learn basics and theory of every part. This course contain hands on example so that you can understand coding in Node.js better. If you have no previous knowledge or experience in Node.js, you will like that the course begins with Node.js basics. otherwise if you have few experience in programming in Node.js, this course can help you learn some new information . This course contain hands on practical examples without neglecting theory and basics. Learn to use Node.js like a professional. This comprehensive course will allow to work on the real world as an expert!
What you’ll learn:

  • Basic Of Node
  • Modules
  • NPM In Node
  • Event
  • Email
  • Uploading File
  • Advance Of Node

Top 7 Most Popular Node.js Frameworks You Should Know

Top 7 Most Popular Node.js Frameworks You Should Know

Node.js is an open-source, cross-platform, runtime environment that allows developers to run JavaScript outside of a browser. In this post, you'll see top 7 of the most popular Node frameworks at this point in time (ranked from high to low by GitHub stars).

Node.js is an open-source, cross-platform, runtime environment that allows developers to run JavaScript outside of a browser.

One of the main advantages of Node is that it enables developers to use JavaScript on both the front-end and the back-end of an application. This not only makes the source code of any app cleaner and more consistent, but it significantly speeds up app development too, as developers only need to use one language.

Node is fast, scalable, and easy to get started with. Its default package manager is npm, which means it also sports the largest ecosystem of open-source libraries. Node is used by companies such as NASA, Uber, Netflix, and Walmart.

But Node doesn't come alone. It comes with a plethora of frameworks. A Node framework can be pictured as the external scaffolding that you can build your app in. These frameworks are built on top of Node and extend the technology's functionality, mostly by making apps easier to prototype and develop, while also making them faster and more scalable.

Below are 7of the most popular Node frameworks at this point in time (ranked from high to low by GitHub stars).

Express

With over 43,000 GitHub stars, Express is the most popular Node framework. It brands itself as a fast, unopinionated, and minimalist framework. Express acts as middleware: it helps set up and configure routes to send and receive requests between the front-end and the database of an app.

Express provides lightweight, powerful tools for HTTP servers. It's a great framework for single-page apps, websites, hybrids, or public HTTP APIs. It supports over fourteen different template engines, so developers aren't forced into any specific ORM.

Meteor

Meteor is a full-stack JavaScript platform. It allows developers to build real-time web apps, i.e. apps where code changes are pushed to all browsers and devices in real-time. Additionally, servers send data over the wire, instead of HTML. The client renders the data.

The project has over 41,000 GitHub stars and is built to power large projects. Meteor is used by companies such as Mazda, Honeywell, Qualcomm, and IKEA. It has excellent documentation and a strong community behind it.

Koa

Koa is built by the same team that built Express. It uses ES6 methods that allow developers to work without callbacks. Developers also have more control over error-handling. Koa has no middleware within its core, which means that developers have more control over configuration, but which means that traditional Node middleware (e.g. req, res, next) won't work with Koa.

Koa already has over 26,000 GitHub stars. The Express developers built Koa because they wanted a lighter framework that was more expressive and more robust than Express. You can find out more about the differences between Koa and Express here.

Sails

Sails is a real-time, MVC framework for Node that's built on Express. It supports auto-generated REST APIs and comes with an easy WebSocket integration.

The project has over 20,000 stars on GitHub and is compatible with almost all databases (MySQL, MongoDB, PostgreSQL, Redis). It's also compatible with most front-end technologies (Angular, iOS, Android, React, and even Windows Phone).

Nest

Nest has over 15,000 GitHub stars. It uses progressive JavaScript and is built with TypeScript, which means it comes with strong typing. It combines elements of object-oriented programming, functional programming, and functional reactive programming.

Nest is packaged in such a way it serves as a complete development kit for writing enterprise-level apps. The framework uses Express, but is compatible with a wide range of other libraries.

LoopBack

LoopBack is a framework that allows developers to quickly create REST APIs. It has an easy-to-use CLI wizard and allows developers to create models either on their schema or dynamically. It also has a built-in API explorer.

LoopBack has over 12,000 GitHub stars and is used by companies such as GoDaddy, Symantec, and the Bank of America. It's compatible with many REST services and a wide variety of databases (MongoDB, Oracle, MySQL, PostgreSQL).

Hapi

Similar to Express, hapi serves data by intermediating between server-side and client-side. As such, it's can serve as a substitute for Express. Hapi allows developers to focus on writing reusable app logic in a modular and prescriptive fashion.

The project has over 11,000 GitHub stars. It has built-in support for input validation, caching, authentication, and more. Hapi was originally developed to handle all of Walmart's mobile traffic during Black Friday.