Node Google Docs REST API Batch update invalid JSON payload/missing documentId

When attempting to perform a batch update and I receive a Invalid JSON payload error; and trying to stringify JSON gives a 'missing documentId' error. The problem isn't with OAuth, the token or document scope as these are all correct and functioning (scope was changed from the example's read only to the full document scope).

When attempting to perform a batch update and I receive a Invalid JSON payload error; and trying to stringify JSON gives a 'missing documentId' error. The problem isn't with OAuth, the token or document scope as these are all correct and functioning (scope was changed from the example's read only to the full document scope).

Since there isn't an example for Google's new batch update API in node I have been having significant issues with batch update. After some troubleshooting with the batchUpdate constructor I have narrowed down my issue to (potentially) a larger issue with the API url constructor, or my syntax is wrong (or both.) Or I am missing steps to create the appropriate objects for the API call (no documentation is present for these tasks)

Inside a callback after successful get document as per google node quickstart guide (mostly)

    let offset = startIndex + 12
    let updateObject = {
      documentId:doc_id,
      requests:
        [{
          insertTextRequest : 
            {
              text : 'John Doe',
              location : {
                index : offset
              }
            }
        }]
      } 
    docs.documents.batchUpdate(updateObject,function(e,r){
      console.log(e)
      console.log(r)
    }

Google API response

'Invalid JSON payload received. Unknown name "requests[insertText][location][index]": Cannot bind query parameter. Field \'requests[insertText][location][index]\' could not be found in request message.\nInvalid JSON payload received. Unknown name "requests[insertText][text]": Cannot bind query parameter. Field \'requests[insertText][text]\' could not be found in request message.',
   domain: 'global',
   reason: 'badRequest' } ] }

Response after trying JSON.stringify(updateObject) - truncated

Error: Missing required parameters: documentId
at node_modules\googleapis-common\build\src\apirequest.js:114:19
at Generator.next (<anonymous>)

My best guess is some kind of google voodoo magic needs to occur for the API to properly encode the JSON object for the request to succeed.

  • Changing the array of a single request to an object had no effect on the above.
  • Using single/double quotes for the request object parameter names/string variable had no effect.
  • Document ID is a string, works, just not shown in the code example.
  • Adding documentId field to the request had no effect.


Angular 7 CRUD with Nodejs and MySQL Example

Angular 7 CRUD with Nodejs and MySQL Example

Angular7 CRUD with nodejs and mysql example - Hey there, Today we will proceed to create a demo for CRUD with Mysql, Express, Angular7(MEAN) and Nodejs from scratch using Angular CLI

Below are the requirements for creating the CRUD on MEAN

  • Node.js
  • Angular CLI
  • Angular 7
  • Mysql
  • IDE or Text Editor

We assume that you have already available the above tools/frameworks and you are familiar with all the above that what individually actually does.

So now we will proceed step by step to achieve the task.

1. Update Angular CLI and Create Angular 7 Application

At first, We have to update the Angular CLI to the latest version. Open the terminal then go to the project folder and then type the below command to update the Angular CLI

sudo npm install -g @angular/cli

Once the above task finishes, Next task is to create new angular application with below command. So go to your project folder and then type below command:

ng new angular7-crud

then go to the newly created folder of angular application with cd /angular7-crud  and type **ng serve. **Now, open the browser then go to http://localhost:4200 you should see this page.

Angular 7 CRUD with Nodejs and MySQL Example

2. Create a server with node.js express and Mysql for REST APIs

create a separate folder named server for server-side stuff, Then move inside folder and create server.js by typing touch server.js

Let’s have a look on the server.js file

let app = require('express')(),
server = require('http').Server(app),
bodyParser = require('body-parser')
express = require('express'),
cors = require('cors'),
http = require('http'),
path = require('path');
 
let articleRoute = require('./Routes/article'),
util = require('./Utilities/util');
 
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({extended: false }));
 
app.use(cors());
 
app.use(function(err, req, res, next) {
return res.send({ "statusCode": util.statusCode.ONE, "statusMessage": util.statusMessage.SOMETHING_WENT_WRONG });
});
 
app.use('/article', articleRoute);
 
// catch 404 and forward to error handler
app.use(function(req, res, next) {
next();
});
 
/*first API to check if server is running*/
app.get('*', (req, res) => {
res.sendFile(path.join(__dirname, '../server/client/dist/index.html'));
})
 
 
server.listen(3000,function(){
console.log('app listening on port: 3000');
});

In the above file we can see, at the top, there are required packages for the app. Below that body parsing, middleware and routing is done.

The next task is to create routes and create a file article.js . So creating a folder name ‘Routes’ and adding article.js within it.

Add the below code for routing in article.js inside routing folder

let express = require('express'),
router = express.Router(),
util = require('../Utilities/util'),
articleService = require('../Services/article');
 
/**Api to create article */
router.post('/create-article', (req, res) => {
articleService.createArticle(req.body, (data) => {
res.send(data);
});
});
 
// /**Api to update article */
router.put('/update-article', (req, res) => {
articleService.updateArticle(req.body, (data) => {
res.send(data);
});
});
 
// /**Api to delete the article */
router.delete('/delete-article', (req, res) => {
articleService.deleteArticle(req.query, (data) => {
res.send(data);
});
});
 
/**Api to get the list of article */
router.get('/get-article', (req, res) => {
documentService.getArticle(req.query, (data) => {
res.send(data);
});
});
 
// /**API to get the article by id... */
router.get('/get-article-by-id', (req, res) => {
articleService.getArticleById(req.query, (data) => {
res.send(data);
});
});
 
module.exports = router;

Now create a folder named Utilities for all config, common methods and mysql connection config.

Now I am adding config values in a file named config.js

let environment = "dev";
 
let serverURLs = {
"dev": {
"NODE_SERVER": "http://localhost",
"NODE_SERVER_PORT": "3000",
"MYSQL_HOST": 'localhost',
"MYSQL_USER": 'root',
"MYSQL_PASSWORD": 'password',
'MYSQL_DATABASE': 'demo_angular7_crud',
}
}
 
let config = {
"DB_URL_MYSQL": {
"host": `${serverURLs[environment].MYSQL_HOST}`,
"user": `${serverURLs[environment].MYSQL_USER}`,
"password": `${serverURLs[environment].MYSQL_PASSWORD}`,
"database": `${serverURLs[environment].MYSQL_DATABASE}`
},
"NODE_SERVER_PORT": {
"port": `${serverURLs[environment].NODE_SERVER_PORT}`
},
"NODE_SERVER_URL": {
"url": `${serverURLs[environment].NODE_SERVER}`
}
};
 
module.exports = {
config: config
};

Now configure mysql connection. So I am writing the connection with database in a separate file. So creating a file named mysqkConfig.js under Utilities folder and adding the below line of code for mysql connection:

var config = require("../Utilities/config").config;
var mysql = require('mysql');
var connection = mysql.createConnection({
host: config.DB_URL_MYSQL.host,
user: config.DB_URL_MYSQL.user,
password: config.DB_URL_MYSQL.password,
database: config.DB_URL_MYSQL.database,
});
 
connection.connect(() => {
require('../Models/Article').initialize();
});
 
let getDB = () => {
return connection;
}
 
module.exports = {
getDB: getDB
}

Now I am creating separate file name util.js to save common methods and common status code/message:

// Define Error Codes
let statusCode = {
OK: 200,
FOUR_ZERO_FOUR: 404,
FOUR_ZERO_THREE: 403,
FOUR_ZERO_ONE: 401,
FIVE_ZERO_ZERO: 500
};
 
// Define Error Messages
let statusMessage = {
SERVER_BUSY : 'Our Servers are busy. Please try again later.',
DATA_UPDATED: 'Data updated successfully.',
DELETE_DATA : 'Delete data successfully',
 
};
 
module.exports = {
statusCode: statusCode,
statusMessage: statusMessage
}

Now the next part is model, So create a folder named Models and create a file **Article.js **and add the below code in it:

let mysqlConfig = require("../Utilities/mysqlConfig");
 
let initialize = () => {
mysqlConfig.getDB().query("create table IF NOT EXISTS article (id INT auto_increment primary key, category VARCHAR(30), title VARCHAR(24))");
 
}
 
module.exports = {
initialize: initialize
}

Now create DAO folder and add a file articleDAO.js for writting the mysql queries common functions:

let dbConfig = require("../Utilities/mysqlConfig");


 
let getArticle = (criteria, callback) => {
//criteria.aricle_id ? conditions += ` and aricle_id = '${criteria.aricle_id}'` : true;
dbConfig.getDB().query(`select * from article where 1`,criteria, callback);
}
 
let getArticleDetail = (criteria, callback) => {
    let conditions = "";
criteria.id ? conditions += ` and id = '${criteria.id}'` : true;
dbConfig.getDB().query(`select * from article where 1 ${conditions}`, callback);
}
 
let createArticle = (dataToSet, callback) => {
console.log("insert into article set ? ", dataToSet,'pankaj')
dbConfig.getDB().query("insert into article set ? ", dataToSet, callback);
}
 
let deleteArticle = (criteria, callback) => {
let conditions = "";
criteria.id ? conditions += ` and id = '${criteria.id}'` : true;
console.log(`delete from article where 1 ${conditions}`);
dbConfig.getDB().query(`delete from article where 1 ${conditions}`, callback);
 
}
 
let updateArticle = (criteria,dataToSet,callback) => {
    let conditions = "";
let setData = "";
criteria.id ? conditions += ` and id = '${criteria.id}'` : true;
dataToSet.category ? setData += `category = '${dataToSet.category}'` : true;
dataToSet.title ? setData += `, title = '${dataToSet.title}'` : true;
console.log(`UPDATE article SET ${setData} where 1 ${conditions}`);
dbConfig.getDB().query(`UPDATE article SET ${setData} where 1 ${conditions}`, callback);
}
module.exports = {
getArticle : getArticle,
createArticle : createArticle,
deleteArticle : deleteArticle,
updateArticle : updateArticle,
getArticleDetail : getArticleDetail
}

Now one create Services folder and add a file article.js for all the logic of API

let async = require('async'),
parseString = require('xml2js').parseString;
 
let util = require('../Utilities/util'),
articleDAO = require('../DAO/articleDAO');
//config = require("../Utilities/config").config;
 
 
/**API to create the atricle */
let createArticle = (data, callback) => {
async.auto({
article: (cb) => {
var dataToSet = {
"category":data.category?data.category:'',
"title":data.title,
}
console.log(dataToSet);
articleDAO.createArticle(dataToSet, (err, dbData) => {
if (err) {
cb(null, { "statusCode": util.statusCode.FOUR_ZERO_ONE, "statusMessage": util.statusMessage.SERVER_BUSY });
return;
}
 
cb(null, { "statusCode": util.statusCode.OK, "statusMessage": util.statusMessage.DATA_UPDATED,"result":dataToSet });
});
}
//]
}, (err, response) => {
callback(response.article);
});
}
 
/**API to update the article */
let updateArticle = (data,callback) => {
async.auto({
articleUpdate :(cb) =>{
if (!data.id) {
cb(null, { "statusCode": util.statusCode.FOUR_ZERO_ONE, "statusMessage": util.statusMessage.PARAMS_MISSING })
return;
}
console.log('phase 1');
var criteria = {
id : data.id,
}
var dataToSet={
"category": data.category,
"title":data.title,
}
console.log(criteria,'test',dataToSet);
                    articleDAO.updateArticle(criteria, dataToSet, (err, dbData)=>{
                        if(err){
cb(null,{"statusCode":util.statusCode.FOUR_ZERO_ONE,"statusMessage":util.statusMessage.SERVER_BUSY});
                        return; 
                        }
                        else{
cb(null, { "statusCode": util.statusCode.OK, "statusMessage": util.statusMessage.DATA_UPDATED,"result":dataToSet });                        
                        }
                    });
}
}, (err,response) => {
callback(response.articleUpdate);
});
}
 
/**API to delete the subject */
let deleteArticle = (data,callback) => {
console.log(data,'data to set')
async.auto({
removeArticle :(cb) =>{
if (!data.id) {
cb(null, { "statusCode": util.statusCode.FOUR_ZERO_ONE, "statusMessage": util.statusMessage.PARAMS_MISSING })
return;
}
var criteria = {
id : data.id,
}
articleDAO.deleteArticle(criteria,(err,dbData) => {
if (err) {
console.log(err);
cb(null, { "statusCode": util.statusCode.FOUR_ZERO_ONE, "statusMessage": util.statusMessage.SERVER_BUSY });
return;
}
cb(null, { "statusCode": util.statusCode.OK, "statusMessage": util.statusMessage.DELETE_DATA });
});
}
}, (err,response) => {
callback(response.removeArticle);
});
}
 
/***API to get the article list */
let getArticle = (data, callback) => {
async.auto({
article: (cb) => {
articleDAO.getArticle({},(err, data) => {
if (err) {
cb(null, {"errorCode": util.statusCode.INTERNAL_SERVER_ERROR,"statusMessage": util.statusMessage.SERVER_BUSY});
return;
}
cb(null, data);
return;
});
}
}, (err, response) => {
callback(response.article);
})
}
 
/***API to get the article detail by id */
let getArticleById = (data, callback) => {
async.auto({
article: (cb) => {
let criteria = {
"id":data.id
}
articleDAO.getArticleDetail(criteria,(err, data) => {
if (err) {
console.log(err,'error----');
cb(null, {"errorCode": util.statusCode.INTERNAL_SERVER_ERROR,"statusMessage": util.statusMessage.SERVER_BUSY});
return;
}
cb(null, data[0]);
return;
});
}
}, (err, response) => {
callback(response.article);
})
}
 
module.exports = {
createArticle : createArticle,
updateArticle : updateArticle,
deleteArticle : deleteArticle,
getArticle : getArticle,
getArticleById : getArticleById
};

3. Create angular component for performing CRUD task of article

ng g component article

Above command will generate all required files for build article component and also automatically added this component to app.module.ts.

create src/app/article/article.component.css (0 bytes)
create src/app/article/article.component.html (23 bytes)
create src/app/article/article.component.spec.ts (614 bytes)
create src/app/article/article.component.ts (321 bytes)
update src/app/app.module.ts (390 bytes)

Now we need to add HttpClientModule to app.module.ts. Open and edit src/app/app.module.ts then add this import. And add it to @NgModule imports after BrowserModule. Now our app.module.ts will have following code:

import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';
import { ReactiveFormsModule } from '@angular/forms';
import { HttpModule } from '@angular/http';
 
import { AppComponent } from './app.component';
import { ArticleComponent } from './article.component';
import { ArticleService } from './article.service';
 
@NgModule({
imports: [
BrowserModule,
HttpModule,
ReactiveFormsModule
],
declarations: [
AppComponent,
ArticleComponent
],
providers: [
ArticleService
],
bootstrap: [
AppComponent
]
})
export class AppModule { }

Now create a service file where we will make all the request to the server for CRUD operation. Command for creating service is ng g service artcle , for now I have just created a file named it article.service.ts. Let's have a look in the code inside this file.

import { Injectable } from '@angular/core';
import { Http, Response, Headers, URLSearchParams, RequestOptions } from '@angular/http';
import { Observable } from 'rxjs';
import 'rxjs/add/operator/map';
import 'rxjs/add/operator/catch';
 
import { Article } from './article';
 
@Injectable()
export class ArticleService {
//URL for CRUD operations
    articleUrl = "http://localhost:3000/article";
    //Create constructor to get Http instance
    constructor(private http:Http) {
    }
    
    //Fetch all articles
getAllArticles(): Observable<Article[]> {
return this.http.get(this.articleUrl+"/get-article")
              .map(this.extractData)
         .catch(this.handleError);
 
}
    //Create article
createArticle(article: Article):Observable<number> {
     let cpHeaders = new Headers({ 'Content-Type': 'application/json' });
let options = new RequestOptions({ headers: cpHeaders });
return this.http.post(this.articleUrl+"/create-article", article, options)
.map(success => success.status)
.catch(this.handleError);
}
    //Fetch article by id
getArticleById(articleId: string): Observable<Article> {
        let cpHeaders = new Headers({ 'Content-Type': 'application/json' });
        let options = new RequestOptions({ headers: cpHeaders });
        console.log(this.articleUrl +"/get-article-by-id?id="+ articleId);
        return this.http.get(this.articleUrl +"/get-article-by-id?id="+ articleId)
             .map(this.extractData)
             .catch(this.handleError);
}   
    //Update article
updateArticle(article: Article):Observable<number> {
     let cpHeaders = new Headers({ 'Content-Type': 'application/json' });
        let options = new RequestOptions({ headers: cpHeaders });
return this.http.put(this.articleUrl +"/update-article", article, options)
.map(success => success.status)
.catch(this.handleError);
}
//Delete article    
deleteArticleById(articleId: string): Observable<number> {
        let cpHeaders = new Headers({ 'Content-Type': 'application/json' });
        let options = new RequestOptions({ headers: cpHeaders });
        return this.http.delete(this.articleUrl +"/delete-article?id="+ articleId)
             .map(success => success.status)
             .catch(this.handleError);
}   
    private extractData(res: Response) {
        let body = res.json();
return body;
}
private handleError (error: Response | any) {
        console.error(error.message || error);
        return Observable.throw(error.status);
}
}

In the above file we have made all the http request for the CRUD operation. Observables of rxjs library has been used to handle the data fetching from http request.

Now let's move to the next file, article.component.ts. Here we have all the login part of the app. Let's have a look code inside this file:

import { Component, OnInit } from '@angular/core';
import { FormControl, FormGroup, Validators } from '@angular/forms';
 
import { ArticleService } from './article.service';
import { Article } from './article';
 
@Component({
selector: 'app-article',
templateUrl: './article.component.html',
styleUrls: ['./article.component.css']
})
export class ArticleComponent implements OnInit {
//Component properties
allArticles: Article[];
statusCode: number;
requestProcessing = false;
articleIdToUpdate = null;
processValidation = false;
//Create form
articleForm = new FormGroup({
title: new FormControl('', Validators.required),
category: new FormControl('', Validators.required)   
});
//Create constructor to get service instance
constructor(private articleService: ArticleService) {
}
//Create ngOnInit() and and load articles
ngOnInit(): void {
     this.getAllArticles();
}
//Fetch all articles
 
getAllArticles() {
        this.articleService.getAllArticles()
         .subscribe(
data => this.allArticles = data,
                errorCode => this.statusCode = errorCode);
                
}
//Handle create and update article
onArticleFormSubmit() {
     this.processValidation = true;
     if (this.articleForm.invalid) {
     return; //Validation failed, exit from method.
     }
     //Form is valid, now perform create or update
this.preProcessConfigurations();
     let article = this.articleForm.value;
     if (this.articleIdToUpdate === null) {
     //Generate article id then create article
this.articleService.getAllArticles()
     .subscribe(articles => {
            
         //Generate article id    
         let maxIndex = articles.length - 1;
         let articleWithMaxIndex = articles[maxIndex];
         let articleId = articleWithMaxIndex.id + 1;
         article.id = articleId;
         console.log(article,'this is form data---');
         //Create article
    this.articleService.createArticle(article)
             .subscribe(successCode => {
                    this.statusCode = successCode;
                    this.getAllArticles();  
                    this.backToCreateArticle();
                 },
                 errorCode => this.statusCode = errorCode
             );
         });        
     } else {
  //Handle update article
article.id = this.articleIdToUpdate;        
     this.articleService.updateArticle(article)
     .subscribe(successCode => {
         this.statusCode = successCode;
                 this.getAllArticles();  
                    this.backToCreateArticle();
             },
         errorCode => this.statusCode = errorCode);  
     }
}
//Load article by id to edit
loadArticleToEdit(articleId: string) {
this.preProcessConfigurations();
this.articleService.getArticleById(articleId)
     .subscribe(article => {
            console.log(article,'poiuytre');
         this.articleIdToUpdate = article.id;
                    this.articleForm.setValue({ title: article.title, category: article.category });
                    this.processValidation = true;
                    this.requestProcessing = false;
         },
         errorCode => this.statusCode = errorCode);
}
//Delete article
deleteArticle(articleId: string) {
this.preProcessConfigurations();
this.articleService.deleteArticleById(articleId)
     .subscribe(successCode => {
         //this.statusCode = successCode;
                    //Expecting success code 204 from server
                    this.statusCode = 204;
                 this.getAllArticles();  
                 this.backToCreateArticle();
             },
         errorCode => this.statusCode = errorCode);
}
//Perform preliminary processing configurations
preProcessConfigurations() {
this.statusCode = null;
     this.requestProcessing = true;
}
//Go back from update to create
backToCreateArticle() {
this.articleIdToUpdate = null;
this.articleForm.reset(); 
     this.processValidation = false;
}
}

Now we have to show the task over browser, So lets have a look inside article.component.html file.

<h1 class="text-center">Angular 7 CRUD Demo App</h1>
<h3 class="text-center" *ngIf="articleIdToUpdate; else create">
Update Article for Id: {{articleIdToUpdate}}
</h3>
<ng-template #create>
<h3 class="text-center"> Create New Article </h3>
</ng-template>
<div>
<form [formGroup]="articleForm" (ngSubmit)="onArticleFormSubmit()">
<table class="table-striped" style="margin:0 auto;">
<tr><td>Enter Title</td><td><input formControlName="title">
   <label *ngIf="articleForm.get('title').invalid && processValidation" [ngClass] = "'error'"> Title is required. </label>
 </td></tr>
<tr><td>Enter Category</td><td><input formControlName="category">
   <label *ngIf="articleForm.get('category').invalid && processValidation" [ngClass] = "'error'"> Category is required. </label>
  </td></tr>  
<tr><td colspan="2">
   <button class="btn btn-default" *ngIf="!articleIdToUpdate">CREATE</button>
    <button class="btn btn-default" *ngIf="articleIdToUpdate">UPDATE</button>
   <button (click)="backToCreateArticle()" *ngIf="articleIdToUpdate">Go Back</button>
  </td></tr>
</table>
</form>
<br/>
<div class="text-center" *ngIf="statusCode; else processing">
<div *ngIf="statusCode === 201" [ngClass] = "'success'">
   Article added successfully.
</div>
<div *ngIf="statusCode === 409" [ngClass] = "'success'">
Article already exists.
</div>   
<div *ngIf="statusCode === 200" [ngClass] = "'success'">
Article updated successfully.
</div>   
<div *ngIf="statusCode === 204" [ngClass] = "'success'">
Article deleted successfully.
</div>   
<div *ngIf="statusCode === 500" [ngClass] = "'error'">
Internal Server Error.
</div> 
</div>
<ng-template #processing>
  <img *ngIf="requestProcessing" src="assets/images/loading.gif">
</ng-template>
</div>
<h3 class="text-center">Article List</h3>
<table class="table-striped" style="margin:0 auto;" *ngIf="allArticles">
<tr><th> Id</th> <th>Title</th><th>Category</th><th></th><th></th></tr>
<tr *ngFor="let article of allArticles" >
<td>{{article.id}}</td> <td>{{article.title}}</td> <td>{{article.category}}</td>
  <td><button class="btn btn-default" type="button" (click)="loadArticleToEdit(article.id)">Edit</button> </td>
  <td><button class="btn btn-default" type="button" (click)="deleteArticle(article.id)">Delete</button></td>
</tr>
</table>

Now since I have created server and client two separate folder for nodejs and angular task. So will run both the apps with npm start over two tabs of terminal.

On the browser, over link http://localhost:4200. App will look like below

Angular CRUD with Nodejs and MySQL Example

That’s all for now. Thank you for reading and I hope this post will be very helpful for creating CRUD operations with angular7,node.js & mysql.

================================================

Thanks for reading :heart: If you liked this post, share it with all of your programming buddies! Follow me on Facebook | Twitter

Build RESTful API In Laravel 5.8 Example

Build RESTful API In Laravel 5.8 Example

In this tutorial, i will explain you how to create rest api in laravel 5.8 application. we will use passport for api authentication. we will create register and login api with product crud api.

In this tutorial, i will explain you how to create rest api in laravel 5.8 application. we will use passport for api authentication. we will create register and login api with product crud api.

If you want to create web services with php than i will must suggest to use laravel 5.8 to create apis because laravel provide structure with authentication using passport. Based on structure it will become a very easily way to create rest apis.

Just Few days ago, laravel released it's new version as laravel 5.8. As we know laravel is a more popular because of security feature. So many of the developer choose laravel to create rest api for mobile app developing. Yes Web services is a very important when you create web and mobile developing, because you can create same database and work with same data.

Follow bellow few steps to create restful api example in laravel 5.8 app.

Step 1: Download Laravel 5.8

I am going to explain step by step from scratch so, we need to get fresh Laravel 5.8 application using bellow command, So open your terminal OR command prompt and run bellow command:

composer create-project --prefer-dist laravel/laravel blog

Step 2: Install Passport

In this step we need to install passport via the Composer package manager, so one your terminal and fire bellow command:

composer require laravel/passport

After successfully install package, we require to get default migration for create new passport tables in our database. so let's run bellow command.

php artisan migrate

Next, we need to install passport using command, Using passport:install command, it will create token keys for security. So let's run bellow command:

php artisan passport:install

Step 3: Passport Configuration

In this step, we have to configuration on three place model, service provider and auth config file. So you have to just following change on that file.

In model we added HasApiTokens class of Passport,

In AuthServiceProvider we added "Passport::routes()",

In auth.php, we added api auth configuration.

app/User.php

<?php
  
namespace App;
  
use Illuminate\Notifications\Notifiable;
use Illuminate\Contracts\Auth\MustVerifyEmail;
use Laravel\Passport\HasApiTokens;
use Illuminate\Foundation\Auth\User as Authenticatable;
  
class User extends Authenticatable implements MustVerifyEmail
{
    use HasApiTokens, Notifiable;
  
    /**
     * The attributes that are mass assignable.
     *
     * @var array
     */
    protected $fillable = [
        'name', 'email', 'password',
    ];
  
    /**
     * The attributes that should be hidden for arrays.
     *
     * @var array
     */
    protected $hidden = [
        'password', 'remember_token',
    ];
}

app/Providers/AuthServiceProvider.php

<?php

namespace App\Providers;

use Laravel\Passport\Passport;
use Illuminate\Support\Facades\Gate;
use Illuminate\Foundation\Support\Providers\AuthServiceProvider as ServiceProvider;

class AuthServiceProvider extends ServiceProvider
{
    /**
     * The policy mappings for the application.
     *
     * @var array
     */
    protected $policies = [
        'App\Model' => 'App\Policies\ModelPolicy',
    ];

    /**
     * Register any authentication / authorization services.
     *
     * @return void
     */
    public function boot()
    {
        $this->registerPolicies();

        Passport::routes();
    }
}

config/auth.php

<?php

return [
    .....
    'guards' => [
        'web' => [
            'driver' => 'session',
            'provider' => 'users',
        ],
        'api' => [
            'driver' => 'passport',
            'provider' => 'users',
        ],
    ],
    .....
]

Step 4: Add Product Table and Model

next, we require to create migration for posts table using Laravel 5.8 php artisan command, so first fire bellow command:

php artisan make:migration create_products_table

After this command you will find one file in following path database/migrations and you have to put bellow code in your migration file for create products table.

<?php

use Illuminate\Support\Facades\Schema;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Database\Migrations\Migration;

class CreateProductsTable extends Migration
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        Schema::create('products', function (Blueprint $table) {
            $table->increments('id');
            $table->string('name');
            $table->text('detail');
            $table->timestamps();
        });
    }

    /**
     * Reverse the migrations.
     *
     * @return void
     */
    public function down()
    {
        Schema::dropIfExists('products');
    }
}

After create migration we need to run above migration by following command:

php artisan migrate

After create "products" table you should create Product model for products, so first create file in this path app/Product.php and put bellow content in item.php file:

app/Product.php

<?php

namespace App;

use Illuminate\Database\Eloquent\Model;

class Product extends Model
{
    /**
     * The attributes that are mass assignable.
     *
     * @var array
     */
    protected $fillable = [
        'name', 'detail'
    ];
}

Step 5: Create API Routes

In this step, we will create api routes. Laravel provide api.php file for write web services route. So, let's add new route on that file.

routes/api.php

<?php
  
/*
|--------------------------------------------------------------------------
| API Routes
|--------------------------------------------------------------------------
|
| Here is where you can register API routes for your application. These
| routes are loaded by the RouteServiceProvider within a group which
| is assigned the "api" middleware group. Enjoy building your API!
|
*/
  
Route::post('register', 'API\[email protected]');
  
Route::middleware('auth:api')->group( function () {
	Route::resource('products', 'API\ProductController');
});

Step 6: Create Controller Files

in next step, now we have create new controller as BaseController, ProductController and RegisterController, i created new folder "API" in Controllers folder because we will make alone APIs controller, So let's create both controller:

app/Http/Controllers/API/BaseController.php

<?php

namespace App\Http\Controllers\API;

use Illuminate\Http\Request;
use App\Http\Controllers\Controller as Controller;

class BaseController extends Controller
{
    /**
     * success response method.
     *
     * @return \Illuminate\Http\Response
     */
    public function sendResponse($result, $message)
    {
    	$response = [
            'success' => true,
            'data'    => $result,
            'message' => $message,
        ];

        return response()->json($response, 200);
    }

    /**
     * return error response.
     *
     * @return \Illuminate\Http\Response
     */
    public function sendError($error, $errorMessages = [], $code = 404)
    {
    	$response = [
            'success' => false,
            'message' => $error,
        ];

        if(!empty($errorMessages)){
            $response['data'] = $errorMessages;
        }

        return response()->json($response, $code);
    }
}

app/Http/Controllers/API/ProductController.php

<?php

namespace App\Http\Controllers\API;

use Illuminate\Http\Request;
use App\Http\Controllers\API\BaseController as BaseController;
use App\Product;
use Validator;

class ProductController extends BaseController
{
    /**
     * Display a listing of the resource.
     *
     * @return \Illuminate\Http\Response
     */
    public function index()
    {
        $products = Product::all();

        return $this->sendResponse($products->toArray(), 'Products retrieved successfully.');
    }

    /**
     * Store a newly created resource in storage.
     *
     * @param  \Illuminate\Http\Request  $request
     * @return \Illuminate\Http\Response
     */
    public function store(Request $request)
    {
        $input = $request->all();

        $validator = Validator::make($input, [
            'name' => 'required',
            'detail' => 'required'
        ]);

        if($validator->fails()){
            return $this->sendError('Validation Error.', $validator->errors());       
        }

        $product = Product::create($input);

        return $this->sendResponse($product->toArray(), 'Product created successfully.');
    }

    /**
     * Display the specified resource.
     *
     * @param  int  $id
     * @return \Illuminate\Http\Response
     */
    public function show($id)
    {
        $product = Product::find($id);

        if (is_null($product)) {
            return $this->sendError('Product not found.');
        }

        return $this->sendResponse($product->toArray(), 'Product retrieved successfully.');
    }

    /**
     * Update the specified resource in storage.
     *
     * @param  \Illuminate\Http\Request  $request
     * @param  int  $id
     * @return \Illuminate\Http\Response
     */
    public function update(Request $request, Product $product)
    {
        $input = $request->all();

        $validator = Validator::make($input, [
            'name' => 'required',
            'detail' => 'required'
        ]);

        if($validator->fails()){
            return $this->sendError('Validation Error.', $validator->errors());       
        }

        $product->name = $input['name'];
        $product->detail = $input['detail'];
        $product->save();

        return $this->sendResponse($product->toArray(), 'Product updated successfully.');
    }

    /**
     * Remove the specified resource from storage.
     *
     * @param  int  $id
     * @return \Illuminate\Http\Response
     */
    public function destroy(Product $product)
    {
        $product->delete();

        return $this->sendResponse($product->toArray(), 'Product deleted successfully.');
    }
}

app/Http/Controllers/API/RegisterController.php

<?php

namespace App\Http\Controllers\API;

use Illuminate\Http\Request;
use App\Http\Controllers\API\BaseController as BaseController;
use App\User;
use Illuminate\Support\Facades\Auth;
use Validator;

class RegisterController extends BaseController
{
    /**
     * Register api
     *
     * @return \Illuminate\Http\Response
     */
    public function register(Request $request)
    {
        $validator = Validator::make($request->all(), [
            'name' => 'required',
            'email' => 'required|email',
            'password' => 'required',
            'c_password' => 'required|same:password',
        ]);

        if($validator->fails()){
            return $this->sendError('Validation Error.', $validator->errors());       
        }

        $input = $request->all();
        $input['password'] = bcrypt($input['password']);
        $user = User::create($input);
        $success['token'] =  $user->createToken('MyApp')->accessToken;
        $success['name'] =  $user->name;

        return $this->sendResponse($success, 'User register successfully.');
    }
}

Now we are ready to to run full restful api and also passport api in laravel. so let's run our example so run bellow command for quick run:

php artisan serve

make sure in details api we will use following headers as listed bellow:

'headers' => [
    'Accept' => 'application/json',
    'Authorization' => 'Bearer '.$accessToken,
]

Here is Routes URL with Verb:

  1. Login: Verb:GET, URL:http://localhost:8000/oauth/token

  2. Register: Verb:GET, URL:http://localhost:8000/api/register

  3. List: Verb:GET, URL:http://localhost:8000/api/products

  4. Create: Verb:POST, URL:http://localhost:8000/api/products

  5. Show: Verb:GET, URL:http://localhost:8000/api/products/{id}

  6. Update: Verb:PUT, URL:http://localhost:8000/api/products/{id}

  7. Delete: Verb:DELETE, URL:http://localhost:8000/api/products/{id}

Now simply you can run above listed url like as bellow screen shot:

Login API:

Register API:

Product List API:

Product Create API:

Product Show API:

Product Update API:

Product Delete API:

I hope it can help you...

Thanks for reading ❤

If you liked this post, share it with all of your programming buddies!

Docker Best Practices for Node Developers

Docker Best Practices for Node Developers

Welcome to the "Docker Best Practices for Node Developers"! With your basic knowledge of Docker and Node.js in hand, Docker Mastery for Node.js is a course for anyone on the Node.js path. This course will help you master them together.

Welcome to the best course on the planet for using Docker with Node.js! With your basic knowledge of Docker and Node.js in hand, Docker Mastery for Node.js is a course for anyone on the Node.js path. This course will help you master them together.

My talk on all the best of Docker for Node.js developers and DevOps dealing with Node apps. From DockerCon 2019. Get the full 9-hour training course with my coupon at http://bit.ly/365ogba

Get the source code for this talk at https://github.com/BretFisher/dockercon19

Some of the many cool things you'll do in this course
  • Build Node.js Images that auto-scan for security vulnerabilities
  • Use Docker's cutting-edge BuildKit with SSH Agents and NPM Caches for better image building
  • Use docker-compose with Visual Studio Code for full Node.js debug support
  • Use BuildKit and Multi-stage Builds to create minimal and flexible Dockerfiles
  • Build custom Node.js images using distro's like CentOS and Alpine
  • Test Docker init, tini, and Node.js as a PID 1 process in containers
  • Create Node.js apps that properly startup and respond to healthchecks
  • Develop ARM based Node.js apps with Docker Desktop, and deploy to AWS A1 Servers
  • Build graceful shutdown code into your apps for zero-downtime deploys
  • Dig into HTTP connections with orchestration, and how Proxies can help
  • Study examples of Docker Swarm and Kubernetes deployments for Node.js
  • Spend time Migrating traditional (legacy) Node.js apps into containers
  • Simplify your microservice solutions with advanced Docker Compose features
What you will learn in this course

You'll start with a quick review about getting set up with Docker, as well as Docker Compose basics. That way we're on the same page for the basics.

Then you'll jump into Node.js Dockerfile basics, that way you'll have a good Dockerfile foundation for new features we'll add throughout the course.

You'll be building on all the different things you learn from each Lecture in the course. Once you have the basics down of Compose, Dockerfile, and Docker Image, then you'll focus on nuances like how Docker and Linux control the Node process and how Docker changes that to make sure you know what options there are for starting up and shutting down Node.js and the right way to do it in different scenarios.

We'll cover advanced, newer features around making the Dockerfile the most efficient and flexible as possible using things like BuildKit and Multi-stage.

Then we'll talk about distributed computing and cloud design to ensure your Node.js apps have 12-factor design in your containers, as well as learning how to migrate old apps into this new way of doing things.

Next we cover Compose and its awesome features to get really efficient local development and test set-up using the Docker Compose command line and Docker Compose YAML file.

With all this knowledge, you'll progress to production concerns and making images production-ready.

Then we'll jump into deploying those containers and running them in production. Whether you use Docker Engine or orchestration with Kubernetes or Swarm, I've got you covered. In addition, we'll cover HTTP connections and reverse proxies for connection handling and routing with multi-container systems.

Lastly, you'll get a final, big assignment where you'll be building and deploying a large, complex solution, including multiple Node.js containers that are doing different things. You'll build Docker images, Dockerfiles, and compose files, and deploy them to a server to test. You'll need to check whether connections failover properly. You'll basically take everything you've learned and apply it in one big project!

How to build a JSON API with Python

How to build a JSON API with Python

The JSON API specification is a powerful way for enabling communication between client and server. It specifies the structure of the requests and responses sent between the two, using the JSON format. The [JSON API...

The JSON API specification is a powerful way for enabling communication between client and server. It specifies the structure of the requests and responses sent between the two, using the JSON format.

The JSON API specification is a powerful way for enabling communication between client and server. It specifies the structure of the requests and responses sent between the two, using the JSON format.

As a data format, JSON has the advantages of being lightweight and readable. This makes it very easy to work with quickly and productively. The specification is designed to minimise the number of requests and the amount of data that needs sending between client and server.

Here, you can learn how to create a basic JSON API using Python and Flask. Then, the rest of the article will show you how to try out some of the features the JSON API specification has to offer.

Flask is a Python library that provides a 'micro-framework' for web development. It is great for rapid development as it comes with a simple-yet-extensible core functionality.

A really basic example of how to send a JSON-like response using Flask is shown below:

from flask import Flask

app = Flask(__name__)

@app.route('/')
def example():
   return '{"name":"Bob"}'

if __name__ == '__main__':
    app.run()

This article will use two add-ons for Flask:

The big picture

The end goal is to create an API that allows client-side interaction with an underlying database. There will be a couple of layers between the database and the client - a data abstraction layer and a resource manager layer.

Here's an overview of the steps involved:

  1. Define a database using Flask-SQLAlchemy
  2. Create a data abstraction with Marshmallow-JSONAPI
  3. Create resource managers with Flask-REST-JSONAPI
  4. Create URL endpoints and start the server with Flask

This example will use a simple schema describing modern artists and their relationships to different artworks.

Install everything

Before getting started, you'll need to set up the project. This involves creating a workspace and virtual environment, installing the modules required, and creating the main Python and database files for the project.

From the command line create a new directory and navigate inside.

$ mkdir flask-jsonapi-demo
$ cd flask-jsonapi-demo/

It is good practice to create virtual environments for each of your Python projects. You can skip this step, but it is strongly recommended.

$ python -m venv .venv
$ source .venv/bin/activate

Once your virtual environment has been created and activated, you can install the modules needed for this project.

$ pip install flask-rest-jsonapi flask-sqlalchemy

Everything you'll need will be installed as the requirements for these two extensions. This includes Flask itself, and SQLAlchemy.

The next step is to create a Python file and database for the project.

$ touch application.py artists.db

Create the database schema

Here, you will start modifying application.py to define and create the database schema for the project.

Open application.py in your preferred text editor. Begin by importing some modules. For clarity, modules will be imported as you go.

Next, create an object called app as an instance of the Flask class.

After that, use SQLAlchemy to connect to the database file you created. The final step is to define and create a table called artists.

from flask import Flask
from flask_sqlalchemy import SQLAlchemy

# Create a new Flask application
app = Flask(__name__)

# Set up SQLAlchemy
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////artists.db'
db = SQLAlchemy(app)

# Define a class for the Artist table
class Artist(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String)
    birth_year = db.Column(db.Integer)
    genre = db.Column(db.String)

# Create the table
db.create_all()

Creating an abstraction layer

The next step uses the Marshmallow-JSONAPI module to create a logical data abstraction over the tables just defined.

The reason to create this abstraction layer is simple. It gives you more control over how your underlying data is exposed via the API. Think of this layer as a lens through which the API client can view the underlying data clearly, and only the bits they need to see.

In the code below, the data abstraction layer is defined as a class which inherits from Marshmallow-JSONAPI's Schema class. It will provide access via the API to both single records and multiple records from the artists table.

Inside this block, the Meta class defines some metadata. Specifically, the name of the URL endpoint for interacting with single records will be artist_one, where each artist will be identified by a URL parameter <id>. The name of the endpoint for interacting with many records will be artist_many.

The remaining attributes defined relate to the columns in the artists table. Here, you can control further how each is exposed via the API.

For example, when making POST requests to add new artists to the database, you can make sure the name field is mandatory by setting required=True.

And if for any reason you didn't want the birth_year field to be returned when making GET requests, you can specify so by setting load_only=True.

from marshmallow_jsonapi.flask import Schema
from marshmallow_jsonapi import fields

# Create data abstraction layer
class ArtistSchema(Schema):
    class Meta:
        type_ = 'artist'
        self_view = 'artist_one'
        self_view_kwargs = {'id': '<id>'}
        self_view_many = 'artist_many'

    id = fields.Integer()
    name = fields.Str(required=True)
    birth_year = fields.Integer(load_only=True)
    genre = fields.Str()

Create resource managers and URL endpoints

The final piece of the puzzle is to create a resource manager and corresponding endpoint for each of the routes /artists and /artists/id.

Each resource manager is defined as a class that inherits from the Flask-REST-JSONAPI classes ResourceList and ResourceDetail.

Here they take two attributes. schema is used to indicate the data abstraction layer the resource manager uses, and data_layer indicates the session and data model that will be used for the data layer.

Next, define api as an instance of Flask-REST-JSONAPI's Api class, and create the routes for the API with api.route(). This method takes three arguments - the data abstraction layer class, the endpoint name, and the URL path.

The last step is to write a main loop to launch the app in debug mode when the script is run directly. Debug mode is great for development, but it is not suitable for running in production.

# Create resource managers and endpoints

from flask_rest_jsonapi import Api, ResourceDetail, ResourceList

class ArtistMany(ResourceList):
    schema = ArtistSchema
    data_layer = {'session': db.session,
                  'model': Artist}

class ArtistOne(ResourceDetail):
    schema = ArtistSchema
    data_layer = {'session': db.session,
                  'model': Artist}

api = Api(app)
api.route(ArtistMany, 'artist_many', '/artists')
api.route(ArtistOne, 'artist_one', '/artists/<int:id>')

# main loop to run app in debug mode
if __name__ == '__main__':
    app.run(debug=True)

Make GET and POST requests

Now you can start using the API to make HTTP requests. This could be from a web browser, or from a command line tool like curl, or from within another program (e.g., a Python script using the Requests library).

To launch the server, run the application.py script with:

$ python application.py

In your browser, navigate to http://localhost:5000/artists.  You will see a JSON output of all the records in the database so far. Except, there are none.

To start adding records to the database, you can make a POST request. One way of doing this is from the command line using curl. Alternatively, you could use a tool like Insomnia, or perhaps code up a simple HTML user interface that posts data using a form.

With curl, from the command line:

curl -i -X POST -H 'Content-Type: application/json' -d '{"data":{"type":"artist", "attributes":{"name":"Salvador Dali", "birth_year":1904, "genre":"Surrealism"}}}' http://localhost:5000/artists

Now if you navigate to http://localhost:5000/artists, you will see the record you just added. If you were to add more records, they would all show here as well, as this URL path calls the artists_many endpoint.

To view just a single artist by their id number, you can navigate to the relevant URL. For example, to see the first artist, try http://localhost:5000/artists/1.

Filtering and sorting

One of the neat features of the JSON API specification is the ability to return the response in more useful ways by defining some parameters in the URL. For instance, you can sort the results according to a chosen field, or filter based on some criteria.

Flask-REST-JSONAPI comes with this built in.

To sort artists in order of birth year, just navigate to http://localhost:5000/artists?sort=birth_year. In a web application, this would save you from needing to sort results on the client side, which could be costly in terms of performance and therefore impact the user experience.

Filtering is also easy. You append to the URL the criteria you wish to filter on, contained in square brackets. There are three pieces of information to include:

  • "name" - the field you are filtering by (e.g., birth_year)
  • "op" - the filter operation ("equal to", "greater than", "less than" etc.)
  • "val" - the value to filter against (e.g., 1900)

For example, the URL below retrieves artists whose birth year is greater than 1900:

http://localhost:5000/artists?filter=[{"name":"birth_year","op":"gt","val":1900}]

This functionality makes it much easier to retrieve only relevant information when calling the API. This is valuable for improving performance, especially when retrieving potentially large volumes of data over a slow connection.

Pagination

Another feature of the JSON API specification that aids performance is pagination. This is when large responses are sent over several "pages", rather than all in one go. You can control the page size and the number of the page you request in the URL.

So, for example, you could receive 100 results over 10 pages instead of loading all 100 in one go. The first page would contain results 1-10, the second page would contain results 11-20, and so on.

To specify the number of results you want to receive per page, you can add the parameter ?page[size]=X to the URL, where X is the number of results. Flask-REST-JSONAPI uses 30 as the default page size.

To request a given page number, you can add the parameter ?page[number]=X, where is the page number. You can combine both parameters as shown below:

http://localhost:5000/artists?page[size]=2&page[number]=2

This URL sets the page size to two results per page, and asks for the second page of results. This would return the third and fourth results from the overall response.

Relationships

Almost always, data in one table will be related to data stored in another. For instance, if you have a table of artists, chances are you might also want a table of artworks. Each artwork is related to the artist who created it.

The JSON API specification allows you to work with relational data easily, and the Flask-REST-JSONAPI lets you take advantage of this. Here, this will be demonstrated by adding an artworks table to the database, and including relationships between artist and artwork.

To implement the artworks example, it will be necessary to make a few changes to the code in application.py.

First, make a couple of extra imports, then create a new table which relates each artwork to an artist:

from marshmallow_jsonapi.flask import Relationship
from flask_rest_jsonapi import ResourceRelationship

# Define the Artwork table
class Artwork(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String)
    artist_id = db.Column(db.Integer, 
        db.ForeignKey('artist.id'))
    artist = db.relationship('Artist',
        backref=db.backref('artworks'))

Next, rewrite the abstraction layer:

# Create data abstraction 
class ArtistSchema(Schema):
    class Meta:
        type_ = 'artist'
        self_view = 'artist_one'
        self_view_kwargs = {'id': '<id>'}
        self_view_many = 'artist_many'

    id = fields.Integer()
    name = fields.Str(required=True)
    birth_year = fields.Integer(load_only=True)
    genre = fields.Str()
    artworks = Relationship(self_view = 'artist_artworks',
        self_view_kwargs = {'id': '<id>'},
        related_view = 'artwork_many',
        many = True,
        schema = 'ArtworkSchema',
        type_ = 'artwork')

class ArtworkSchema(Schema):
    class Meta:
        type_ = 'artwork'
        self_view = 'artwork_one'
        self_view_kwargs = {'id': '<id>'}
        self_view_many = 'artwork_many'

    id = fields.Integer()
    title = fields.Str(required=True)
    artist_id = fields.Integer(required=True)

This defines an abstraction layer for the artwork table, and adds a relationship between artist and artwork to the ArtistSchema class.

Next, define new resource managers for accessing artworks many at once and one at a time, and also for accessing the relationships between artist and artwork.

class ArtworkMany(ResourceList):
    schema = ArtworkSchema
    data_layer = {'session': db.session,
                  'model': Artwork}

class ArtworkOne(ResourceDetail):
    schema = ArtworkSchema
    data_layer = {'session': db.session,
                  'model': Artwork}

class ArtistArtwork(ResourceRelationship):
    schema = ArtistSchema
    data_layer = {'session': db.session,
                  'model': Artist}

Finally, add some new endpoints:

api.route(ArtworkOne, 'artwork_one', '/artworks/<int:id>')
api.route(ArtworkMany, 'artwork_many', '/artworks')
api.route(ArtistArtwork, 'artist_artworks',
    '/artists/<int:id>/relationships/artworks')

Run application.py and trying posting some data from the command line via curl:

curl -i -X POST -H 'Content-Type: application/json' -d '{"data":{"type":"artwork", "attributes":{"title":"The Persistance of Memory", "artist_id":1}}}' http://localhost:5000/artworks

This will create an artwork related to the artist with id=1.

In the browser, navigate to http://localhost:5000/artists/1/relationships/artworks. This should show the artworks related to the artist with id=1. This saves you from writing a more complex URL with parameters to filter artworks by their artist_id field. You can quickly list all the relationships between a given artist and their artworks.

Another feature is the ability to include related results in the response to calling the artists_one endpoint:

http://localhost:5000/artists/1?include=artworks

This will return the usual response for the artists endpoint, and also results for each of that artist's artworks.

Sparse Fields

One last feature worth mentioning - sparse fields. When working with large data resources with many complex relationships, the response sizes can blow up real fast. It is helpful to only retrieve the fields you are interested in.

The JSON API specification lets you do this by adding a fields parameter to the URL. For example URL below gets the response for a given artist and their related artworks. However, instead of returning all the fields for the given artwork, it returns only the title.

http://localhost:5000/artists/1?include=artworks&fields[artwork]=title

This is again very helpful for improving performance, especially over slow connections. As a general rule, you should only make requests to and from the server with the minimal amount of data required.

Final remarks

The JSON API specification is a very useful framework for sending data between server and client in a clean, flexible format. This article has provided an overview of what you can do with it, with a worked example in Python using the Flask-REST-JSONAPI library.

So what will you do next? There are many possibilities. The example in this article has been a simple proof-of-concept, with just two tables and a single relationship between them. You can develop an application as sophisticated as you like, and create a powerful API to interact with it using all the tools provided here.

Thanks for reading, and keep coding in Python!

Understanding Protocol Buffers - Will they replace JSON?

Understanding Protocol Buffers - Will they replace JSON?

In this article, we will see what is a protocol buffer and how it works and ask the question will protocol buffers replace JSON?

In this article, we will see what is a protocol buffer and how it works and ask the question will protocol buffers replace JSON?

What are Protocol Buffers?

Protocol buffers are a flexible, efficient way of serializing structured data. you define how you want your data to be structured once, then you can use special generated source code to write and read your structured data.

it is a way of encoding structured data in an efficient and extensible format.

Protocol Buffers are developed and backed by Google. Just like Google, other companies have their own implementation like Facebook has Apache Thrift and Microsoft has Microsoft Bond Protocol Buffers in addition to concrete Remote Procedure Call(RPC)

Protocol Buffers vs JSON

Before we compare protocol buffers with JSON(Javascript Object Notation), we will see how JSON is used in the current tech world.

Let’s consider an application life cycle when a user clicks a button in an application.

The browser makes an API request to the Server.

The Server requests data from the Database.

Once the database returns the data and sends it to the application server, it then sends it to the browser to display.

Here, the way the browser and server communicate is happening with JSON.
This is image title

JSON is based on key-value pair. JSON has different value types, such as

  • Array
  • Boolean
  • Number
  • Object
  • String

Above all , Each entity in JSON should have a key and value associated with it.

Now, let’s come back to the topic of why we need Protobuf over JSON and what kind of benefits we get using protocol buffers.

Why Protocol buffers over JSON

Firstly, protocol buffers have more data types than JSON. Protocol buffers are not only a message format, but it is also a set of rules and tools that defines the exchange of messages.

Simple Analogy

Let’s say that you want to travel from location A to location B every day to reach the destination.

Now, 1000’s people will be traveling between these two locations at the same time every day.

if you travel in an SUV, it occupies more spaces on the road, it will delay your travel due to traffic. Now, instead, if you travel by bike, you reach the destination in a minimal amount of time as well as less traffic.

Let’s relate this our concept protocol buffers.

Location A to B => Sender and receiverRoads – Network bandwidth

Data – the vehicle that you are traveling in

Here sending data between a source and destination through a network using

XML -> traveling alone via truck

JSON -> traveling via SUV(better than a truck)

Protobuf -> traveling via bike (best possible way to reduce network bandwidth and allowing more request to flow via network)

How Do Protocol Buffers work?

This is image title

Protocol buffer works through binary serialization. It encodes the data using the determined schema and sends the data.

The receiver decodes the data with the schema to get the message. Here, the message is encoded with a schema and stream it as a binary data to the receiver.

The receiver decodes the data with the same schema and get the message from the binary stream.

Implementing Protocol Buffer

Like I said before, protocol work with a schema to transmit the message.

Schemas are fields that are indicated and aliased with a number and a tag.

you can have keywords such as required, optional and repeated. schema also allows messages to be extensible.

For example, User in the UserList is extensible of message User.

message UserList {
  repeated User user = 1;
}

message User{
  required string name = 1;
  required string email = 2;
}Copy

  • You specify how you want the information you are serializing to be structured by defining protocol buffer message types in .proto files.

  • That is to say, Each protocol buffer is a logical record of information, containing a series of name-value pairs.

The above is a very basic example of .proto file that defines a message containing information about User model

As you can see, the message format is simple – each message type has one or more uniquely numbered fields, and each field has a name and a value type, where value types can be numbers, booleans, strings, etc.

Once, you’ve defined your message,you run the protocol buffer compiler for the language of your application on your .proto file to generate data access classes.

This repository contains the implementation of a to-do application with Protobufs in a NodeJS application.