Building a Custom Sitemap for Your Gatsby.js Site

Building static sites with React.js using Gatsby.js seem to be more and more a topic of debate when you require an easy to deploy setup, blazing fast speed and smooth developer experience. Gatsby has been consistently growing, and I am super glad to see this shifting trend as we see a large number of blogs, marketing, and e-commerce sites built with it. 

Building static sites with React.js using Gatsby.js seem to be more and more a topic of debate when you require an easy to deploy setup, blazing fast speed and smooth developer experience. Gatsby has been consistently growing, and I am super glad to see this shifting trend as we see a large number of blogs, marketing, and e-commerce sites built with it. 

Every time, you build a site you can help search engine's crawlers to improve organic search rankings. You have to ensure that search engines like Google can understand your site's architecture and index it intelligently. You can do all that by including a sitemap.xml file at the root of your site.

Considering the growing popularity of Gatsby.*js *#JAMstack sites, I've decided to write about how you can automatically generate a custom XML sitemap file in your Gatsby powered websites.

So, let’s start with the basic stuff before we get more technical.


What is an XML Sitemap?

A website is composed of several web pages like About, Contact, Blog, Subscribe, etc. A sitemap file maintains a list of all these pages to tell search engines like Google about the organization of your site content. Search engine web crawlers like Googlebot read through this file and crawl your site intelligently.

Back in the early days of the web, HTML sitemap which was a manually generated bullet-list were in trend. But, owing to their popularity and importance, sitemaps are published in XML format instead of HTML since their target audience is search engines and not people. I hope we can move to JSON format someday.

So, an XML sitemap file communicates with the search engines about all the pages which exist on your website.


Importance of Adding a Sitemap File

Considering Search Engine Optimization (SEO), sitemaps are very important. However, they do not affect your search rankings. Instead, if there's a web page which is not indexed, then sitemap tells the search engines about that page to get it appropriately indexed.

Sitemaps are equally important for both new, and old sites. Especially if your site is relatively new then I'd definitely recommend adding one since it is difficult for search engines to find posts and pages of a new site. You want to make the search engine's job as easy as possible to get the most out of it.

You will find sitemap.xml files on literally every popular website. This helps the search engine bots to keep a tab on various updates and basically everything that goes about on a site that should be indexed.


Adding a Sitemap in Gatsby

One key highlight of Gatsby is its growing collection of plugins which implement Gatsby API through simple NPM packages. With time, good folks behind Gatsby have built an extensive ecosystem for plugins which helps you to extend and modify web functionalities.

Now, to create a sitemap you don't have to bother writing several lines of code. There is a Gatsby plugin to generate the sitemap file called the gatsby-plugin-sitemap.

Before I dive into its installation make sure you have a Gatsby site up and running.

Installation

To install the gatsby-plugin-sitemap package, run the following command in the root folder:

npm install --save gatsby-plugin-sitemap

Recently, I purchased my Maedah.dev domain where I'll be creating a sitemap file since I am setting it up a new blog with Gatsby. It'll take just a few seconds to install.

Let's install that:


Using the Plugin: gatsby-plugin-sitemap

After the plugin is successfully installed, now it's time to add this plugin to the gatsby-config.js file. A quick reminder to those who are new to Gatsby; that inside gatsby-config.js file you'll find all the site configuration options which are placed in the root folder.

One of these configuration options is for plugins. Here you find an array of plugins which implement Gatsby APIs. Some plugins are listed by name, while others may take options as well — and gatsby-plugin-sitemap carries options as well (more on this later).

So, add the following lines of code in your gatsby-config.js file:

module.exports = {
  siteMetadata: {
    title: 'Your Site Title',
    siteUrl: 'https://yoursite.com',
  },
  plugins: ['gatsby-plugin-sitemap'],
}

Make sure that the inside siteMetadata you change the title and siteUrl according to your project details.


Generating A Sitemap File

To create a sitemap you need to generate a production build and start the server. In your terminal type the following command and hit Enter.

npm run build

Wait for a few seconds and you get a working sitemap with Gatsby. Here's a GIF:

By default, the sitemap is generated in the root of your website which is a folder called public . When you deploy your site, you can access it through /sitemap.xml and it will display all of your site’s pages which are currently accessible to users.

You can access the sitemap of your site with the following URL:

https://yoursite.com/sitemap.xml

But as I have mentioned earlier that the gatsby-plugin-sitemap plugin supports advanced custom options so this default functionality can be changed accordingly. Let's dig a little deep with these options.


Advanced Options for gatsby-plugin-sitemap

The gatsby-plugin-sitemap support different advanced options which you can customize to gain more control over your sitemap.xml files. Let's take a look at some of these:

  • output: The default file name is sitemap.xml you can use the output option to change the name of output file. E.g. with output: '/some-other-sitemap.xml', the URL now becomes https://yoursite.com/some-other-sitemap.xml.
  • exclude: This option can help you exclude any links from the sitemap for whatever reasons.
  • query: Custom GraphQL query to fetch info like siteMetadata, siteURL, allSitePage, etc.

There are a couple of other handy options as well for e.g., for sitemapSize and sitemap index. You can visit the official plugin repo for more info here.


Customized Options Example

For the sake of sharing an example, I tried customizing the plugin's options to generate data of my choice. Here, I have customized the GraphQL query that I will explain below.

 {
      resolve: `gatsby-plugin-sitemap`,
      options: {
        query: `{
          site {
            siteMetadata {
              siteUrlNoSlash
            }
          }
          allSitePage {
            edges {
              node {
                path
              }
            }
          }
          allMarkdownRemark {
            edges {
              node {
                fields {
                  slug
                }
              }
            }
          }
        }`,
        serialize: ({ site, allSitePage, allMarkdownRemark }) => {
          let pages = []
          allSitePage.edges.map(edge => {
            pages.push({
              url: site.siteMetadata.siteUrlNoSlash + edge.node.path,
              changefreq: `daily`,
              priority: 0.7,
            })
          })
          allMarkdownRemark.edges.map(edge => {
            pages.push({
              url: `${site.siteMetadata.siteUrlNoSlash}/${
                edge.node.fields.slug
              }`,
              changefreq: `daily`,
              priority: 0.7,
            })
          })
      return pages
    },
  },
},

Here, I am using the query option to fetch data for my site which includes info about siteMetadata and siteUrlNoSlash. Further, I am quering the allSitePage to get all site pages URL paths i.e. to retreiving path property for each graph node through all edges. And finally, I used the allMarkdownRemark which reads files written in markdown and then converts them into HTML pages. Here we are getting the slug info for each markdown post from inside the field property.

Towards the end, I've called the serialize function to map data which is fetched for allSitePage and allMarkdownRemark. Each returns a page url with changefreq: 'daily' and a priority: 0.7.

This was one demonstration of playing around with custom options for the gatsby-plugin-sitemap, you can do it according to the requirement of your project.

Moreover, for a live demo of sitemap.xml file, I created it in the VSCode.pro course site that my better-half Ahmad Awais had built. You can access this live sitemap demo for Gatsby.js here.

Gif or didn't happen?!


Get to Code!

That's about it! Generating sitemaps should now be very easy with Gatsby.js. With time, Gatsby has stepped up its game in performance and a lot of effort is being put in improving the DX (Developer Experience). The result of creating a sitemap.xml file with gatsby-plugin-sitemap is a clear indication for that. 

Thanks for reading ❤

If you liked this post, share it with all of your programming buddies!

Follow me on Facebook | Twitter

Learn More

The Web Developer Bootcamp

Angular 7 (formerly Angular 2) - The Complete Guide

The Complete JavaScript Course 2019: Build Real Projects!

Modern React with Redux [2019 Update]

Vue JS 2 - The Complete Guide (incl. Vue Router & Vuex)

Build Responsive Real World Websites with HTML5 and CSS3

The Complete Web Developer Course 2.0

Top 10 Podcasts for Web Developers

Build Your First PWA with Angular

Building a Desktop App with Vue: NW.js

How to Perform Web-Scraping using Node.js

Build Progressive Web Apps with React

Build a web scraper with Node

Mobile App Development Company India | Ecommerce Web Development Company India

Mobile App Development Company India | Ecommerce Web Development Company India

Best Mobile App Development Company India, WebClues Global is one of the leading web and mobile app development company. Our team offers complete IT solutions including Cross-Platform App Development, CMS & E-Commerce, and UI/UX Design.

We are custom eCommerce Development Company working with all types of industry verticals and providing them end-to-end solutions for their eCommerce store development.

Know more about Top E-Commerce Web Development Company

React Web Development: A Guide to Develop Progressive Web Applications

React Web Development: A Guide to Develop Progressive Web Applications

Outperform your competition with progressive web development and React.

Progressive web applications (PWAs) quickly gained popularity because they are web performance applications based on fast performance, streamlined to provide a mobile app-like experience. PWAs are built using HTML, CSS, and JavaScript to create a level of availability and performance equivalent to that of native mobile applications. They respond quickly, consume less data, store more space, and support push notifications and offline use in the browser.

Building a progressive web application has now become the web development trend that every business wants to follow. Significant players like Twitter and Flipboard have recently rolled out their progressive web apps to provide a mobile experience for users, without requiring them to install the app. In this article, you will learn how to build a progressive web application using React. Let's get started.

Step One — Set up the React Application

First, create a React application with create-react-app. To do so, you need to run the following commands:

  • npm install -g create-react-app
  • create-react-app pwa-app

Now, install React Router:

You need to replace the content of src / App.js using the code below to get a basic template with navigation.

import React, { Component } from 'react';
import { Router, browserHistory, Route, Link } from 'react-router';
import './App.css';

const NavBar = () => (
<div className="navbar">
<Link to="/">Feed</Link>
<Link to="/profile">Profile</Link>
</div>
);

const Template = ({ title }) => (
<div>
<NavBar />
<p className="page-info">
This is the {title} page.
</p>
</div>
);

const Feed = (props) => (
<Template title="Feed"/>
);

const Profile = (props) => (
<Template title="Profile"/>
);

class App extends Component {
render() {
return (
<Router history={browserHistory}>
<Route path="/" component={Feed}/>
<Route path="/profile" component={Profile}/>
</Router>
);
}
}
export default App;

Now, you'll have to update the default styles by replacing your src/App.css with the styles below to make your application look clean.

.navbar {
background-color: #01C8E5;
text-align: center;
}

.navbar a {
display: inline-block;
padding: 10px;
color: #fff;
text-decoration: none;
}

.page-info {
text-align: center;
font-weight: bold;
}

Then, run npm start to test the application in the browser. It is basically an application with two routes. Now, you will convert it to PWA.

Step Two — Lighthouse Setting and Auditing

Lighthouse is an automated open-source tool for testing applications against PWA checklists. It facilitates audits for accessibility, performance, and more.

Check your application with Lighthouse. Click the Lighthouse icon from the top right corner in Chrome and then click the "Create Report" button. The generated report will look like this:

Creating a report to test your PWA with Lighthouse

Fix all failed audits.

Step 3 — Sign Up for a Service Staff

Service employees are proxy servers that connect the application and the network. With Service Worker, you will have to block network requests and save cached files. It allows your application to work even with an unavailable system.

Create an empty worker.js file in your application's general directory and add the following code to that file.

// Flag for enabling cache in production
var doCache = false;

var CACHE_NAME = 'pwa-app-cache';

// Delete old caches
self.addEventListener('activate', event => {
const currentCachelist = [CACHE_NAME];
event.waitUntil(
caches.keys()
.then(keyList =>
Promise.all(keyList.map(key => {
if (!currentCachelist.includes(key)) {
return caches.delete(key);
}
}))
)
);
});

// This triggers when user starts the app
self.addEventListener('install', function(event) {
if (doCache) {
event.waitUntil(
caches.open(CACHE_NAME)
.then(function(cache) {
fetch('asset-manifest.json')
.then(response => {
response.json();
})
.then(assets => {
// We will cache initial page and the main.js
// We could also cache assets like CSS and images
const urlsToCache = [
'/',
assets['main.js']
];
cache.addAll(urlsToCache);
})
})
);
}
});

// Here we intercept request and serve up the matching files
self.addEventListener('fetch', function(event) {
if (doCache) {
event.respondWith(
caches.match(event.request).then(function(response) {
return response || fetch(event.request);
})
);
}
});

Now, check if the browsers support service staff and then register worker.js. To do this, you need to add the following script to the file public/index.html (note that shrink-to-fit=no in the viewport meta tag has been deleted).

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="theme-color" content="#000000">
<link rel="manifest" href="%PUBLIC_URL%/manifest.json">
<link rel="shortcut icon" href="%PUBLIC_URL%/favicon.ico">
<title>React App</title>
</head>
<body>
<noscript>
You need to enable JavaScript to run this app.
</noscript>
<div id="root"></div>
<script>
if ('serviceWorker' in navigator) {
window.addEventListener('load', function() {
navigator.serviceWorker.register('worker.js').then(function(registration) {
console.log('Worker registration successful', registration.scope);
}, function(err) {
console.log('Worker registration failed', err);
}).catch(function(err) {
console.log(err);
});
});
} else {
console.log('Service Worker is not supported by browser.');
}
</script>
</body>
</html>

You must restart your application and reload the browser after which you will see the Successful registration of member workers on the developer console. Now, recreate the Lighthouse report.

Step 4 - Improve the Progressive Nature of the Application

Your application will display an original blank div until the JavaScript loads and React hooks the original route. You must make sure your application works without downloading JS and displaying a bit of CSS and HTML before React takes effect. Your updated Index.html will look like this:

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="theme-color" content="#000000">
<link rel="manifest" href="%PUBLIC_URL%/manifest.json">
<link rel="shortcut icon" href="%PUBLIC_URL%/favicon.ico">
<title>React App</title>
<style type="text/css">
body {
margin: 0;
padding: 0;
font-family: sans-serif;
}
.navbar {
background-color: #01C8E5;
text-align: center;
}
.navbar a {
display: inline-block;
padding: 10px;
color: #fff;
text-decoration: none;
}
.page-info {
text-align: center;
font-weight: bold;
}
</style>
</head>
<body>
<noscript>
You need to enable JavaScript to run this app.
</noscript>
<div id="root">
<div class="navbar">
<a href="/">Feed</a>
</div>
<p class="page-info">
Loading an awesome app...
</p>
</div>
<script>
if ('serviceWorker' in navigator) {
window.addEventListener('load', function() {
navigator.serviceWorker.register('worker.js').then(function(registration) {
console.log('Worker registration successful', registration.scope);
}, function(err) {
console.log('Worker registration failed', err);
}).catch(function(err) {
console.log(err);
});
});
} else {
console.log('Service Worker is not supported by browser.');
}
</script>
</body>
</html>

Now, use Lighthouse to retest your application, and you'll see an improvement in the performance of the application.

Step 5 - Add the Splash Icon

You are required to add a 512x512 icon to display on the screen. To do so, you will have to update the manifest.json file and add the t0 icon to the public directory.

{
"short_name": "React App",
"name": "Create React App Sample",
"icons": [
{
"src": "icon-192x192.png",
"sizes": "192x192",
"type": "image/png"
},
{
"src": "icon-512x512.png",
"sizes": "512x512",
"type": "image/png"
}
],
"start_url": "/",
"display": "standalone",
"theme_color": "#000000",
"background_color": "#ffffff"
}

Also, use the following meta tags to allow the browser to determine that your application is a PWA.

<!-- Tell the browser it's a PWA -->
<meta name="mobile-web-app-capable" content="yes">
<!-- Tell iOS it's a PWA -->
<meta name="apple-mobile-web-app-capable" content="yes">
Step 6 — Implement PWA

Now, only HTTPS is missing, and the cache can be fixed after you deploy the application. Update the doCache flag with true in the worker.js file. Create a new project in the firebase dashboard and name it "Pwa Application." Then, run the following command in the project directory:

npm install -g firebase-tools
firebase login
firebase init

Your Firebase.json will look like this:

{
"hosting": {
"public": "build",
"rewrites": [
{
"source": "**",
"destination": "/index.html"
}
]
}
}

After initializing, building and deploying your application.

  • npm run build
  • firebase deploy

You will see results after you test the application using Lighthouse on the deployed URL.

Final product with Lighthouse

Finally, you have created your first progressive web application with React.js!

Originally published by Manoj Kumar Bardhan at  dzone.com

=======================================================

Thanks for reading :heart: If you liked this post, share it with all of your programming buddies! Follow me on Facebook | Twitter

Learn More

☞ Understanding TypeScript

☞ Typescript Masterclass & FREE E-Book

☞ React - The Complete Guide (incl Hooks, React Router, Redux)

☞ Modern React with Redux [2019 Update]

☞ The Complete React Developer Course (w/ Hooks and Redux)

☞ React JS Web Development - The Essentials Bootcamp

☞ React JS, Angular & Vue JS - Quickstart & Comparison

☞ The Complete React Js & Redux Course - Build Modern Web Apps

☞ React JS and Redux Bootcamp - Master React Web Development


ReactJS Development Company in Canada| Hire React JS Developers

a successful and trustworthy ReactJS development company in Ontario, Canada with presence in India. Hire our professional React JS developer who will guarantee you the custom ReactJS development and create bug-free, creative, seamless and interactive user interface apps. Get our services at the most competitive prices.