A Complete Guide to JavaScript Tooling Systems

A Complete Guide to JavaScript Tooling Systems

Want to become a productive JavaScript developer? Confused no more with a complete guide to JavaScript tooling systems.

JavaScript has become very popular and powerful; you can use it to build web applications, servers, mobile applications and so much more. Huge adoption leads to a big ecosystem, the community has been building tons of tools to ease the developer experience. JavaScript tooling can be confusing even to experienced developers like me.

Let's dive into the pool of development tools built around JavaScript over the years, glimpse at the concepts behind these tools, and review some popular options in each category.

Motivation

JavaScript started as a simple scripting language in the browser, then developed into a general-purpose programming language, and many features have been introduced over the years. But browsers have different support for these features, we need transpilers to use the latest JavaScript features today.

The flexibility of JavaScript makes it error-prone; we need static type checkers to improve the safety of your code, linters to catch problems early, and formatters to enforce consistency across the codebase.

We literally can learn JavaScript once and write anything to run in multiple environments; we need build tools to prepare production-ready artifacts, task runners to automate repetitive tasks, package managers to handle dependencies, and module bundlers to build static websites.

Node.js runtime brings so much capability to web development; we need process managers to manage Node.js servers in both development and production, or debuggers to fix bugs.

We also need scaffolding tools to increase the speed of bootstrapping new projects, and many other tools in between to glue the JavaScript tooling ecosystem.

Each tool can belong to multiple categories, I'll put it where it is famous for. These tools are evolving very fast, you better check official documents before using to minimize duplication and complexity in your development workflow.

Static Type Checkers

JavaScript only supports dynamic type checking, which means type safety is only verified at runtime; it brings the flexibility to the language but allows unexpected errors at runtime.

Static type checking, the process of checking type safety based on source code at compile-time, has several awesome benefits like caching errors early, limiting type errors, supporting auto-completion, generating documentation, and resulting in faster compilation.

We can bring static type checking into JavaScript by creating static type checkers for it. They work by extending JavaScript with type systems, which will be removed at compile-time, and offering many highly-productive development tools and practices like static checking and code refactoring.

Flow is a static type checker for JavaScript, developed at Facebook, super easy to setup in an existing codebase, supports optional type system, type annotations, library definitions, linting, etc.

TypeScript is actually a programming language but can be used as a static type checker, developed by Microsoft, is a typed superset of JavaScript, compiled to plain JavaScript in the end, and has great support from major text editors and IDEs.

Flow and TypeScript have their own strengths and weaknesses. Using a static type checker for Javascript can greatly improve the safety of your code, but it does come with drawbacks like a big investment in learning, increasing the verbosity of your code, or slowing you down.

TypeScript is getting more and more popular when many companies and open source projects are migrating into it.

Code Linters

Linters are tools that analyze source code to detect problems based on formatting and code quality rules then output as warnings or errors. They are used to increase code quality, configured manually, and often run automatically when code changes.

Even though JavaScript compilers or static type checkers have evolved to include many of linting functions, linters have also evolved to detect an even wider variety of suspicious constructs: warnings about syntax errors, uses of undeclared variables, calls to deprecated functions, spacing and formatting conventions, misuse of scope, implicit fallthrough in switch statements, or missing license headers.

The three most popular JS linters are ESLint, JSHint and JSLint, they provide online GUI to test some code, they also all offer plugins for text editors like Sublime Text, VS Code, and Atom.

ESLint is an extremely configurable linter that also supports JSX and can auto format scripts to match your preferred code formatting style, too much customization makes it harder to just pick up and start using.

JSLint is highly opinionated and based on Douglas Crockford’s Javascript: The Good Parts, super easy to just start using, but does not provide enough customization.

JSHint comes loaded with sensible defaults but allows for a lot more configuration than JSLint, provides the perfect blend of ease and functionality.

Code Formatters

Code formatters are tools to format the codebase automatically based on formatting rules to enforce a consistent coding style. They can behave like linters in term of formatting rules, but can not do anything to help with code quality rules. They can also be integrated into workflows with linters to do both formating and linting in one step.

Prettier is the best code formatter ever existed; it supports many languages like Javascript, JSX, HTML, CSS, Markdown, etc; integrates with all major text editors; easy to start and has a huge ecosystem of plugins.

StandardJS is a JavaScript style guide, linter, and formatter. It has three core features: no configuration needed, automatically format code and catch style issues & programmer errors early. Adopting standard style means ranking the importance of code clarity and community conventions higher than personal style. It is available both as an npm package and text editor plugins.

Package Managers

Packages are pieces of code that you can share and reuse like basic components, libraries or frameworks. These packages are versioned and installed based on semantic versioning, your applications can use these packages as dependencies, and each package may or may not depend on other packages.

Package managers are tools that help you manage packages as dependencies and might also provide a global package registry. They work based on manifest files that keep track application metadata and needed dependencies, lock files to offer deterministic installs.

In the Node ecosystem, dependencies get placed within a node_modules directory in your project. The install process starts with resolving dependencies by making requests to the registry and recursively looking up each dependency, then fetching the package tarballs if needed, and finally linking everything together.

Npm is the most popular JavaScript package manager; it consists a website to discover packages, a CLI to interact with packages in terminal, and a global registry to share both private and pubic packages.

Yarn is a JavaScript package manager released by Facebook in 2016, compatible with the npm registry; it focuses on building a CLI client that is super fast, secure, and deterministic.

Bower is a package manager optimized for frontend; it can manage components that contain HTML, CSS, JavaScript, fonts or even image files. It keeps track packages in bower.json and puts installed packages in bower_components folder. It was created at the time npm only supported node packages, now fading away when both npm and yarn can support both node and browser packages with little help from module bundlers like Webpack or Browserify.

Task Runners

There are many repetitive tasks, sometimes also mundane, painful, or time-consuming, need to be automated like concatenating files, minification, compilation, unit testing, linting, etc.

Task runners are tools to orchestrate those kinds of tasks in an efficient way, the workflow made easy with the ecosystem of plugins to automate all kinds of tasks you can imagine, the speed can be optimized behind the scene to leverage multi-threads, and can be used as standalone or integrated into a more complicated pipeline.

Grunt is an open-source JavaScript task runner created in 2012, known for highly customizable extensive configuration, available as a command-line tool, has more than 5000 plugins.

Gulp is a free JavaScript task runner, known for flexible code over configuration, faster than Grunt, available as a command-line tool, has more than 4000 plugins.

Grunt and Gulp are two generic task runners with huge plugins ecosystem; other less capable alternatives are Broccoli.js and Brunch. If you think the above task runners are overkill, you can make other tools like bash scripts, npm scripts or webpack to behave like task runners with trade-offs of implementing many tasks yourself.

Module Bundlers

Module bundlers are tools that bundle JavaScript and non-JavScript modules into a bundle -- web apps to run on browsers, backend apps to run on Node.js, or npm packages. Most of them can bundle JavaScript files into a single concatenated file or multiple files, but they are hugely different in the capability of bundling non-JavaScript files like css, json, png, xml, etc.

Module bundlers can be considered as limited task runners and built tools as well, like focusing on automating frontend tasks and generate a website only.

Webpack is the most popular and powerful module bundler for JavaScript applications (both frontend and backend), can load many kinds of files, has extensive plugin ecosystem, often used to pack web applications.

Rollup is a JavaScript module bundler, famous for tree shaking, often used to bundle JavaScript libraries.

Parcel is a zero-configuration module bundler, only used for web applications, has support for many common transforms and transpilers built-in out of the box.

Browserify is a tool for bundling Node packages for the browser, happens to work for browser-based apps pretending to be Node packages.

Build Tools

Build tools are tools that can generate production build artifacts - web apps, server-only apps, libraries - through an automated process including multiple tasks like compiling, packaging, testing, linking, etc.

With the above definition in mind; those specialized task runners (Gulp, Grunt, etc.) and module bundlers (Webpack, Rollup, Parcel, etc.) are technically build tools for JavaScript. Each has tried to solve a specific problem in its way, you can achieve similar results with different tools, but often it's best to use them together to complement each other.

Make is a famous general-purpose build tool, allows you to write separate tasks for various purposes. Even though Make is mostly used with C projects, it's not tied to C in any way; you can use Make to build JavaScript projects as well.

Backpack is a minimalistic build tool, lets you create modern Node.js apps and services with zero configuration; handles all the file-watching, live-reloading, transpiling, and bundling, so you don't have to.

Transpilers

Transpilers in JavaScript are source-to-source compilers that transform source code in non-JavaScript languages (CoffeeScript, TypeScript, LiveScript, etc.) or in modern JavaScript versions (ES2015, ES2017, ESNext, etc.) to equivalent JavaScript source code that meets some conditions (browser compatible, minified, strict, etc.)

You might see people use compiler and transpiler interchangeably in JavaScript world. But keep in mind a source-to-source compiler translates between programming languages that operate at approximately the same level of abstraction; while a traditional compiler translates from a higher-level programming language to a lower-level programming language like C to assembler or Java to bytecode.

Babel is the most dominant JavaScript transpiler, a toolchain that is mainly used to convert ECMAScript 2015+ code into a backward-compatible version of JavaScript in current and older browsers or environments. Babel can transform syntax, polyfill features that are missing in your target environment, transform source code, and many more.

Debuggers

Debuggers are tools that allow you to inspect running code in Node.js, in browser, or both; They often support pausing execution, stepping through function calls manually, inspecting variables, profiling memory allocations, and CPU usage, viewing execution logs, etc.

Chrome DevTools is a set of web developer tools built directly into the Google Chrome browser; it allows you to debug CSS, prototype CSS, debug JavaScript, analyze load performance and many more.

node-inspector is a Node.js debugger based on Blink Developer Tools, already deprecated because Node.js already provides a built-in DevTools-based debugger.

Visual Studio Code also has built-in debugging support for the Node.js runtime and can debug JavaScript, TypeScript, and any other language that gets transpiled to JavaScript.

Module Loaders

Modular programming has many benefits; when you write modular JavaScript applications, you usually end up having one file per module.

In the old day, modules in JavaScript were implemented via libraries like CommonJS Modules (CJS), Asynchronous Module Definition (AMD), and Universal Module Definition (UMD). Since ES2015, JavaScript had built-in modules called ECMAScript Modules (ESM).

Besides the above popular JavaScript modules, we also consider other JavaScript module formats like System.register or global; and non-JavaScript modules like JSON modules, CSS modules, or Web Assembly.

Module loaders are libraries that can handle loading modules using the above formats for further processing or executing, they may be different in terms of synchronous or asynchronous loading, static or dynamic loading.

RequireJS is a JavaScript file and module loader, optimized for in-browser use, can also be used in Node, implements AMD.

SystemJS is a dynamic module loader that can load all kinds of JavaScript modules and many non-JavaScript modules, used in both browser and Node.

ES Module Loader has been implemented in most browsers, available in Node.js via --experimental-modules flag.

Above module loaders are built for browsers and Node. Don't be confused with module loaders used in build-tools, they are plugins used to load JavaScript source in multiple formats and non-JavaScript static assets for processing and transformation. In the end, they'll output module formats that browser or Node can understand.

Node Process Managers

Node process managers are tools to manage your Node applications at run time; they provide high availability, automatic restart, file watcher, runtime performance and resource consumption insights, and clustering.

Forever is a simple command-line interface tool to ensure that a script runs continuously forever, simple interface makes it ideal for running smaller deployments of Node.js apps and scripts.

PM2 is a production process manager for Node.js applications that has a built-in load balancer, enables you to keep applications alive forever, reloads them without downtime, helps you to manage application logging, monitoring, and clustering.

StrongLoop Process Manager (Strong-PM) is a production process manager for Node.js applications with built-in load balancing, monitoring, and multi-host deployment. Includes a CLI to build, package, and deploy Node.js applications to a local or remote system.

SystemD is the default process manager on modern Linux distributions, which makes it simple to run a Node application as a service.

Scaffolding Tools

Scaffolding tools allow you to generate projects automatically, also help with upgrading an existing project that has already been generated with the tool, keep it up to date with new best practices, enforce standards, allow for rapidly getting started on new projects and streamlines the maintenance of existing projects.

Yeoman is a generic scaffolding system, language-agnostic, allows the creation of any kind of app in any language, every decision is made by generators which are basically plugins in the Yeoman environment, can be integrated seamlessly with build-tools and package managers to streamline your workflows.

Monorepo Systems

Monorepo is an architecture of organizing source codes of multiple applications, services, and libraries into a single repository.

Monorepo systems in JavaScript are tools to manage JavaScript projects with multiple packages, useful for code sharing; making changes across many repositories; testing across repositories. Projects like Babel, React, Angular,
Ember, Meteor, Jest, and many other open-source projects develop all of their packages within a single repository.

Lerna is a tool that optimizes the workflow around managing multi-package
repositories with git and npm/yarn, link any cross-dependencies, shares common dependencies across packages, share common commands across packages.

Yarn workspaces allows you to setup multiple packages in such a way that you only need to run yarn install once to install all of them in a single pass.

Conclusions

Phew, what a long post! Don't be overwhelmed or intimidated, just use this post as a reference list of available tools in JavaScript tooling ecosystem.

These tools could have started in one category and then expanding into multiple categories, the boundary is sometimes very blurred. Emerging technologies approach the problem from different angles, sometimes they build on top of other tools, and at times they can be used together.

There is no right or wrong silver bullet solution. The combination of tools you use can be completely up to you depending on how much effort you can put into configuring and setting up. You can achieve the same results with different tools and settings.

It's best for beginners to use starter projects with recommended configurations especially in production builds. Configuring a production-ready build system requires a steep learning curve, very time consuming, and best suited for experienced developers.

Start small, keep the system simple, expand gradually, and stop when you’re happy.


This post was originally published at hoangbkit.com

JavaScript Tutorial in 2020 - Learn JavaScript to Build Your Web Apps

JavaScript Tutorial in 2020 - Learn JavaScript to Build Your Web Apps

Free JavaScript Tutorial for Beginners in 2020: Get Started with JavaScript and learn the JavaScript basics to build your own web apps!

Free JavaScript Tutorial for Beginners in 2020: Get Started with JavaScript and learn the JavaScript basics to build your own web apps!

Limited Offer! Join the Full JavaScript Course at 90% off: http://bit.ly/2q22e9l

Modern JavaScript from the beginning - all the way the way up to JS expert level! THE must-have JavaScript resource.

JavaScript is THE most important programming language you need to learn as a web developer - and with this course, you make sure that you will not miss a single thing you have to know as a JavaScript developer!

This is the most comprehensive and modern course you can find on JavaScript - it's based on all my JavaScript knowledge AND teaching experience. It's both a complete guide, starting with the core basics of the language, as well as an extensive reference of the JavaScript language and environment, ensuring that both newcomers as well as experienced JavaScript developers get a lot out of this course!

It's a huge course because it's packed with important knowledge and helpful content. From the core basics, over advanced concepts and JavaScript specialties, all the way up to expert topics like performance optimization and testing - this course has it all. My goal was to create your go-to resource for the JavaScript language, which you can not just use for learning it but also as a resource you can come back to and look up important topics.

The course is based on my experience as a long-term JavaScript developer as well as a teacher with more than 1,000,000 students on Udemy as well as on my YouTube channel Academind. It's packed with examples, demos, projects, assignments, quizzes and of course videos - all with the goal of giving you the best possible way of learning JavaScript.

What's in the course?

This course is obviously packed with content - I therefore strongly recommend that you check out the full course curriculum to get a clear idea of all the topics covered in the course. In general, here's what you'll find in the course:

  • Modern JavaScript from the start: The JavaScript syntax changed over time - in this course, you'll learn the latest syntax from the start (you'll also learn about the old one though, so that you can work in ANY JS project)
  • ALL the Basics: Variables, constants, functions, how scripts are loaded etc
  • Arrays & Objects: We'll explore these very important data structures in great detail
  • Control Structures: Understand how to run code conditionally and in loops
  • A look behind the Scenes: How JavaScript engines work behind the scenes and what that means for us
  • Deep dives into Core Concepts: ALL the special things about JavaScript function, different syntaxes
  • Working with the DOM: How to manipulate web pages dynamically via JavaScript (including deep dives and different use-cases)
  • Events in JavaScript: Learn how to listen to a broad variety of events (e.g. drag & drop) and execute appropriate code
  • Classes & Object-oriented Programming: Learn how to work with classes, prototypes, the "this" keyword, constructor functions and much more
  • Asynchronous and Synchronous Programming: We'll explore callbacks, promises, async/ await and other important tools and language features to execute code correctly
  • Http Requests: Learn how to send Http requests via JavaScript
  • Tooling, Optimizations & Browser Support: Code splitting, producing small code and ensuring that scripts work in all browsers - this matters and hence is covered in great detail
  • Libraries & Frameworks: Learn about libraries like Axios or frameworks like React.js - why they matter and how to use them
  • Node.js: Whilst focusing on the browser-side for the majority of the course (because the syntax is the same), we'll also have a dedicated section on Node.js to learn all about that JS host environment
  • Security & Performance Optimizations: Of course security matters, so does performance - no surprise that both is covered in the course!
  • Automated Testing: Testing manually is hard work and can be unreliable - in this course you'll also get an introduction into automated testing

What you'll learn:

  • JavaScript from scratch - beginner to advanced
  • All core features and concepts you need to know in modern JavaScript development
  • Everything you need to become a JavaScript expert and apply for JavaScript jobs
  • Project-driven learning with plenty of examples
  • All about variables, functions, objects and arrays
  • Object-oriented programming
  • Deep dives into prototypes, JavaScript engines & how it works behind the scenes
  • Manipulating web pages (= the DOM) with JavaScript
  • Event handling, asynchronous coding and Http requests
  • Meta-programming, performance optimization, memory leak busting
  • Testing, security and deployment
  • And so much more!