Install C++ Build Tools for Windows using npm

Install C++ Build Tools for Windows using npm

Install C++ Build Tools for Windows using npm

Windows-Build-Tools

On Windows? Want to compile native Node modules? Install the build tools with this one-liner. Start PowerShell as Administrator and run:

npm install --global windows-build-tools 

Or, if you are using Yarn:

yarn global add windows-build-tools 

After installation, npm will automatically execute this module, which downloads and installs Visual C++ Build Tools, provided free of charge for most users by Microsoft (as part of Visual Studio Community, please consult the license to determine whether or not you're eligible). These tools are required to compile popular native modules. If not already installed, it will also install Python 2.7, configuring your machine and npm appropriately.

[Windows Vista / 7 only] requires .NET Framework 4.5.1 (Currently not installed automatically by this package)

Both installations are conflict-free, meaning that they do not mess with existing installations of Visual Studio, C++ Build Tools, or Python. If you see anything that indicates otherwise, please file a bug.


Visual Studio 2017 vs Visual Studio 2015

This module is capable of installing either the build tools from Visual Studio 2017 or Visual Studio 2015.

By default, this tool will install the 2017 build tools. To change that, run this script with the --vs2015 parameter.


Usage
npm [--python-mirror=''] [--proxy=''] [--debug] [--strict-ssl] [--resume] [--sockets=5] [--vcc-build-tools-parameters=''] [--vs2015] [--dry-run-only] install --global windows-build-tools 

Optional arguments:

  • --offline-installers: Path to a folder with already downloaded installers. See
  • --python-mirror: Use a given mirror to download Python (like --python_mirror=https://npm.taobao.org/mirrors/python/). You can alternatively set a PYTHON_MIRROR environment variable.
  • --proxy: Use a given proxy. You can alternatively set a PROXY environment variable.
  • --debug: Be extra verbose in the logger output. Equal to setting the environment variable DEBUG to *.
  • --strict-ssl: Enables "Strict SSL" mode. Defaults to false.
  • --resume: By default, windows-build-tools will resume aborted downloads. Set to false to disable.
  • --sockets: Specifies the number of http sockets to use at once (this controls concurrency). Defaults to infinity.
  • --vcc-build-tools-parameters: Specifies additional parameters for the Visual C++ Build Tools 2015. See below for more detailed usage instructions.
  • --silent: The script will not output any information.
  • --vs2015: Install the Visual Studio 2015 Build Tools instead of the Visual Studio 2017 ones.
  • --dry-run-only: Don't actually do anything, just print what the script would have done.
  • --include-arm64-tools: Include the optional Visual Studio components required to build binaries for ARM64 Windows. Only available with the 2017 and newer build tools and Node.js v12 and up.

Supplying Parameters to the VCC Build Tools

You can pass additional parameters directly to the VCC Build Tools installer. This tool does not check if the parameters make sense - passing incorrect parameters might break the whole installation.

Supply parameters to windows-build-tools as a JSON array. Here's quick example (note the double quotes):

npm --vcc-build-tools-parameters='[""--allWorkloads""]' install --global windows-build-tools 


Visual Studio 2015 Parameters

If you run windows-build-tools with --vs2015, these parameters are available:

  • /AdminFile: Specifies the installation control file.
  • /CreateAdminFile: Specifies the location to create a control file that can then be used
  • /CustomInstallPath: Set Custom install location.
  • /ForceRestart: Always restart the system after installation.
  • /Full: Install all product features.
  • /InstallSelectableItems: <item1;item2;...;itemN> Choose which selectable item(s) to be installed. -selectable item to be installed, just pass in this switch without any value.
  • /Layout: Create a copy of the media in specified folder.
  • /NoRefresh: Prevent setup checking for updates from the internet.
  • /NoRestart: Do not restart during or after installation.
  • /NoWeb: Prevent setup downloading from the internet.
  • /Passive: Display progress but do not wait for user input.
  • /ProductKey: <25-character product key> Set custom product key (no dashes).
  • /PromptRestart: Prompt the user before restarting the system.
  • /Repair: Repair the product.
  • /Uninstall: Uninstall the product.
  • /Uninstall /Force: Uninstall the product and features shared with other products.


Visual Studio 2017 Parameters

The available parameters are documented here.


Offline Installation

By default, windows-build-tools will download the latest installers from Microsoft each time it's installed. Alternatively, you can prepare a folder that contains installers. They need to have their original names:

  • Visual Studio Build Tools: vs_BuildTools.exe or BuildTools_Full.exe
  • Python: python-2.7.15.amd64.msi or python-2.7.15.msi

Then, run windows-build-tools with the --offline-installers argument:

npm install -g windows-build-tools --offline-installers="C:\Users\John\installers"



Support & Help

This package currently only handles the most common use case, none of the edge cases. If you encounter errors, we'd greatly appreciate error reports (and even pull requests). This is currently tested on Windows 10.


Node versions

  • windows-build-tools 4.0 and up require at least Node v8.
  • windows-build-tools 3.0 and up require at least Node v6.
  • windows-build-tools 1.0 and up require at least Node v4.


Where is Python installed?

It's saved under %USERPROFILE%\.windows-build-tools\python27.


Installing as a Non-Administrator

windows-build-tools works best if installed from an account with administrative rights. However, thanks to @brucejo75, the following steps can be taken to install to a different user account:

  1. From your non-admin account (e.g. <Me>) run cmd.exe as administrator.
  2. Set the following environment variables in the new command shell:
set APPDATA=C:\Users\<Me>\AppData\Roaming npm config set prefix C:\Users\<Me>\AppData\Roaming\npm set USERNAME=<Me> set USERPROFILE=C:\Users\<Me> 

Ensure that the variables passed match your location of npm's roaming data and the location of user profiles on your machine. For <me>, substitute the name of the account you want to install windows-build-tools for. For more information, see the npm config set prefix description here.

  1. Run npm install -g windows-build-tools

Examples of Modules Supported

In theory, windows-build-tools supports all pure C++ addons for Node.js (and virtually everything else that requires a native compiler toolchain to be installed on your machine).

To ensure that that's true, we take a fresh Windows 10 installation, add windows-build-tools, and ensure that the most popular native Node addons compile from source. Those are: node-sass, bcrypt, sqlite3, serialport, websocket, deasync, grpc, canvas, sharp, hiredis, leveldown, nodegit, zqm, ffi, libxmljs, iconv, ref, sleep, microtime, couchbase, bignum, kerberos, and ursa.


License & Credits

The Python installation was made possible by Ali Hajimirza, who kindly wrestled with Python's MSIs until they surrendered. For details regarding the license agreements applicable to Python, see History and License 2.x and 3.x.

Use of Microsoft software is subject to the terms of the corresponding license agreements. For details regarding the license agreements applicable to Visual Studio products, refer to their License Directory page. (See also this discussion for the gist of it.)

Copyright (C) 2018 Felix Rieseberg. Licensed MIT. For more details, please see LICENSE. This license applies to this package only, not to its dependencies or the 3rd party software that it installs.


Top 7 Most Popular Node.js Frameworks You Should Know

Top 7 Most Popular Node.js Frameworks You Should Know

Node.js is an open-source, cross-platform, runtime environment that allows developers to run JavaScript outside of a browser. In this post, you'll see top 7 of the most popular Node frameworks at this point in time (ranked from high to low by GitHub stars).

Node.js is an open-source, cross-platform, runtime environment that allows developers to run JavaScript outside of a browser.

One of the main advantages of Node is that it enables developers to use JavaScript on both the front-end and the back-end of an application. This not only makes the source code of any app cleaner and more consistent, but it significantly speeds up app development too, as developers only need to use one language.

Node is fast, scalable, and easy to get started with. Its default package manager is npm, which means it also sports the largest ecosystem of open-source libraries. Node is used by companies such as NASA, Uber, Netflix, and Walmart.

But Node doesn't come alone. It comes with a plethora of frameworks. A Node framework can be pictured as the external scaffolding that you can build your app in. These frameworks are built on top of Node and extend the technology's functionality, mostly by making apps easier to prototype and develop, while also making them faster and more scalable.

Below are 7of the most popular Node frameworks at this point in time (ranked from high to low by GitHub stars).

Express

With over 43,000 GitHub stars, Express is the most popular Node framework. It brands itself as a fast, unopinionated, and minimalist framework. Express acts as middleware: it helps set up and configure routes to send and receive requests between the front-end and the database of an app.

Express provides lightweight, powerful tools for HTTP servers. It's a great framework for single-page apps, websites, hybrids, or public HTTP APIs. It supports over fourteen different template engines, so developers aren't forced into any specific ORM.

Meteor

Meteor is a full-stack JavaScript platform. It allows developers to build real-time web apps, i.e. apps where code changes are pushed to all browsers and devices in real-time. Additionally, servers send data over the wire, instead of HTML. The client renders the data.

The project has over 41,000 GitHub stars and is built to power large projects. Meteor is used by companies such as Mazda, Honeywell, Qualcomm, and IKEA. It has excellent documentation and a strong community behind it.

Koa

Koa is built by the same team that built Express. It uses ES6 methods that allow developers to work without callbacks. Developers also have more control over error-handling. Koa has no middleware within its core, which means that developers have more control over configuration, but which means that traditional Node middleware (e.g. req, res, next) won't work with Koa.

Koa already has over 26,000 GitHub stars. The Express developers built Koa because they wanted a lighter framework that was more expressive and more robust than Express. You can find out more about the differences between Koa and Express here.

Sails

Sails is a real-time, MVC framework for Node that's built on Express. It supports auto-generated REST APIs and comes with an easy WebSocket integration.

The project has over 20,000 stars on GitHub and is compatible with almost all databases (MySQL, MongoDB, PostgreSQL, Redis). It's also compatible with most front-end technologies (Angular, iOS, Android, React, and even Windows Phone).

Nest

Nest has over 15,000 GitHub stars. It uses progressive JavaScript and is built with TypeScript, which means it comes with strong typing. It combines elements of object-oriented programming, functional programming, and functional reactive programming.

Nest is packaged in such a way it serves as a complete development kit for writing enterprise-level apps. The framework uses Express, but is compatible with a wide range of other libraries.

LoopBack

LoopBack is a framework that allows developers to quickly create REST APIs. It has an easy-to-use CLI wizard and allows developers to create models either on their schema or dynamically. It also has a built-in API explorer.

LoopBack has over 12,000 GitHub stars and is used by companies such as GoDaddy, Symantec, and the Bank of America. It's compatible with many REST services and a wide variety of databases (MongoDB, Oracle, MySQL, PostgreSQL).

Hapi

Similar to Express, hapi serves data by intermediating between server-side and client-side. As such, it's can serve as a substitute for Express. Hapi allows developers to focus on writing reusable app logic in a modular and prescriptive fashion.

The project has over 11,000 GitHub stars. It has built-in support for input validation, caching, authentication, and more. Hapi was originally developed to handle all of Walmart's mobile traffic during Black Friday.

How to Write Python C Extension Modules using the Python API

How to Write Python C Extension Modules using the Python API

There are several ways in which you can extend the functionality of Python. One of these is to write your Python module in C or C++. In this tutorial, you’ll discover how to use the Python API to write Python C extension modules.

You’ll learn how to:

  • Invoke C functions from within Python
  • Pass arguments from Python to C and parse them accordingly
  • Raise exceptions from C code and create custom Python exceptions in C
  • Define global constants in C and make them accessible in Python
  • Test, package, and distribute your Python C extension module

Table of Contents

  • Extending Your Python Program
  • Writing a Python Interface in C
    • Understanding fputs()
    • Writing the C Function for fputs()
    • Wrapping fputs()
    • Writing the Init Function
    • Putting It All Together
  • Packaging Your Python C Extension Module
    • Building Your Module
    • Running Your Module
  • Raising Exceptions
    • Raising Exceptions From C Code
    • Raising Custom Exceptions
  • Defining Constants
  • Testing Your Module
  • Considering Alternatives
  • Conclusion
Extending Your Python Program

One of the lesser-known yet incredibly powerful features of Python is its ability to call functions and libraries defined in compiled languages such as C or C++. This allows you to extend the capabilities of your program beyond what Python’s built-in features have to offer.

There are many languages you could choose from to extend the functionality of Python. So, why should you use C? Here are a few reasons why you might decide to build a Python C extension module:

  1. To implement new built-in object types: It’s possible to write a Python class in C, and then instantiate and extend that class from Python itself. There can be many reasons for doing this, but more often than not, performance is primarily what drives developers to turn to C. Such a situation is rare, but it’s good to know the extent to which Python can be extended.

  2. To call C library functions and system calls: Many programming languages provide interfaces to the most commonly used system calls. Still, there may be other lesser-used system calls that are only accessible through C. The os module in Python is one example.

This is not an exhaustive list, but it gives you the gist of what can be done when extending Python using C or any other language.

To write Python modules in C, you’ll need to use the Python API, which defines the various functions, macros, and variables that allow the Python interpreter to call your C code. All of these tools and more are collectively bundled in the Python.h header file.

Writing a Python Interface in C

In this tutorial, you’ll write a small wrapper for a C library function, which you’ll then invoke from within Python. Implementing a wrapper yourself will give you a better idea about when and how to use C to extend your Python module.

Understanding fputs()

fputs() is the C library function that you’ll be wrapping:

int fputs(const char *, FILE *)

This function takes two arguments:

  1. const char * is an array of characters.
  2. FILE * is a file stream pointer.

fputs() writes the character array to the file specified by the file stream and returns a non-negative value. If the operation is successful, then this value will denote the number of bytes written to the file. If there’s an error, then it returns EOF. You can read more about this C library function and its other variants in the manual page entry.

Writing the C Function for fputs()

This is a basic C program that uses fputs() to write a string to a file stream:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main() {
    FILE *fp = fopen("write.txt", "w");
    fputs("Real Python!", fp);
    fclose(fp);
    return 1;
}

This snippet of code can be summarized as follows:

  1. Open the file write.txt.
  2. Write the string "Real Python!" to the file.

Note: The C code in this article should build on most systems. It has been tested on GCC without using any special flags.

In the following section, you’ll write a wrapper for this C function.

Wrapping fputs()

It might seem a little weird to see the full code before an explanation of how it works. However, taking a moment to inspect the final product will supplement your understanding in the following sections. The code block below shows the final wrapped version of your C code:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;


    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {

        return NULL;

    }


    FILE *fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);

    fclose(fp);


    return PyLong_FromLong(bytes_copied);

}

This code snippet references three object structures:

  1. PyObject
  2. PyArg_ParseTuple()
  3. PyLong_FromLong()

These are used for data type definition for the Python language. You’ll go through each of them now.

PyObject

PyObject is an object structure that you use to define object types for Python. All Python objects share a small number of fields that are defined using the PyObject structure. All other object types are extensions of this type.

PyObject tells the Python interpreter to treat a pointer to an object as an object. For instance, setting the return type of the above function as PyObject defines the common fields that are required by the Python interpreter in order to recognize this as a valid Python type.

Take another look at the first few lines of your C code:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;


    /* Snip */

In line 2, you declare the argument types you wish to receive from your Python code:

  1. char *str is the string you want to write to the file stream.
  2. char *filename is the name of the file to write to.

PyArg_ParseTuple()

PyArg_ParseTuple() parses the arguments you’ll receive from your Python program into local variables:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;


    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {

        return NULL;

    }


    /* Snip */

If you look at line 6, then you’ll see that PyArg_ParseTuple() takes the following arguments:

  • args are of type PyObject.

  • "ss" is the format specifier that specifies the data type of the arguments to parse. (You can check out the official documentation for a complete reference.)

  • &str and &filename are pointers to local variables to which the parsed values will be assigned.

PyArg_ParseTuple() evaluates to false on failure. If it fails, then the function will return NULL and not proceed any further.

fputs()

As you’ve seen before, fputs() takes two arguments, one of which is the FILE * object. Since you can’t parse a Python textIOwrapper object using the Python API in C, you’ll have to use a workaround:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;


    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {

        return NULL;

    }


    FILE *fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);

    fclose(fp);


    return PyLong_FromLong(bytes_copied);

}

Here’s a breakdown of what this code does:

  • In line 10, you’re passing the name of the file that you’ll use to create a FILE * object and pass it on to the function.
  • In line 11, you call fputs() with the following arguments:
    • str is the string you want to write to the file.
    • fp is the FILE * object you defined in line 10.

You then store the return value of fputs() in bytes_copied. This integer variable will be returned to the fputs() invocation within the Python interpreter.

PyLong_FromLong(bytes_copied)

PyLong_FromLong() returns a PyLongObject, which represents an integer object in Python. You can find it at the very end of your C code:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;


    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {

        return NULL;

    }


    FILE *fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);

    fclose(fp);


    return PyLong_FromLong(bytes_copied);

}

Line 14 generates a PyLongObject for bytes_copied, the variable to be returned when the function is invoked in Python. You must return a PyObject* from your Python C extension module back to the Python interpreter.

Writing the Init Function

You’ve written the code that makes up the core functionality of your Python C extension module. However, there are still a few extra functions that are necessary to get your module up and running. You’ll need to write definitions of your module and the methods it contains, like so:

static PyMethodDef FputsMethods[] = {
    {"fputs", method_fputs, METH_VARARGS, "Python interface for fputs C library function"},
    {NULL, NULL, 0, NULL}
};


static struct PyModuleDef fputsmodule = {
    PyModuleDef_HEAD_INIT,
    "fputs",
    "Python interface for the fputs C library function",
    -1,
    FputsMethods
};

These functions include meta information about your module that will be used by the Python interpreter. Let’s go through each of the structs above to see how they work.

PyMethodDef

In order to call the methods defined in your module, you’ll need to tell the Python interpreter about them first. To do this, you can use PyMethodDef. This is a structure with 4 members representing a single method in your module.

Ideally, there will be more than one method in your Python C extension module that you want to be callable from the Python interpreter. This is why you need to define an array of PyMethodDef structs:

static PyMethodDef FputsMethods[] = {
    {"fputs", method_fputs, METH_VARARGS, "Python interface for fputs C library function"},
    {NULL, NULL, 0, NULL}
};

Each individual member of the struct holds the following info:

  • "fputs" is the name the user would write to invoke this particular function.

  • method_fputs is the name of the C function to invoke.

  • METH_VARARGS is a flag that tells the interpreter that the function will accept two arguments of type PyObject*:

    1. self is the module object.
    2. args is a tuple containing the actual arguments to your function. As explained previously, these arguments are unpacked using PyArg_ParseTuple().
  • The final string is a value to represent the method docstring.

PyModuleDef

Just as PyMethodDef holds information about the methods in your Python C extension module, the PyModuleDef struct holds information about your module itself. It is not an array of structures, but rather a single structure that’s used for module definition:

static struct PyModuleDef fputsmodule = {
    PyModuleDef_HEAD_INIT,
    "fputs",
    "Python interface for the fputs C library function",
    -1,
    FputsMethods
};

There are a total of 9 members in this struct, but not all of them are required. In the code block above, you initialize the following five:

  1. PyModuleDef_HEAD_INIT is a member of type PyModuleDef_Base, which is advised to have just this one value.

  2. "fputs" is the name of your Python C extension module.

  3. The string is the value that represents your module docstring. You can use NULL to have no docstring, or you can specify a docstring by passing a const char * as shown in the snippet above. It is of type Py_ssize_t. You can also use PyDoc_STRVAR() to define a docstring for your module.

  4. -1 is the amount of memory needed to store your program state. It’s helpful when your module is used in multiple sub-interpreters, and it can have the following values:

    • A negative value indicates that this module doesn’t have support for sub-interpreters.
    • A non-negative value enables the re-initialization of your module. It also specifies the memory requirement of your module to be allocated on each sub-interpreter session.
  5. FputsMethods is the reference to your method table. This is the array of PyMethodDef structs you defined earlier.

For more information, check out the official Python documentation on PyModuleDef.

PyMODINIT_FUNC

Now that you’ve defined your Python C extension module and method structures, it’s time to put them to use. When a Python program imports your module for the first time, it will call PyInit_fputs():

PyMODINIT_FUNC PyInit_fputs(void) {
    return PyModule_Create(&fputsmodule);
}

PyMODINIT_FUNC does 3 things implicitly when stated as the function return type:

  1. It implicitly sets the return type of the function as PyObject*.
  2. It declares any special linkages.
  3. It declares the function as extern “C.” In case you’re using C++, it tells the C++ compiler not to do name-mangling on the symbols.

PyModule_Create() will return a new module object of type PyObject *. For the argument, you’ll pass the address of the method structure that you’ve already defined previously, fputsmodule.

Note: In Python 3, your init function must return a PyObject * type. However, if you’re using Python 2, then PyMODINIT_FUNC declares the function return type as void.

Putting It All Together

Now that you’ve written the necessary parts of your Python C extension module, let’s take a step back to see how it all fits together. The following diagram shows the components of your module and how they interact with the Python interpreter:

When you import your Python C extension module, PyInit_fputs() is the first method to be invoked. However, before a reference is returned to the Python interpreter, the function makes a subsequent call to PyModule_Create(). This will initialize the structures PyModuleDef and PyMethodDef, which hold meta information about your module. It makes sense to have them ready since you’ll make use of them in your init function.

Once this is complete, a reference to the module object is finally returned to the Python interpreter. The following diagram shows the internal flow of your module:

The module object returned by PyModule_Create() has a reference to the module structure PyModuleDef, which in turn has a reference to the method table PyMethodDef. When you call a method defined in your Python C extension module, the Python interpreter uses the module object and all of the references it carries to execute the specific method. (While this isn’t exactly how the Python interpreter handles things under the hood, it’ll give you an idea of how it works.)

Similarly, you can access various other methods and properties of your module, such as the module docstring or the method docstring. These are defined inside their respective structures.

Now you have an idea of what happens when you call fputs() from the Python interpreter. The interpreter uses your module object as well as the module and method references to invoke the method. Finally, let’s take a look at how the interpreter handles the actual execution of your Python C extension module:

Once method_fputs() is invoked, the program executes the following steps:

  1. Parse the arguments you passed from the Python interpreter with PyArg_ParseTuple()
  2. Pass these arguments to fputs(), the C library function that forms the crux of your module
  3. Use PyLong_FromLong to return the value from fputs()

To see these same steps in code, take a look at method_fputs() again:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;


    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {

        return NULL;

    }


    FILE *fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);

    fclose(fp);


    return PyLong_FromLong(bytes_copied);

}

To recap, your method will parse the arguments passed to your module, send them on to fputs(), and return the results.

Packaging Your Python C Extension Module

Before you can import your new module, you first need to build it. You can do this by using the Python package distutils.

You’ll need a file called setup.py to install your application. For this tutorial, you’ll be focusing on the part specific to the Python C extension module.

A minimal setup.py file for your module should look like this:

from distutils.core import setup, Extension

def main():
    setup(name="fputs",
          version="1.0.0",
          description="Python interface for the fputs C library function",
          author="<your name>",
          author_email="[email protected].com",
          ext_modules=[Extension("fputs", ["fputsmodule.c"])])

if __name__ == "__main__":
    main()

The code block above shows the standard arguments that are passed to setup(). Take a closer look at the last positional argument, ext_modules. This takes a list of objects of the Extensions class. An object of the Extensions class describes a single C or C++ extension module in a setup script. Here, you pass two keyword arguments to its constructor, namely:

  • name is the name of the module.
  • [filename] is a list of paths to files with the source code, relative to the setup script.
[Remove ads](/account/join/)

Building Your Module

Now that you have your setup.py file, you can use it to build your Python C extension module. It’s strongly advised that you use a virtual environment to avoid conflicts with your Python environment.

Navigate to the directory containing setup.py and run the following command:

$ python3 setup.py install

This command will compile and install your Python C extension module in the current directory. If there are any errors or warnings, then your program will throw them now. Make sure you fix these before you try to import your module.

By default, the Python interpreter uses clang for compiling the C code. If you want to use gcc or any other C compiler for the job, then you need to set the CC environment variable accordingly, either inside the setup script or directly on the command line. For instance, you can tell the Python interpreter to use gcc to compile and build your module this way:

$ CC=gcc python3 setup.py install

However, the Python interpreter will automatically fall back to gcc if clang is not available.

Running Your Module

Now that everything is in place, it’s time to see your module in action! Once it’s successfully built, fire up the interpreter to test run your Python C extension module:

>>> import fputs
>>> fputs.__doc__
'Python interface for the fputs C library function'
>>> fputs.__name__
'fputs'
>>> # Write to an empty file named `write.txt`
>>> fputs.fputs("Real Python!", "write.txt")
13
>>> with open("write.txt", "r") as f:
>>>     print(f.read())
'Real Python!'

Your function performs as expected! You pass a string "Real Python!" and a file to write this string to, write.txt. The call to fputs() returns the number of bytes written to the file. You can verify this by printing the contents of the file.

Also recall how you passed certain arguments to the PyModuleDef and PyMethodDef structures. You can see from this output that Python has used these structures to assign things like the function name and docstring.

With that, you have a basic version of your module ready, but there’s a lot more that you can do! You can improve your module by adding things like custom exceptions and constants.

Raising Exceptions

Python exceptions are very different from C++ exceptions. If you want to raise Python exceptions from your C extension module, then you can use the Python API to do so. Some of the functions provided by the Python API for exception raising are as follows:

You can use any of these to raise an exception. However, which to use and when depends entirely on your requirements. The Python API has all the standard exceptions pre-defined as PyObject types.

Raising Exceptions From C Code

While you can’t raise exceptions in C, the Python API will allow you to raise exceptions from your Python C extension module. Let’s test this functionality by adding PyErr_SetString() to your code. This will raise an exception whenever the length of the string to be written is less than 10 characters:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;


    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &fd)) {

        return NULL;

    }


    if (strlen(str) < 10) {

        PyErr_SetString(PyExc_ValueError, "String length must be greater than 10");

        return NULL;

    }


    fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);

    fclose(fp);


    return PyLong_FromLong(bytes_copied);

}

Here, you check the length of the input string immediately after you parse the arguments and before you call fputs(). If the string passed by the user is shorter than 10 characters, then your program will raise a ValueError with a custom message. The program execution stops as soon as the exception occurs.

Note how method_fputs() returns NULL after raising the exception. This is because whenever you raise an exception using PyErr_*(), it automatically sets an internal entry in the exception table and returns it. The calling function is not required to subsequently set the entry again. For this reason, the calling function returns a value that indicates failure, usually NULL or -1. (This should also explain why there was a need to return NULL when you parse arguments in method_fputs() using PyArg_ParseTuple().)

Raising Custom Exceptions

You can also raise custom exceptions in your Python C extension module. However, things are a bit different. Previously, in PyMODINIT_FUNC, you were simply returning the instance returned by PyModule_Create and calling it a day. But for your custom exception to be accessible by the user of your module, you need to add your custom exception to your module instance before you return it:

static PyObject *StringTooShortError = NULL;

PyMODINIT_FUNC PyInit_fputs(void) {
    /* Assign module value */
    PyObject *module = PyModule_Create(&fputsmodule);

    /* Initialize new exception object */
    StringTooShortError = PyErr_NewException("fputs.StringTooShortError", NULL, NULL);

    /* Add exception object to your module */
    PyModule_AddObject(module, "StringTooShortError", StringTooShortError);

    return module;
}

As before, you start off by creating a module object. Then you create a new exception object using PyErr_NewException. This takes a string of the form module.classname as the name of the exception class that you wish to create. Choose something descriptive to make it easier for the user to interpret what has actually gone wrong.

Next, you add this to your module object using PyModule_AddObject. This takes your module object, the name of the new object being added, and the custom exception object itself as arguments. Finally, you return your module object.

Now that you’ve defined a custom exception for your module to raise, you need to update method_fputs() so that it raises the appropriate exception:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;


    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &fd)) {

        return NULL;

    }


    if (strlen(str) < 10) {

        /* Passing custom exception */

        PyErr_SetString(StringTooShortError, "String length must be greater than 10");

        return NULL;

    }


    fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);

    fclose(fp);


    return PyLong_FromLong(bytes_copied);

}

After building the module with the new changes, you can test that your custom exception is working as expected by trying to write a string that is less than 10 characters in length:

>>> import fputs
>>> # Custom exception
>>> fputs.fputs("RP!", fp.fileno())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
fputs.StringTooShortError: String length must be greater than 10

When you try to write a string with fewer than 10 characters, your custom exception is raised with a message explaining what went wrong.

Defining Constants

There are cases where you’ll want to use or define constants in your Python C extension module. This is quite similar to how you defined custom exceptions in the previous section. You can define a new constant and add it to your module instance using PyModule_AddIntConstant():

PyMODINIT_FUNC PyInit_fputs(void) {
    /* Assign module value */
    PyObject *module = PyModule_Create(&fputsmodule);

    /* Add int constant by name */
    PyModule_AddIntConstant(module, "FPUTS_FLAG", 64);

    /* Define int macro */
    #define FPUTS_MACRO 256

    /* Add macro to module */
    PyModule_AddIntMacro(module, FPUTS_MACRO);

    return module;
}

This Python API function takes the following arguments:

  • The instance of your module
  • The name of the constant
  • The value of the constant

You can do the same for macros using PyModule_AddIntMacro():

PyMODINIT_FUNC PyInit_fputs(void) {
    /* Assign module value */
    PyObject *module = PyModule_Create(&fputsmodule);

    /* Add int constant by name */
    PyModule_AddIntConstant(module, "FPUTS_FLAG", 64);

    /* Define int macro */
    #define FPUTS_MACRO 256

    /* Add macro to module */
    PyModule_AddIntMacro(module, FPUTS_MACRO);

    return module;
}

This function takes the following arguments:

  • The instance of your module
  • The name of the macro that has already been defined

Note: If you want to add string constants or macros to your module, then you can use PyModule_AddStringConstant() and PyModule_AddStringMacro(), respectively.

Open up the Python interpreter to see if your constants and macros are working as expected:

>>> import fputs
>>> # Constants
>>> fputs.FPUTS_FLAG
64
>>> fputs.FPUTS_MACRO
256

Here, you can see that the constants are accessible from within the Python interpreter.

[Remove ads](/account/join/)
Testing Your Module

You can test your Python C extension module just as you would any other Python module. This can be demonstrated by writing a small test function for pytest:

import fputs

def test_copy_data():
    content_to_copy = "Real Python!"
    bytes_copied = fputs.fputs(content_to_copy, 'test_write.txt')

    with open('test_write.txt', 'r') as f:
        content_copied = f.read()

    assert content_copied == content_to_copy

In the test script above, you use fputs.fputs() to write the string "Real Python!" to an empty file named test_write.txt. Then, you read in the contents of this file and use an assert statement to compare it to what you had originally written.

You can run this test suite to make sure your module is working as expected:

$ pytest -q
test_fputs.py                                                 [100%]
1 passed in 0.03 seconds
Considering Alternatives

In this tutorial, you’ve built an interface for a C library function to understand how to write Python C extension modules. However, there are times when all you need to do is invoke some system calls or a few C library functions, and you want to avoid the overhead of writing two different languages. In these cases, you can use Python libraries such as ctypes or cffi.

These are Foreign Function libraries for Python that provide access to C library functions and data types. Though the community itself is divided as to which library is best, both have their benefits and drawbacks. In other words, either would make a good choice for any given project, but there are a few things to keep in mind when you need to decide between the two:

  • The ctypes library comes included in the Python standard library. This is very important if you want to avoid external dependencies. It allows you to write wrappers for other languages in Python.

  • The cffi library is not yet included in the standard library. This might be a dealbreaker for your particular project. In general, it’s more Pythonic in nature, but it doesn’t handle preprocessing for you.

For more information on these libraries, check out Extending Python With C Libraries and the “ctypes” Module and Interfacing Python and C: The CFFI Module.

Note: Apart from ctypes and cffi, there are various other tools available. For instance, you can also use swig and boost::Py.

Conclusion

In this tutorial, you’ve learned how to write a Python interface in the C programming language using the Python API. You wrote a Python wrapper for the fputs() C library function. You also added custom exceptions and constants to your module before building and testing it.

The Python API provides a host of features for writing complex Python interfaces in the C programming language. At the same time, libraries such as cffi or ctypes can lower the amount of overhead involved in writing Python C extension modules. Make sure you weigh all the factors before making a decision!

How to Install Node.js with npm on Debian 10

How to Install Node.js with npm on Debian 10

Install Node.js with npm on Debian 10 In this tutorial, we are going to learn how to install Node.js with npm on Debian 10

Table of Contents

Install Node.js with npm on Debian 10

In this tutorial, we are going to learn how to install Node.js with npm on Debian 10. Node.js is the opensource JavaScript Run-time environment for server-side execution of JavaScript code. Node.js built on Chrome’s V8 JavaScript engine so it can be used to build different types of server-side applications.

Where npm stands for Node Package Manager which is the default package manager for Node.js. npm is the world’s largest software registry for Node.js packages with thousands of packages available.

In this tutorial we will install Node.js in following two ways:

  1. Install Node.js and npm using Debian repository
  2. Install Node.js and npm using nvm
  3. Install Node.js from the NodeSource repository.

1. Install Node.js and npm using Debian repository

First, Update Debian apt package manager index by running the following command.

sudo apt update

Install Node.js from Debian global repository by typing

sudo apt install node

Confirm the installation of Node.js by typing

node --version

Install npm by running following command

sudo apt install npm

Confirm the installation of npm by typing

npm --version

2. Install Node.js and npm using nvm

NVM stands for Node Version Manager which is used to manage multiple Node.js versions. If you want to install or uninstall different versions of Node.js then NVM is there for you.

First, we will install NVM (Node Package Manager) on your system. So download the NVM installation script running the following command.

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash

Check nvm version and confirm installation typing

node --version

Now install Node.js by using the following command.

nvm install node

Verify Node.js installation by typing

node --version

The output should be:

Output

v10.14.0

You can install multiple versions of Node.js. To do so type the following:

nvm install 8.14
nvm install --lts
nvm install 11.3

To list all the versions installed run following command.

nvm ls

You can change the current default version of Node.js by using the following command.

nvm use 8.14

To uninstall a Node.js version type following command

nvm uninstall 11.14

Install Node.js from NodeSource Repository

NodeSource company provides enterprise-grade node support also maintains the repository containing the latest version of Node.js.

To enable the NodeSource repository on your system run following command.

curl -sL https://deb.nodesource.com/setup_10.x | sudo bash -

NOTE: The latest LTS version of Node.js is 10.x if you want to install 8.x version then just replace setup_10.x with setup_8.x

Now install Node.js and npm package typing.

sudo apt install nodejs

Verify installation of Node.js and npm running following command

node --version
npm --version

Install Development Tools

Now install some packages needed for development by running following command

sudo apt install gcc g++ make

Uninstall Node.js and npm

Uninstall Node.js use following command

sudo apt remove nodejs npm
sudo apt autoremove

To uninstall node.js version using nvm type following command

nvm uninstall 10.14

Conclusion

You have successfully learned how to install Node.js with npm on Debian 10. If you have any queries don’t forget to comment below.