Building Native Node.js Add-ons on z/OS

Here I will show how you can build a native C++ addon for Node.js on z/OS.

Warning! It is not easy but it works.

IBM has ported Node.js to z/OS so you can run Node.js server applications on z/OS. You can download it from the IBM website: IBM SDK for Node.js — z/OS — Overview.

We see how to install it together with the provided C/C++ compiler so you will be able to build native extensions with node-gyp. We will start by building a “Hello World” native addon on Mac and then building the same extension on z/OS.

The latest ported version of Node.js on z/OS is 6.17.0. We will use the latest version on macOS. The native addon code is the same on both platforms.

Why you should be interested in Node.js? Some of the reasons are mention in https://www.monterail.com/blog/nodejs-development-enterprises. Why you would you want to run it on z/OS? z/OS delivers a secure, scalable, available infrastructure for on-premises. If a lot of your data resides on z/OS, the performance of your Node.js applications will benefit from it.

When do you need to write a native add-on for Node.js on z/OS? There are two main reasons. First one is to optimize the performance sensitive code such as CPU intensive calculations. Second, is to interoperate with a native mainframe application that you have on z/OS.

Building native Node.js addon on macOS

Let’s start with a platform that is used by many developers. You can skip this section if you are already familiar with this process on macOS, Linux, or Windows.

We will build an example addon from https://github.com/nodejs/node-addon-examples/tree/master/1_hello_world/nan.

This example used nan — C++-based abstraction between Node and direct V8 APIs. It is very simple and it prints just:

world

We need to get the source code first:

git clone https://github.com/nodejs/node-addon-examples.git

Then change directory to the example:

cd node-addon-examples/1_hello_world/nan/

If you do not have Node.js yet installed, you can use Homebrew to install it:

brew install node

You need to have a C++ compiler on your system. You need to install the Command Line Tools via Xcode:

xcode-select --install

Next step is to compile the native addon:

npm install

You should get a similar output to this:

> hello_world@0.0.0 install /Users/petrplavjanik/workspace/node-addon-examples/1_hello_world/nan
> node-gyp rebuild
CXX(target) Release/obj.target/hello/hello.o
  SOLINK_MODULE(target) Release/hello.node
audited 2 packages in 3.677s
found 0 vulnerabilities

You are ready to issue node hello.js and get world.

Before we will be able to do the same thing on z/OS, we need to install Node.js on z/OS with all prerequisites.

Installing Node.js on z/OS

You need to have all the prerequisites listed at https://www.ibm.com/support/knowledgecenter/en/SSTRRS_6.0.0/com.ibm.nodejs.zos.v6.doc/install.htm:

z/OS V2R2 with PTF UI46658, z/OS V2R3, or higher
Integrated Cryptographic Services Facility (ICSF) must be enabled on systems where SDK for Node.js is run
Python 2.7.13 or higher that is provided by Rocket Software. Note that Python 3.x is not compatible with node-gyp that is used for building native add-ons
GNU Make 4.1 or higher that is provided by Rocket Software
Bash 4.3 or higher that is provided by Rocket Software and is required to install Python 2.7
Gzip 1.9 or higher that is provided by Rocket Software and is required to unpackage other software provided by Rocket Software

The Rocket Software open-source downloads are available at https://my.rocketsoftware.com/RocketCommunity#/downloads.

If you are using the z/OS trial version of Node.js from https://developer.ibm.com/node/sdk/ztp/ instead of the SMP/E installed version then you need Perl 5.24.0 or higher that is provided by Rocket Software as well.

Since there is a lot of steps to do and they are described in multiple guides on different places, we will cover all of them here.

We will use Zowe CLI to simplify the interaction with z/OS. You can install Zowe CLI by following the instructions on https://zowe.github.io/docs-site/latest/user-guide/cli-installcli.html#installing-zowe-cli-from-an-online-registry including setting up the z/OSMF profile. If you do not have Zowe CLI or z/OSMF then you can do all steps manually.

Downloading Rocket Software open-source

We will download all the archives first before installing them. Download all of them to the same directory.

Go to https://my.rocketsoftware.com/RocketCommunity#/downloads
Log in or sign up to the Rocket Community
Search for make and download make-4.1_b0002.160426.tar.gz
Search for python and download python-2017–04–12-py27.tar.gz
Search for perl and download perl-5.24.0_b007.180202.tar.gz
Search for bash and download bash-4.3_b018.170518.tar.gz
Search for gzip and download gzip-1.9-edc_b002.180703.tar

Create zFS Filesystem for Rocket Software open-source and Node.js

You will need about 4 GB of space. It is recommended to allocate new zFS filesystem for it. You can use the following job template and replace ${jobcard} and ${prefix} variables with your job card and the prefix for the data sets with the zFS file system.

${jobcard}
//DEFINE EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DEFINE CLUSTER (NAME(${prefix}.ZFS) -
LINEAR -
TRACKS(80000 80000) -
SHAREOPTIONS(3))
//CREATE EXEC PGM=IOEFSUTL,REGION=0M,COND=(0,LT),
// PARM=('format -aggregate ${prefix}.ZFS')
//SYSPRINT DD SYSOUT=*

JCL that creates a zFS filesystem on z/OS

You can submit this JCL easily using Zowe CLI:

curl -O https://gist.githubusercontent.com/plavjanik/fbf4fbf4318a1b7a44ac0381a1a39717/raw/19cc8feabd757549e719bb6c1e49b1dc9f6d9986/create_zfs.jcl

Edit the file in your favorite text editor and replace the variables. Then you can submit it:

zowe zos-jobs submit local-file create_zfs.jcl

Give Zowe CLI a try! It allows you to interact with z/OS jobs, data sets, z/OS UNIX files and much more from your workstation:

Submitting a job via Zowe CLI

Recorded by plavjanikasciinema.org

After the job completes, you need to mount the filesystem. Login to your z/OS system via ssh or Putty. Issue following command (you need to replace ${rocket_home} and ${prefix} variables):

export ROCKET_HOME=${rocket_home}
mkdir ${ROCKET_HOME}
/usr/sbin/mount -f ${prefix}.ZFS ${ROCKET_HOME}
cd ${ROCKET_HOME}
mkdir download

Now, you need to get back to your workstation terminal and copy the packages to zFS:

zowe zos-files upload file-to-uss --binary gzip-1.9-edc_b002.180703.tar ${ROCKET_HOME}/download/gzip-1.9-edc_b002.180703.tar
zowe zos-files upload file-to-uss --binary perl-5.24.0_b007.180202.tar.gz ${ROCKET_HOME}/download/perl-5.24.0_b007.180202.tar.gz
zowe zos-files upload file-to-uss --binary python-2017-04-12-py27.tar.gz ${ROCKET_HOME}/download/python-2017-04-12-py27.tar.gz
zowe zos-files upload file-to-uss --binary make-4.1_b0002.160426.tar.gz ${ROCKET_HOME}/download/make-4.1_b0002.160426.tar.gz

Go back to the z/OS shell session and unpack these packages:

tar -C ${ROCKET_HOME} -xovf ${ROCKET_HOME}/download/gzip-1.9-edc_b002.180703.tar
${ROCKET_HOME}/bin/gunzip --stdout {ROCKET_HOME}/download/perl-5.24.0_b007.180202.tar.gz | tar xoUXf -
${ROCKET_HOME}/bin/gunzip --stdout ${ROCKET_HOME}/download/python-2017-04-12-py27.tar.gz | tar xoUXf -
${ROCKET_HOME}/bin/gunzip --stdout ${ROCKET_HOME}/download/bash-4.3_b018.170518.tar.gz | tar xoUXf -
${ROCKET_HOME}/bin/gunzip --stdout ${ROCKET_HOME}/download/make-4.1_b0002.160426.tar.gz | tar xoUXf -
${ROCKET_HOME}/bin/gunzip --stdout ${ROCKET_HOME}/download/python-2017-04-12-py27.tar.gz | tar xoUXf -

Now is the time to assign correct access rights:

chmod -R 755 ${ROCKET_HOME}/bin/*
find ${ROCKET_HOME}/lib -type f -exec chmod 644 {} \\;
find ${ROCKET_HOME}/lib -type f -name '*.so' -exec chmod 755 {} \\;

We need to set the environment variables before these tools can be read. They require auto conversion to be on.

These are the recommended settings:

Recommended .profile settings

# Runtime options require by Rocket open-source:
export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
export _BPXK_AUTOCVT=ON
export _TAG_REDIR_ERR=txt
export _TAG_REDIR_IN=txt
export _TAG_REDIR_OUT=txt
export _BPX_SHAREAS=YES
export _BPX_SPAWN_SCRIPT=YES
export ROCKET_HOME=${rocket_home}
# Perl:
export PERL5LIB=$ROCKET_HOME/lib/perl5:$PERL5LIB
export LIBPATH=$ROCKET_HOME/lib/perl5/5.24.0/os390/CORE:$LIBPATH
# Python 2.7:
export PY_RELEASE_NAME=python-2017-04-12
export PY_RELEASE_TYPE=py27
export PY_RELEASE_DIR=$ROCKET_HOME/${PY_RELEASE_NAME}-${PY_RELEASE_TYPE}
export PYTHON_ENV=python27
export PYTHON_HOME=$PY_RELEASE_DIR/$PYTHON_ENV
export PATH=$PYTHON_HOME/bin:$PATH
export LIBPATH=$PYTHON_HOME/lib:$LIBPATH
export FFI_LIB=$PYTHON_HOME/lib/ffi
export TERMINFO=$PYTHON_HOME/share/terminfo
export PKG_CONFIG_PATH=$PYTHON_HOME/lib/pkgconfig:$PYTHON_HOME/share/pkgconfig
export CURL_CA_BUNDLE=$PYTHON_HOME/etc/ssl/cacert.pem
# Rocket Ported Tools:
export PATH=$ROCKET_HOME/bin:$PATH
export MANPATH=$ROCKET_HOME/man/:$MANPATH
export INFOPATH=$ROCKET_HOME/info/:$INFOPATH
export CHARSETALIASDIR=$ROCKET_HOME/lib

You need to put them to a directory in zFS:

curl -O https://gist.githubusercontent.com/plavjanik/94d859826f5064691266406c1d431037/raw/249265ad8b71a4aef1e3140322ec613789024ea4/rocket_activate.sh
# edit rocket_activate.sh and replace ${rocket_home}
zowe zos-files upload file-to-uss rocket_activate.sh ${rocket_home}/rocket_activate.sh

Then you can make it active in the z/OS shell by:

. ${ROCKET_HOME}/rocket_activate.sh

You can put this line to your ~/.profile if you want to have it activate for each session.

The last step is to complete the installation of Python:

export BASH_PREFIX=${ROCKET_HOME}/bin
export PERL_PREFIX=${ROCKET_HOME}/bin
export RELEASE_NAME=python-2017-04-12
export RELEASE_TYPE=py27
export RELEASE_DIR=${ROCKET_HOME}/${RELEASE_NAME}-${RELEASE_TYPE}
export PKGS_BASE=${RELEASE_DIR}/pkgs
cd ${PY_RELEASE_DIR}/python27; bin/install_all_packages

If everything is fine, delete the packages:

rm -Rf download

Download and install Node.js for z/OS

Now we are ready to install Node.js for z/OS. Download the PAX file from https://developer.ibm.com/node/sdk/ztp/.

Upload the PAX file ibm-6.17.0.0-trial-node-v6.17.0-os390-s390x.pax.Z to zFS.

zowe zos-files upload file-to-uss --binary ibm-6.17.0.0-trial-node-v6.17.0-os390-s390x.pax.Z ${rocket_home}/ibm-6.17.0.0-trial-node-v6.17.0-os390-s390x.pax.Z

In the z/OS shell:

cd ${ROCKET_HOME}
pax -rf ibm-6.17.0.0-trial-node-v6.17.0-os390-s390x.pax.Z -x pax

Node.js installed to run applications but we need to make the Node.js SDK C/C++ compiler working. This can be done by unpack.pl script but that script needs manual updates in order to be working properly.

You can download it from the following link: https://pastebin.com/MJDDeF1u

curl https://pastebin.com/raw/MJDDeF1u > unpack2.pl
zowe zos-files upload file-to-uss unpack2.pl ${rocket_home}/node-v6.17.0-os390-s390x/unpack2.pl

On z/OS:

cd ${ROCKET_HOME}/node-v6.17.0-os390-s390x
ln -s bin/njsc++ bin/g++
chmod a+x *.pl
COMPILER_INSTALL_ACCOUNT_NUMBER=${account_number} COMPILER_INSTALL_MSGCLASS=X COMPILER_INSTALL_HLQ=${rocket_hlq}.CC COMPILER_INSTALL_VOLSER=${volser} perl ./unpack2.pl

For example:

COMPILER_INSTALL_ACCOUNT_NUMBER=129300000 COMPILER_INSTALL_MSGCLASS=X COMPILER_INSTALL_HLQ=MFAAS.NODE.CC COMPILER_INSTALL_VOLSER=TSU001 perl ./unpack2.pl

This will take a while and in the end, you should see the following message:

Hello C++ World!
<INFO> installation complete!

If you do not see the message, check two jobs that were submitted by this script.The C/C++ environment comes pre-configured when using npm to build native add-ons. However, if you invoke node-gyp directly to build native code, the following C/C++ compiler environment variables need to be set:

export CC=eb2as.sh
export CXX=eb2as.sh
export LINK=eb2as.sh
export _C89_CCMODE=1
export _CC_CCMODE=1
export _CXX_CCMODE=1

You should add the following lines to the activation shell script rocket_activate.sh:

# Node.js:
NODE_VERSION="v6.17.0"
export NODE_HOME=${ROCKET_HOME}/node-${NODE_VERSION}-os390-s390x
export PATH=${NODE_HOME}/bin:$PATH
export LIBPATH=${NODE_HOME}/lib:${LIBPATH}
export MANPATH=${NODE_HOME}/man/:${MANPATH}
export CC=eb2as.sh
export CXX=eb2as.sh
export LINK=eb2as.sh
export _C89_CCMODE=1
export _CC_CCMODE=1
export _CXX_CCMODE=1

And activate it:

. ${ROCKET_HOME}/rocket_activate.sh
mkdir -p 1_hello_world/nan

Verify that is working by issuing node --version which should return v6.17.0.

Building the native add-on on z/OS

Go to the directory with the sample add-on on your workstation:

cd node-addon-examples/1_hello_world/nan/

Copy the source files to the z/OS:

zowe zos-files upload file-to-uss binding.gyp ${rocket_home}/1_hello_world/nan/binding.gyp
zowe zos-files upload file-to-uss hello.cc ${rocket_home}/1_hello_world/nan/hello.cc
zowe zos-files upload file-to-uss hello.js ${rocket_home}/1_hello_world/nan/hello.js
zowe zos-files upload file-to-uss package.json ${rocket_home}/1_hello_world/nan/package.json

We are ready to build it on z/OS:

cd ${ROCKET_HOME}/1_hello_world/nan
npm install --nodedir=${NODE_HOME}

The option --nodedir=${NODE_HOME} is important because it makes node-gyp to use C++ headers from Node.js on z/OS instead of downloading not-z/OS version from the internet.

Then you can issue node hello.js.

The output should be world!

In the future, we will work with VSAM datasets from Node.js application and we will create simple API service on z/OS that reads and stores data in a VSAM dataset vsam.js and Express. My mate Dan Kelosky will show you how to use NestJS with TypeScript on z/OS.

You are ready to take advantages of the Node.js and npm ecosystem: https://www.monkeyuser.com/2017/npm-delivery/ 😊