Tech2 etc

Tech2 etc

1660741784

React Navbar Tutorial - Responsive Navbar React JS

Advance React JS Responsive Navbar

Over the years, navigation bars, or navbars, have become a staple feature in building websites. They really come in handy for navigating across a website.

We will Create react navbar using advanced react js methods. In our previous tutorial, we have created a responsive navbar using basic react js. But in this tutorial we are going to use advance functionalities for creating this navbar using react js.

In this tutorial, we’ll go over the basics of building navbars in React. We’ll build a navbar that’s fixed to the top of the page, and the user will be able to switch between webpages like you can on a dashboard, for instance. We’ll then put all our newfound skills into practice by building a live sample app.

Our sample app is an animal gallery that we’ll use to view some pictures of different types of animals after we click the animal’s name on the navigation bar.

Installation

Let’s get right into the code. To create a React app, make sure you have Node.js installed on your computer. If you haven’t built a React app before, you can check to see if you have Node.js installed by typing the following into your terminal:

node -v

If not, just go to the Node.js website to download the latest version.

Once that’s done, we can get started with our React app by running this command:

npx create-react-app nav-bar                                             

Then, we navigate into our project folder on the terminal:

cd nav-bar

Setting up the React Router library

Using the React Router library in our application allows us to navigate between different pages or components in React, and actually makes these changes to the URL of each page or component. In fact, it can also handle more complex functionalities, like passing data between pages through queries and nested routes — but that’s beyond the scope of this article.

For this example, we’ll only need to install the React Router library to help us switch views of the animals in our app when we click on the nav names on the links.

npm install react-router-dom

Check out the tutorial for complete project.

Resources:

Code Sandbox: https://codesandbox.io/

Level 1(Previous Tutorial): https://youtu.be/23BHwAFIZmk

Source Code1: https://www.buymeacoffee.com/tech2etc/e/85299

Source Code2: https://ko-fi.com/s/7f02c9c3fd

Template & Setup(VS Code): https://github.com/tech2etc/React_Navbar_2

 

#react-native #react #nestjs #dom #htmlcss #web-development #dev 

React Navbar Tutorial - Responsive Navbar React JS
Gordon  Taylor

Gordon Taylor

1660737060

Web-push: Web Push Library for Node.js

web-push

Why

Web push requires that push messages triggered from a backend be done via the Web Push Protocol and if you want to send data with your push message, you must also encrypt that data according to the Message Encryption for Web Push spec.

This module makes it easy to send messages and will also handle legacy support for browsers relying on GCM for message sending / delivery.

Install

Installation is simple, just install via npm.

npm install web-push --save

Usage

The common use case for this library is an application server using a GCM API key and VAPID keys.

const webpush = require('web-push');

// VAPID keys should be generated only once.
const vapidKeys = webpush.generateVAPIDKeys();

webpush.setGCMAPIKey('<Your GCM API Key Here>');
webpush.setVapidDetails(
  'mailto:example@yourdomain.org',
  vapidKeys.publicKey,
  vapidKeys.privateKey
);

// This is the same output of calling JSON.stringify on a PushSubscription
const pushSubscription = {
  endpoint: '.....',
  keys: {
    auth: '.....',
    p256dh: '.....'
  }
};

webpush.sendNotification(pushSubscription, 'Your Push Payload Text');

Using VAPID Key for applicationServerKey

When subscribing to push messages, you'll need to pass your VAPID key, which you can do like so:

registration.pushManager.subscribe({
  userVisibleOnly: true,
  applicationServerKey: '<Your Public Key from generateVAPIDKeys()>'
});

Command Line

You can install web-push globally and use it for sending notifications and / or generating VAPID keys.

Install like so:

npm install web-push -g

Then you can run the following commands:

Usage:

  web-push send-notification --endpoint=<url> [--key=<browser key>] [--auth=<auth secret>] [--payload=<message>] [--encoding=<aesgcm | aes128gcm>] [--ttl=<seconds>] [--vapid-subject=<vapid subject>] [--vapid-pubkey=<public key url base64>] [--vapid-pvtkey=<private key url base64>] [--proxy=<http proxy uri>] [--gcm-api-key=<api key>]

  web-push generate-vapid-keys [--json]

Example of send notification:

  > web-push generate-vapid-keys --json
  > {"publicKey":"BGtkbcjrO12YMoDuq2sCQeHlu47uPx3SHTgFKZFYiBW8Qr0D9vgyZSZPdw6_4ZFEI9Snk1VEAj2qTYI1I1YxBXE","privateKey":"I0_d0vnesxbBSUmlDdOKibGo6vEXRO-Vu88QlSlm5j0"}

The subscription value:

  { 
    "endpoint": "https://fcm.googleapis.com/fcm/send/d61c5u920dw:APA91bEmnw8utjDYCqSRplFMVCzQMg9e5XxpYajvh37mv2QIlISdasBFLbFca9ZZ4Uqcya0ck-SP84YJUEnWsVr3mwYfaDB7vGtsDQuEpfDdcIqOX_wrCRkBW2NDWRZ9qUz9hSgtI3sY", 
    "expirationTime": null, 
    "keys": { 
      "p256dh": "BL7ELU24fJTAlH5Kyl8N6BDCac8u8li_U5PIwG963MOvdYs9s7LSzj8x_7v7RFdLZ9Eap50PiiyF5K0TDAis7t0", 
      "auth": "juarI8x__VnHvsOgfeAPHg" 
    } 
  }

The command example:

  web-push send-notification  \
  --endpoint=https://fcm.googleapis.com/fcm/send/d61c5u920dw:APA91bEmnw8utjDYCqSRplFMVCzQMg9e5XxpYajvh37mv2QIlISdasBFLbFca9ZZ4Uqcya0ck-SP84YJUEnWsVr3mwYfaDB7vGtsDQuEpfDdcIqOX_wrCRkBW2NDWRZ9qUz9hSgtI3sY \
  --key=BL7ELU24fJTAlH5Kyl8N6BDCac8u8li_U5PIwG963MOvdYs9s7LSzj8x_7v7RFdLZ9Eap50PiiyF5K0TDAis7t0 \
  --auth=juarI8x__VnHvsOgfeAPHg \
  --vapid-subject=mailto:example@qq.com \
  --vapid-pubkey=BGtkbcjrO12YMoDuq2sCQeHlu47uPx3SHTgFKZFYiBW8Qr0D9vgyZSZPdw6_4ZFEI9Snk1VEAj2qTYI1I1YxBXE \
  --vapid-pvtkey=I0_d0vnesxbBSUmlDdOKibGo6vEXRO-Vu88QlSlm5j0 \
  --payload=Hello

API Reference

sendNotification(pushSubscription, payload, options)

const pushSubscription = {
  endpoint: '< Push Subscription URL >',
  keys: {
    p256dh: '< User Public Encryption Key >',
    auth: '< User Auth Secret >'
  }
};

const payload = '< Push Payload String >';

const options = {
  gcmAPIKey: '< GCM API Key >',
  vapidDetails: {
    subject: '< \'mailto\' Address or URL >',
    publicKey: '< URL Safe Base64 Encoded Public Key >',
    privateKey: '< URL Safe Base64 Encoded Private Key >'
  },
  timeout: <Number>
  TTL: <Number>,
  headers: {
    '< header name >': '< header value >'
  },
  contentEncoding: '< Encoding type, e.g.: aesgcm or aes128gcm >',
  proxy: '< proxy server options >',
  agent: '< https.Agent instance >'
}

webpush.sendNotification(
  pushSubscription,
  payload,
  options
);

Note: sendNotification() you don't need to define a payload, and this method will work without a GCM API Key and / or VAPID keys if the push service supports it.

Input

Push Subscription

The first argument must be an object containing the details for a push subscription.

The expected format is the same output as JSON.stringify'ing a PushSubscription in the browser.

Payload

The payload is optional, but if set, will be the data sent with a push message.

This must be either a string or a node Buffer.

Note: In order to encrypt the payload, the pushSubscription must have a keys object with p256dh and auth values.

Options

Options is an optional argument that if defined should be an object containing any of the following values defined, although none of them are required.

  • gcmAPIKey can be a GCM API key to be used for this request and this request only. This overrides any API key set via setGCMAPIKey().
  • vapidDetails should be an object with subject, publicKey and

privateKey values defined. These values should follow the VAPID Spec.

  • timeout is a value in milliseconds that specifies the request's socket timeout. On timeout, the request will be destroyed and the promise will be rejected with a meaningful error. It's a common misconception that a socket timeout is the timeout to receive the full response. So if you have a socket timeout of 1 second, and a response comprised of 3 TCP packets, where each response packet takes 0.9 seconds to arrive, for a total response time of 2.7 seconds, then there will be no timeout. Once a socket 'timeout' triggers the request will be aborted by the library (by default undefined).
  • TTL is a value in seconds that describes how long a push message is retained by the push service (by default, four weeks).
  • headers is an object with all the extra headers you want to add to the request.
  • contentEncoding is the type of push encoding to use (e.g. 'aes128gcm', by default, or 'aesgcm').
  • proxy is the HttpsProxyAgent's constructor argument that may either be a string URI of the proxy server (eg. http://< hostname >:< port >) or an "options" object with more specific properties.
  • agent is the HTTPS Agent instance which will be used in the https.request method. If the proxy options defined, agent will be ignored!

Returns

A promise that resolves if the notification was sent successfully with details of the request, otherwise it rejects.

In both cases, resolving or rejecting, you'll be able to access the following values on the returned object or error.

  • statusCode, the status code of the response from the push service;
  • headers, the headers of the response from the push service;
  • body, the body of the response from the push service.

generateVAPIDKeys()

const vapidKeys = webpush.generateVAPIDKeys();

// Prints 2 URL Safe Base64 Encoded Strings
console.log(vapidKeys.publicKey, vapidKeys.privateKey);

Input

None.

Returns

Returns an object with publicKey and privateKey values which are URL Safe Base64 encoded strings.

Note: You should create these keys once, store them and use them for all future messages you send.


setGCMAPIKey(apiKey)

webpush.setGCMAPIKey('Your GCM API Key');

Input

This method expects the GCM API key that is linked to the gcm_sender_id in your web app manifest.

You can use a GCM API Key from the Google Developer Console or the Cloud Messaging tab under a Firebase Project.

Returns

None.


encrypt(userPublicKey, userAuth, payload, contentEncoding)

const pushSubscription = {
  endpoint: 'https://....',
  keys: {
    p256dh: '.....',
    auth: '.....'
  }
};
webPush.encrypt(
  pushSubscription.keys.p256dh,
  pushSubscription.keys.auth,
  'My Payload',
  'aes128gcm'
)
.then(encryptionDetails => {

});

Encrypts the payload according to the Message Encryption for Web Push standard.

(sendNotification will automatically encrypt the payload for you, so if you use sendNotification you don't need to worry about it).

Input

The encrypt() method expects the following input:

  • userPublicKey: the public key of the receiver (from the browser).
  • userAuth: the auth secret of the receiver (from the browser).
  • payload: the message to attach to the notification.
  • contentEncoding: the type of content encoding to use (e.g. aesgcm or aes128gcm).

Returns

This method returns an object with the following fields:

  • localPublicKey: The public key matched the private key used during encryption.
  • salt: A string representing the salt used to encrypt the payload.
  • cipherText: The encrypted payload as a Buffer.

getVapidHeaders(audience, subject, publicKey, privateKey, contentEncoding, expiration)

const parsedUrl = url.parse(subscription.endpoint);
const audience = parsedUrl.protocol + '//' +
  parsedUrl.hostname;

const vapidHeaders = vapidHelper.getVapidHeaders(
  audience,
  'mailto: example@web-push-node.org',
  vapidDetails.publicKey,
  vapidDetails.privateKey,
  'aes128gcm'
);

The getVapidHeaders() method will take in the values needed to create an Authorization and Crypto-Key header.

Input

The getVapidHeaders() method expects the following input:

  • audience: the origin of the push service.
  • subject: the mailto or URL for your application.
  • publicKey: the VAPID public key.
  • privateKey: the VAPID private key.
  • contentEncoding: the type of content encoding to use (e.g. aesgcm or aes128gcm).

Returns

This method returns an object with the following fields:

  • localPublicKey: The public key matched the private key used during encryption.
  • salt: A string representing the salt used to encrypt the payload.
  • cipherText: The encrypted payload as a Buffer.

generateRequestDetails(pushSubscription, payload, options)

const pushSubscription = {
  endpoint: '< Push Subscription URL >';
  keys: {
    p256dh: '< User Public Encryption Key >',
    auth: '< User Auth Secret >'
  }
};

const payload = '< Push Payload String >';

const options = {
  gcmAPIKey: '< GCM API Key >',
  vapidDetails: {
    subject: '< \'mailto\' Address or URL >',
    publicKey: '< URL Safe Base64 Encoded Public Key >',
    privateKey: '< URL Safe Base64 Encoded Private Key >',
  }
  TTL: <Number>,
  headers: {
    '< header name >': '< header value >'
  },
  contentEncoding: '< Encoding type, e.g.: aesgcm or aes128gcm >',
  proxy: '< proxy server options >'
}

try {
  const details = webpush.generateRequestDetails(
    pushSubscription,
    payload,
    options
  );
} catch (err) {
  console.error(err);
}

Note: When calling generateRequestDetails() the payload argument does not need to be defined, passing in null will return no body and exclude any unnecessary headers. Headers related to the GCM API Key and / or VAPID keys will be included if supplied and required.

Input

Push Subscription

The first argument must be an object containing the details for a push subscription.

The expected format is the same output as JSON.stringify'ing a PushSubscription in the browser.

Payload

The payload is optional, but if set, will be encrypted and a Buffer will be returned via the payload parameter.

This argument must be either a string or a node Buffer.

Note: In order to encrypt the payload, the pushSubscription must have a keys object with p256dh and auth values.

Options

Options is an optional argument that if defined should be an object containing any of the following values defined, although none of them are required.

  • gcmAPIKey can be a GCM API key to be used for this request and this request only. This overrides any API key set via setGCMAPIKey().
  • vapidDetails should be an object with subject, publicKey and

privateKey values defined. These values should follow the VAPID Spec.

  • TTL is a value in seconds that describes how long a push message is retained by the push service (by default, four weeks).
  • headers is an object with all the extra headers you want to add to the request.
  • contentEncoding is the type of push encoding to use (e.g. 'aesgcm', by default, or 'aes128gcm').
  • proxy is the HttpsProxyAgent's constructor argument that may either be a string URI of the proxy server (eg. http://< hostname >:< port >) or an "options" object with more specific properties.

Returns

An object containing all the details needed to make a network request, the object will contain:

  • endpoint, the URL to send the request to;
  • method, this will be 'POST';
  • headers, the headers to add to the request;
  • body, the body of the request (As a Node Buffer).

Browser Support

BrowserPush without PayloadPush with PayloadVAPIDNotes
Chrome✓ v42+✓ v50+✓ v52+In v51 and less, the `gcm_sender_id` is needed to get a push subscription.
Edge✓ v17+ (April 2018)✓ v17+ (April 2018)✓ v17+ (April 2018) 
Firefox✓ v44+✓ v44+✓ v46+ 
Opera✓ v39+ *✓ v39+ ** Opera supports push on Android but not on desktop. 

The `gcm_sender_id` is needed to get a push subscription.
Safari 
Samsung Internet Browser✓ v4.0.10-53+✓ v5.0.30-40+The `gcm_sender_id` is needed to get a push subscription.

Help

Service Worker Cookbook

The Service Worker Cookbook is full of Web Push examples using this library.

Running tests

Prerequisites:

To run tests:

npm test

Download Details:

Author: Web-push-libs
Source Code: https://github.com/web-push-libs/web-push 
License: View license

#javascript #web #node 

Web-push: Web Push Library for Node.js

Escher.jl: Composable Web UIs in Julia

Escher

Escher has been repurposed to be a metapackage around Interact.jl and other packages for web deployment (so far it includes Mux.jl but more things may be added as they become available). Refer to the Interact documentation (in particular the deployment section).

You can replace using Interact, Mux with using Escher, i.e.:

using Escher
ui = @manipulate for i in 1:100
    i
end
webio_serve(page("/", req -> ui))

Download Details:

Author: JuliaGizmos
Source Code: https://github.com/JuliaGizmos/Escher.jl 
License: View license

#julia #web 

Escher.jl: Composable Web UIs in Julia
Mike  Kozey

Mike Kozey

1660386360

A Library That Contains Widgets That Utilize Web Specific Code

touchkit_web

A library that contains widgets that utilize web specific code.

Installing

Use this package as a library

Depend on it

Run this command:

With Flutter:

 $ flutter pub add touchkit_web

This will add a line like this to your package's pubspec.yaml (and run an implicit flutter pub get):

dependencies:
  touchkit_web: ^0.0.1

Alternatively, your editor might support flutter pub get. Check the docs for your editor to learn more.

Import it

Now in your Dart code, you can use:

import 'package:touchkit_web/touchkit_web.dart';

Original article source at: https://pub.dev/packages/touchkit_web 

#flutter #dart #web 

A Library That Contains Widgets That Utilize Web Specific Code
Mike  Kozey

Mike Kozey

1660331580

The Web (JS & WASM) Platform Implementation Of Dargon2_flutter

dargon2_flutter_web

A plugin to handle Web argon2 password hashing bindings in Flutter application.

Uses the Argon2 implementation from hash-wasm

This is ONLY the web-specific platform interface. Please use the federated dargon2_flutter plugin for hashing functionality, which provides mobile & web support

Installing

Use this package as a library

Depend on it

Run this command:

With Flutter:

 $ flutter pub add dargon2_flutter_web

This will add a line like this to your package's pubspec.yaml (and run an implicit flutter pub get):

dependencies:
  dargon2_flutter_web: ^3.1.0

Alternatively, your editor might support flutter pub get. Check the docs for your editor to learn more.

Import it

Now in your Dart code, you can use:

import 'package:dargon2_flutter_web/dargon2_flutter_web.dart';

Download Details:

Author: Tmthecoder
Source Code: https://github.com/tmthecoder/dargon2 
License: MIT license

#flutter #dart #web #wasm 

The Web (JS & WASM) Platform Implementation Of Dargon2_flutter

Pgweb: Cross-platform Client for PostgreSQL Databases

pgweb

Web-based PostgreSQL database browser written in Go.    

Overview

Pgweb is a web-based database browser for PostgreSQL, written in Go and works on OSX, Linux and Windows machines. Main idea behind using Go for backend development is to utilize ability of the compiler to produce zero-dependency binaries for multiple platforms. Pgweb was created as an attempt to build very simple and portable application to work with local or remote PostgreSQL databases.

See application screenshots

Features

  • Cross-platform support OSX/Linux/Windows 32/64-bit
  • Simple installation (distributed as a single binary)
  • Zero dependencies
  • Works with PostgreSQL 9.1+
  • SSH Connections
  • Multiple database sessions
  • Simple database browser
  • Execute and analyze custom SQL queries
  • Table and query data export to CSV/JSON/XML
  • Query history
  • Server bookmarks

Visit WIKI for more details

Pgweb Pro

Pgweb Pro is the next major version of Pgweb and includes features like:

  • Table structure editing
  • Data editing (update row content)
  • Charting
  • History persistence
  • Multiple tabs

Please get in touch via: https://goo.gl/forms/euQOGWg5uPdPH70b2

Demo

Visit https://pgweb-demo.herokuapp.com to see pgweb in action.

Installation

Usage

Start server:

pgweb

You can also provide connection flags:

pgweb --host localhost --user myuser --db mydb

Connection URL scheme is also supported:

pgweb --url postgres://user:password@host:port/database?sslmode=[mode]
pgweb --url "postgres:///database?host=/absolute/path/to/unix/socket/dir"

Multiple database sessions

To enable multiple database sessions in pgweb, start the server with:

pgweb --sessions

Or set environment variable:

SESSIONS=1 pgweb

Deploy on Heroku

Heroku Deploy

Testing

Before running tests, make sure you have PostgreSQL server running on localhost:5432 interface. Also, you must have postgres user that could create new databases in your local environment. Pgweb server should not be running at the same time.

Execute test suite:

make test

If you're using Docker locally, you might also run pgweb test suite against all supported PostgreSQL version with a single command:

make test-all

Contribute

  • Fork this repository
  • Create a new feature branch for a new functionality or bugfix
  • Commit your changes
  • Execute test suite
  • Push your code and open a new pull request
  • Use issues for any questions
  • Check wiki for extra documentation

Download Details:

Author: sosedoff
Source Code: https://github.com/sosedoff/pgweb 
License: MIT license

#go #golang #web #crossplatform 

Pgweb: Cross-platform Client for PostgreSQL Databases
Robert  Jack

Robert Jack

1660202939

DeFi Exchange Development Services

Create your own DeFi crypto exchange with cutting edge features from the expert DeFi exchange development services company.

DeFi exchange platform development company

DeFi or open finance is one of the most talked about topics of the crypto market as it has captured a wide range of Fintech Industry and blockchain arena with large benefits. The system has disrupted the traditional finance procedures to an improvised decentralized platform that provides much needed solutions like services like quick loans, lending and borrowing etc. DeFi exchange development provides a variety of services like the ability to manage their own funds by the DeFi participants, easy earning by staking, yield farming and many more. One of the notable highlights of the DeFi crypto exchange platform is its superfast transaction speed and automated transactions with smart contracts DeFi exchange platform development company

As a pioneer Decentralised finance exchange development company, Shamla tech helps to create your own DeFi crypto exchange to bring together global traders by providing a permissionless and trustworthy defi platform.

Benefits of DeFi in the crypto exchange

  • Decentralized platform
  • Secure Transactions
  • Tokenization of Assets
  • Cost-Effective Trade
  • High Transparency
  • Tokens Security
  • Easy cross border Trade
  • Peer-to-Peer Transactions
  • Multi-currency Support.

Features of our DeFi cryptocurrency exchange development

No Intermediary

The lack of intermediary offers great trading experience with improved security and trustworthy peer to peer transactions

Smart Contracts

Smart contracts offer completely reliable and super fast trade with automated transactions also avoiding manual errors.

Top notch Security

The decentralised system upholds security and privacy to the data and funds eliminating the possibility of any hack or unauthorized activity.

Low fees

The transaction fees are cut down drastically without the influence of third parties assuring easy and cheaper cross border trade.

Liquidity

Defi for crypto exchange enhances the liquidity with robust automated liquidity protocols allowing trouble-free trading.

Control over funds

Defi based cryptocurrency exchanges facilitate total ownership to the buyer and seller over their funds and data.

Voting options

Usage of governance tokens in the platform helps build a frictionless financial framework with voting rights for the users.

Transparency

Decentralised exchanges are known for their transparency and this helps the users to avoid any disputes or fraud.

Robust solutions from Leading DeFi exchange development services company

shamlatech is a one stop solution for all the cryptocurrency exchange development script and DeFi exchange development solutions to deal with every sector. We offer popular and trending P2P cryptocurrency exchange script developed skillfully by our developers. Our Decentralised finance based solutions are popular in this field and are experts in offering custom and scalable solutions of the desired platform.
Read more: DeFi exchange development solutions

#Defiexchange

#cryptocurrency 

#Defiexchangedevelopment

#defidevelopment 

#defi 

#exchanges 

#web-development 

#crypto 

#Decentralized platform


 

DeFi Exchange Development Services
Nat  Grady

Nat Grady

1660187460

CRAN Task View: WebTechnologies

CRAN Task View: Web Technologies and Services

This Task View contains information about to use R and the world wide web together. The base version of R does not ship with many tools for interacting with the web. Thankfully, there are an increasingly large number of tools for interacting with the web. This task view focuses on packages for obtaining web-based data and information, frameworks for building web-based R applications, and online services that can be accessed from R. A list of available packages and functions is presented below, grouped by the type of activity. The rOpenSci Task View: Open Data provides further discussion of online data sources that can be accessed from R.

If you have any comments or suggestions for additions or improvements for this Task View, go to GitHub and submit an issue , or make some changes and submit a pull request . If you can’t contribute on GitHub, send Scott an email . If you have an issue with one of the packages discussed below, please contact the maintainer of that package. If you know of a web service, API, data source, or other online resource that is not yet supported by an R package, consider adding it to the package development to do list on GitHub .

Tools for Working with the Web from R

Core Tools For HTTP Requests

There are three main packages that should cover most use cases of interacting with the web from R. crul is an R6-based HTTP client that provides asynchronous HTTP requests, a pagination helper, HTTP mocking via webmockr, and request caching for unit tests via vcr. crul targets R developers more so than end users. httr provides more of a user facing client for HTTP requests and differentiates from the former package in that it provides support for OAuth. Note that you can pass in additional curl options when you instantiate R6 classes in crul, and the config parameter in httr. curl is a lower-level package that provides a closer interface between R and the libcurl C library , but is less user-friendly. curl underlies both crul and httr. curl may be useful for operations on web-based XML or to perform FTP operations (as crul and httr are focused primarily on HTTP). curl::curl() is an SSL-compatible replacement for base R’s url() and has support for http 2.0, SSL (https, ftps), gzip, deflate and more. For websites serving insecure HTTP (i.e. using the “http” not “https” prefix), most R functions can extract data directly, including read.table and read.csv; this also applies to functions in add-on packages such as jsonlite::fromJSON() and XML::parseXML. For more specific situations, the following resources may be useful:

  • RCurl is another low level client for libcurl. Of the two low-level curl clients, we recommend using curl. httpRequest is another low-level package for HTTP requests that implements the GET, POST and multipart POST verbs, but we do not recommend its use.
  • request provides a high-level package that is useful for developing other API client packages. httping provides simplified tools to ping and time HTTP requests, around httr calls. httpcache provides a mechanism for caching HTTP requests.
  • For dynamically generated webpages (i.e., those requiring user interaction to display results), RSelenium can be used to automate those interactions and extract page contents. It provides a set of bindings for the Selenium 2.0 webdriver using the JsonWireProtocol . It can also aid in automated application testing, load testing, and web scraping. seleniumPipes (GitHub ) provides a “pipe”-oriented interface to the same. An alternative to the former two packages is splashr that vouches to be a lightweight altnernative. rdom (not on CRAN) uses phantomjs to access a webpage’s Document Object Model (DOM).
  • For capturing static content of web pages postlightmercury is a client for the web service Mercury that turns web pages into structured and clean text.
  • Another, higher-level alternative package useful for webscraping is rvest, which is designed to work with magrittr to make it easy to express common web scraping tasks.
  • Many base R tools can be used to download web content, provided that the website does not use SSL (i.e., the URL does not have the “https” prefix). download.file() is a general purpose function that can be used to download a remote file. For SSL, the download() function in downloader wraps download.file(), and takes all the same arguments.
  • Tabular data sets (e.g., txt, csv, etc.) can be input using read.table(), read.csv(), and friends, again assuming that the files are not hosted via SSL. An alternative is to use httr::GET (or RCurl::getURL) to first read the file into R as a character vector before parsing with read.table(text=...), or you can download the file to a local directory. rio (GitHub ) provides an import() function that can read a number of common data formats directly from an https:// URL. The repmis function source_data() can load and cache plain-text data from a URL (either http or https). That package also includes source_Dropbox() for downloading/caching plain-text data from non-public Dropbox folders and source_XlsxData() for downloading/caching Excel xlsx sheets.
  • Authentication: Using web resources can require authentication, either via API keys, OAuth, username:password combination, or via other means. Additionally, sometimes web resources that require authentication be in the header of an http call, which requires a little bit of extra work. API keys and username:password combos can be combined within a url for a call to a web resource (api key: http://api.foo.org/?key=yourkey; user/pass: http://username:password@api.foo.org), or can be specified via commands in RCurl or httr. OAuth is the most complicated authentication process, and can be most easily done using httr. See the 6 demos within httr, three for OAuth 1.0 (linkedin, twitter, vimeo) and three for OAuth 2.0 (facebook, GitHub, google). ROAuth is a package that provides a separate R interface to OAuth. OAuth is easier to to do in httr, so start there. googleAuthR provides an OAuth 2.0 setup specifically for Google web services, and AzureAuth provides similar functionality for Azure Active Directory.

Handling HTTP Errors/Codes

  • fauxpas brings a set of Ruby or Python like R6 classes for each individual HTTP status code, allowing simple and verbose messages, with a choice of using messages, warnings, or stops.
  • httpcode is a simple package to help a user/package find HTTP status codes and associated messages by name or number.

Parsing Structured Web Data

The vast majority of web-based data is structured as plain text, HTML, XML, or JSON (javascript object notation). Web service APIs increasingly rely on JSON, but XML is still prevalent in many applications. There are several packages for specifically working with these format. These functions can be used to interact directly with insecure webpages or can be used to parse locally stored or in-memory web files.

  • XML: There are two packages for working with XML: XML and xml2 (GitHub ). Both support general XML (and HTML) parsing, including XPath queries. The package xml2 is less fully featured, but more user friendly with respect to memory management, classes (e.g., XML node vs. node set vs. document), and namespaces. Of the two, only the XML supports de novo creation of XML nodes and documents. The XML2R (GitHub ) package is a collection of convenient functions for coercing XML into data frames. An alternative to XML is selectr , which parses CSS3 Selectors and translates them to XPath 1.0 expressions. XML package is often used for parsing xml and html, but selectr translates CSS selectors to XPath, so can use the CSS selectors instead of XPath.
  • HTML: All of the tools that work with XML also work for HTML, though HTML is - in practice - more prone to be malformed. Some tools are designed specifically to work with HTML. xml2::read_html() is a good first function to use for importing HTML. htmltools provides functions to create HTML elements. htmltab (GitHub ) extracts structured information from HTML tables, similar to XML::readHTMLTable of the XML package, but automatically expands row and column spans in the header and body cells, and users are given more control over the identification of header and body rows which will end up in the R table. The selectorgadget browser extension can be used to identify page elements. RHTMLForms reads HTML documents and obtains a description of each of the forms it contains, along with the different elements and hidden fields. scrapeR provides additional tools for scraping data from HTML documents. htmltidy (GitHub ) provides tools to “tidy” messy HTML documents. htm2txt uses regex to converts html documents to plain text by removing all html tags. Rcrawler does crawling and scraping of web pages.
  • JSON: There are several packages for reading and writing JSON: rjson, RJSONIO, and jsonlite. jsonlite includes a different parser from RJSONIO called yajl . We recommend using jsonlite. Check out the paper describing jsonlite by Jeroen Ooms https://arxiv.org/abs/1403.2805 . jqr provides bindings for the fast JSON library, jq . jsonvalidate (GitHub ) validates JSON against a schema using the “is-my-json-valid” Javascript library; ajv does the same using the ajv Javascript library. ndjson (GitHub ) supports the “ndjson” format.
  • RSS/Atom: feedeR can be used to parse RSS or Atom feeds. tidyRSS parses RSS, Atom XML/JSON and geoRSS into a tidy data.frame.
  • swagger can be used to automatically generate functions for working with an web service API that provides documentation in Swagger.io format.

Tools for Working with URLs

  • The httr::parse_url() function can be used to extract portions of a URL. The RCurl::URLencode() and utils::URLencode() functions can be used to encode character strings for use in URLs. utils::URLdecode() decodes back to the original strings. urltools (GitHub ) can also handle URL encoding, decoding, parsing, and parameter extraction.
  • iptools can facilitate working with IPv4 addresses, including for use in geolocation.
  • urlshorteneR offers URL expansion and analysis for Bit.ly, Goo.gl, and is.gd. longurl uses the longurl.org API to provide similar functionality.
  • gdns provides access to Google’s secure HTTP-based DNS resolution service.

Tools for Working with Scraped Webpage Contents

  • Several packages can be used for parsing HTML documents. boilerpipeR provides generic extraction of main text content from HTML files; removal of ads, sidebars and headers using the boilerpipe Java library. RTidyHTML interfaces to the libtidy library for correcting HTML documents that are not well-formed. This library corrects common errors in HTML documents. W3CMarkupValidator provides an R Interface to W3C Markup Validation Services for validating HTML documents.
  • For XML documents, the XMLSchema package provides facilities in R for reading XML schema documents and processing them to create definitions for R classes and functions for converting XML nodes to instances of those classes. It provides the framework for meta-computing with XML schema in R. xslt is an extension for the xml2 package to transform XML documents by applying an xslt style-sheet. (It can be seen as a modern replacement for Sxslt, which is an interface to Dan Veillard’s libxslt translator, and the SXalan package.) This may be useful for webscraping, as well as transforming XML markup into another human- or machine-readable format (e.g., HTML, JSON, plain text, etc.). SSOAP provides a client-side SOAP (Simple Object Access Protocol) mechanism. Beware, SSOAP itself may not install, and/or its dependencies. The best bet is to get the web service maintainers to switch to REST. XMLRPC provides an implementation of XML-RPC, a relatively simple remote procedure call mechanism that uses HTTP and XML. This can be used for communicating between processes on a single machine or for accessing Web services from within R.
  • Rcompression (not on CRAN): Interface to zlib and bzip2 libraries for performing in-memory compression and decompression in R. This is useful when receiving or sending contents to remote servers, e.g. Web services, HTTP requests via RCurl.
  • tm.plugin.webmining: Extensible text retrieval framework for news feeds in XML (RSS, ATOM) and JSON formats. Currently, the following feeds are implemented: Google Blog Search, Google Finance, Google News, NYTimes Article Search, Reuters News Feed, Yahoo Finance and Yahoo Inplay.
  • webshot uses PhantomJS to provide screenshots of web pages without a browser. It can be useful for testing websites (such as Shiny applications).

Security

  • securitytxt identifies and parses web Ssecurity policy files.

Other Useful Packages and Functions

  • Javascript: V8 (GitHub ) is an R interface to Google’s open source, high performance JavaScript engine. It can wrap Javascript libraries as well as NPM packages. The SpiderMonkey package provides another means of evaluating JavaScript code, creating JavaScript objects and calling JavaScript functions and methods from within R. This can work by embedding the JavaScript engine within an R session or by embedding R in an browser such as Firefox and being able to call R from JavaScript and call back to JavaScript from R. The js package wraps V8 and validates, reformats, optimizes and analyzes JavaScript code.
  • Email:: mailR is an interface to Apache Commons Email to send emails from within R. sendmailR provides a simple SMTP client. gmailr provides access the Google’s gmail.com RESTful API.
  • Mocking:: webmockr is a library for stubbing and setting expectations on HTTP requests. It is inspired from Rubys webmock. This package only helps mock HTTP requests, and returns nothing when requests match expectations. webmockr integrates with the HTTP packages crul and httr. See Testing for mocking with returned responses.
  • Testing:: vcr provides an interface to easily cache HTTP requests in R package test suites (but can be used outside of testing use cases as well). vcr relies on webmockr to do the HTTP request mocking. vcr integrates with the HTTP packages crul and httr. httptest provides a framework for testing packages that communicate with HTTP APIs, offering tools for mocking APIs, for recording real API responses for use as mocks, and for making assertions about HTTP requests, all without requiring a live connection to the API server at runtime. httptest only works with httr.
  • Miscellaneous: webutils contains various functions for developing web applications, including parsers for application/x-www-form-urlencoded as well as multipart/form-data. mime (GitHub ) guesses the MIME type for a file from its extension. rsdmx provides tools to read data and metadata documents exchanged through the Statistical Data and Metadata Exchange (SDMX) framework. The package currently focuses on the SDMX XML standard format (SDMX-ML). robotstxt provides functions and classes for parsing robots.txt files and checking access permissions; spiderbar does the same. uaparserjs (GitHub ) uses the javascript “ua-parser” library to parse User-Agent HTTP headers. rjsonapi consumes APIs that Follow the JSON API Specification . rapiclient is a client for consuming APIs that follow the Open API format . restfulr models a RESTful service as if it were a nested R list.

Web and Server Frameworks

  • Model Operationalization (previously DeployR) is a Microsoft product that provides support for deploying R and Python models and code to a server as a web service to later consume.
  • The shiny package makes it easy to build interactive web applications with R.
  • dash is a web framework which is available for Python, R and Julia, with components written in React.js.
  • Other web frameworks include: fiery that is meant to be more flexible but less easy to use than shiny (reqres and routr are utilities used by fiery that provide HTTP request and response classes, and HTTP routing, respectively); rcloud provides an iPython notebook-style web-based R interface; and Rook, which contains the specification and convenience software for building and running Rook applications.
  • The opencpu framework for embedded statistical computation and reproducible research exposes a web API interfacing R, LaTeX and Pandoc. This API is used for example to integrate statistical functionality into systems, share and execute scripts or reports on centralized servers, and build R based apps.
  • Several general purpose server/client frameworks for R exist. Rserve and RSclient provide server and client functionality for TCP/IP or local socket interfaces. httpuv provides a low-level socket and protocol support for handling HTTP and WebSocket requests directly within R. Another related package, perhaps which httpuv replaces, is websockets . servr provides a simple HTTP server to serve files under a given directory based on httpuv.
  • Several packages offer functionality for turning R code into a web API. FastRWeb provides some basic infrastructure for this. plumber allows you to create a REST API by decorating existing R source code.
  • The WADL package provides tools to process Web Application Description Language (WADL) documents and to programmatically generate R functions to interface to the REST methods described in those WADL documents. (not on CRAN)
  • The RDCOMServer provides a mechanism to export R objects as (D)COM objects in Windows. It can be used along with the RDCOMClient package which provides user-level access from R to other COM servers. (not on CRAN)
  • rapporter.net provides an online environment (SaaS) to host and run rapport statistical report templates in the cloud.
  • radiant (GitHub ) is Shiny-based GUI for R that runs in a browser from a server or local machine.
  • The Tiki Wiki CMS/Groupware framework has an R plugin (PluginR ) to run R code from wiki pages, and use data from their own collected web databases (trackers). A demo: https://r.tiki.org/tiki-index.php .
  • The MediaWiki has an extension (Extension:R ) to run R code from wiki pages, and use uploaded data. A mailing list used to be available: R-sig-mediawiki.
  • whisker: Implementation of logicless templating based on Mustache in R. Mustache syntax is described in http://mustache.github.io/mustache.5.html
  • CGIwithR (not on CRAN) allows one to use R scripts as CGI programs for generating dynamic Web content. HTML forms and other mechanisms to submit dynamic requests can be used to provide input to R scripts via the Web to create content that is determined within that R script.

Web Services

Cloud Computing and Storage

  • The cloudyr project , which is currently under active development on GitHub, aims to provide interfaces to popular Amazon, Azure and Google cloud services without the need for external system dependencies.
  • Amazon Web Services is a popular, proprietary cloud service offering a suite of computing, storage, and infrastructure tools. aws.signature provides functionality for generating AWS API request signatures.
    • Elastic Cloud Compute (EC2) is a cloud computing service. segue (not on CRAN) is a package for managing EC2 instances and S3 storage, which includes a parallel version of lapply() for the Elastic Map Reduce (EMR) engine called emrlapply(). It uses Hadoop Streaming on Amazon’s EMR in order to get simple parallel computation.
    • DBREST: RAmazonDBREST provides an interface to Amazon’s Simple DB API.
    • paws (GitHub ) is an interface to nearly all AWS APIs, including compute, storage, databases, and machine learning. It also requires no external system dependencies.
  • Azure is Microsoft’s cloud computing service. It provides Paas, SaaS and IaaS and supports many different tools and frameworks, including both Microsoft-specific and third-party systems.
    • Azure Active Directory (AAD) is a centralised directory and identity service. AzureAuth is an R client for AAD; use this to obtain OAuth tokens for authenticating with other Azure services, including Resource Manager and storage (see next).
    • Azure Resource Manager (ARM) is a service for deploying other Azure services. AzureRMR is an R interface to ARM, and allows managing subscriptions, resource groups, resources and templates. It exposes a general R6 class framework that can extended to provide extra functionality for specific services (see next).
    • Azure Storage Accounts are a general-purpose data storage facility. Different types of storage are available: file, blob, table, Data Lake, and more. AzureStor provides an R interface to storage. Features include clients for file, blob and Data Lake Gen2 storage, parallelized file transfers, and an interface to Microsoft’s cross-platform AzCopy commandline utility. Also supplied is an ARM interface, to allow creation and managing of storage accounts.
    • Data Science Virtual Machines (DSVMs) are Azure VMs that come preloaded with a wide variety of software for statistics and machine learning, including R, Python, TensorFlow, and Spark. AzureVM is a package for managing DSVMs from within R. It also allows deploying arbitrary VMs by supplying a suitable deployment template.
    • AzureContainers provides a unified facility for working with containers in Azure. Specifically, it includes R interfaces to Azure Container Instances (ACI) , Azure Docker Registry (ACR) and Azure Kubernetes Service (AKS) . Create Docker images and push them to an ACR repository; spin up ACI containers; deploy Kubernetes services in AKS.
    • Azure Data Explorer , also known as Kusto , is a fast, scalable data exploration and analytics service. AzureKusto (not yet on CRAN) is an R interface to ADE/Kusto. It includes a dplyr client interface similar to that provided by dbplyr for SQL databases, a DBI client interface, and an ARM interface for deploying and managing Kusto clusters and databases.
  • googleComputeEngineR interacts with the Google Compute Engine API, and lets you create, start and stop instances in the Google Cloud.
  • Cloud Storage: googleCloudStorageR interfaces with Google Cloud Storage. boxr (GitHub ) is a lightweight, high-level interface for the box.com API . rdrop2 is a Dropbox interface that provides access to a full suite of file operations, including dir/copy/move/delete operations, account information (including quotas) and the ability to upload and download files from any Dropbox account.
  • Docker: analogsea is a general purpose client for the Digital Ocean v2 API. In addition, the package includes functions to install various R tools including base R, RStudio server, and more. There’s an improving interface to interact with docker on your remote droplets via this package.
  • crunch GitHub provides an interface to the crunch.io storage and analytics platform. crunchy GitHub facilitates making Shiny apps on Crunch.
  • rrefine provides a client for the OpenRefine (formerly Google Refine) data cleaning service.

Document and Code Sharing

  • Code Sharing: gistr (GitHub ) works with GitHub gists (gist.github.com ) from R, allowing you to create new gists, update gists with new files, rename files, delete files, get and delete gists, star and un-star gists, fork gists, open a gist in your default browser, get embed code for a gist, list gist commits, and get rate limit information when authenticated. git2r provides bindings to the git version control system and gh is a client for the GitHub API. gitlabr is a GitLab -specific client.
  • Data archiving: dataverse (GitHub ) provides access to Dataverse 4 APIs. rfigshare (GitHub ) connects with Figshare.com . dataone (GitHub ) provides a client for DataONE repositories.
  • Google Drive/Google Documents: The RGoogleDocs package is an example of using the RCurl and XML packages to quickly develop an interface to the Google Documents API. RGoogleStorage provides programmatic access to the Google Storage API. This allows R users to access and store data on Google’s storage. We can upload and download content, create, list and delete folders/buckets, and set access control permissions on objects and buckets.
  • Google Sheets: googlesheets (GitHub ) can access private or public Google Sheets by title, key, or URL. Extract data or edit data. Create, delete, rename, copy, upload, or download spreadsheets and worksheets. gsheet (GitHub ) can download Google Sheets using just the sharing link. Spreadsheets can be downloaded as a data frame, or as plain text to parse manually.
  • imguR (GitHub ) is a package to share plots using the image hosting service Imgur.com . knitr also has a function imgur_upload() to load images from literate programming documents.

Data Analysis and Processing Services

  • Geospatial/Geolocation/Geocoding: Several packages connect to geolocation/geocoding services. rgeolocate (GitHub ) offers several online and offline tools. rydn (not on CRAN) is an interface to the Yahoo Developers network geolocation APIs, and ipapi can be used to geolocate IPv4/6 addresses and/or domain names using the http://ip-api.com/ API. threewords connects to the What3Words API , which represents every 3-meter by 3-meter square on earth as a three-word phrase. opencage (GitHub ) provides access to to the OpenCage geocoding service. nominatim (not on CRAN) connects to the OpenStreetMap Nominatim API for reverse geocoding. PostcodesioR (not on CRAN) provides post code lookup and geocoding for the United Kingdom. geosapi is an R client for the GeoServer REST API, an open source implementation used widely for serving spatial data. geonapi provides an interface to the GeoNetwork legacy API, an opensource catalogue for managing geographic metadata. ows4R is a new R client for the OGC standard Web-Services, such Web Feature Service (WFS) for data and Catalogue Service (CSW) for metadata.
  • Machine Learning as a Service: Several packages provide access to cloud-based machine learning services. OpenML (GitHub ) is the official client for the OpenML API . clarifai (GitHub ) is a Clarifai.com client that enables automated image description. rLTP (GitHub ) accesses the ltp-cloud service . languagelayeR is a client for Languagelayer, a language detection API. googlepredictionapi (not on CRAN): is an R client for the Google Prediction API , a suite of cloud machine learning tools. yhatr lets you deploy, maintain, and invoke models via the Yhat REST API. datarobot works with Data Robot’s predictive modeling platform. mscsweblm4r (GitHub ) interfaces with the Microsoft Cognitive Services Web Language Model API and mscstexta4r (GitHub ) uses the Microsoft Cognitive Services Text Analytics REST API. rosetteApi links to the Rosette text analysis API. googleLanguageR provides interfaces to Google’s Cloud Translation API, Natural Language API, Cloud Speech API, and the Cloud Text-to-Speech API.
  • Machine Translation: translate provides bindings for the Google Translate API v2 and translateR provides bindings for both Google and Microsoft translation APIs. RYandexTranslate (GitHub ) connects to Yandex Translate . transcribeR provides automated audio transcription via the HP IDOL service.
  • Document Processing: abbyyR GitHub and captr (GitHub ) connect to optical character recognition (OCR) APIs. pdftables (GitHub ) uses the PDFTables.com webservice to extract tables from PDFs.
  • Mapping: osmar provides infrastructure to access OpenStreetMap data from different sources to work with the data in common R manner and to convert data into available infrastructure provided by existing R packages (e.g., into sp and igraph objects). osrm (GitHub ) provides shortest paths and travel times from OpenStreetMap. osmplotr (GitHub ) extracts customizable map images from OpenStreetMap. RgoogleMaps serves two purposes: it provides a comfortable R interface to query the Google server for static maps, and use the map as a background image to overlay plots within R. R2GoogleMaps provides a mechanism to generate JavaScript code from R that displays data using Google Maps. RKMLDevice allows to create R graphics in Keyhole Markup Language (KML) format in a manner that allows them to be displayed on Google Earth (or Google Maps), and RKML provides users with high-level facilities to generate KML. plotKML can visualization spatial and spatio-temporal objects in Google Earth. ggmap allows for the easy visualization of spatial data and models on top of Google Maps, OpenStreetMaps, Stamen Maps, or CloudMade Maps using ggplot2. mapsapi is an sf-compatible interface to Google Maps API. leafletR: Allows you to display your spatial data on interactive web-maps using the open-source JavaScript library Leaflet. openadds (GitHub ) is an Openaddresses client, and banR provides access to the “Base Adresses Nationale” (BAN) API for French addresses.
  • Online Surveys: qualtRics provide functions to interact with Qualtrics . WufooR (GitHub ) can retrieve data from Wufoo.com forms. redcapAPI (GitHub ) can provide access to data stored in a REDCap (Research Electronic Data CAPture) database, which is a web application for building and managing online surveys and databases developed at Vanderbilt University. formr facilitates use of the formr survey framework, which is built on openCPU. Rexperigen is a client for the Experigen experimental platform .
  • Visualization: Plot.ly is a company that allows you to create visualizations in the web using R (and Python), which is accessible via plotly. googleVis provides an interface between R and the Google chart tools. The RUbigraph package provides an R interface to a Ubigraph server for drawing interactive, dynamic graphs. You can add and remove vertices/nodes and edges in a graph and change their attributes/characteristics such as shape, color, size.
  • Other :

Social Media Clients

  • plusser has been designed to to facilitate the retrieval of Google+ profiles, pages and posts. It also provides search facilities. Currently a Google+ API key is required for accessing Google+ data.
  • Rfacebook provide an interface to the Facebook API.
  • The Rflickr package provides an interface to the Flickr photo management and sharing application Web service. (not on CRAN)
  • instaR (GitHub ) is a client for the Instagram API .
  • Rlinkedin (not on CRAN) is a client for the LinkedIn API. Auth is via OAuth.
  • rpinterest connects to the Pintrest API.
  • vkR is a client for VK, a social networking site based in Russia.
  • meetupr is a client for the Meetup.com API.
  • Twitter: twitteR (GitHub ) provides an interface to the Twitter web API. It claims to be deprecated in favor of rtweet (GitHub ). twitterreport (not on CRAN) focuses on report generation based on Twitter data. streamR provides a series of functions that allow users to access Twitter’s filter, sample, and user streams, and to parse the output into data frames. OAuth authentication is supported. tweet2r is an alternative implementation geared toward SQLite and postGIS databases. graphTweets produces a network graph from a data.frame of tweets. tweetscores (not on CRAN) implements a political ideology scaling measure for specified Twitter users.
  • brandwatchR is a package to retrieve a data from the Brandwatch social listening API. Both raw text and aggregate statistics are available, as well as project and query management functions.

Web Analytics Services

  • Google Trends: gtrendsR offers functions to perform and display Google Trends queries. RGoogleTrends provides an alternative.
  • Google Analytics: googleAnalyticsR, ganalytics, and RGA provide functions for accessing and retrieving data from the Google Analytics APIs . The latter supports OAuth 2.0 authorization. RGA provides a shiny app to explore data. searchConsoleR links to the Google Search Console (formerly Webmaster Tools).
  • Online Advertising: fbRads can manage Facebook ads via the Facebook Marketing API. RDoubleClick (not on CRAN) can retrieve data from Google’s DoubleClick Campaign Manager Reporting API. RSmartlyIO (GitHub ) loads Facebook and Instagram advertising data provided by Smartly.io .
  • Other services: RSiteCatalyst has functions for accessing the Adobe Analytics (Omniture SiteCatalyst) Reporting API.
  • RAdwords (GitHub ) is a package for loading Google Adwords data.
  • webreadr (GitHub ) can process various common forms of request log, including the Common and Combined Web Log formats and AWS logs.

Web Services for R Package Development

  • R-Hub http://log.r-hub.io/ is a project to enable package builds across all architectures. rhub is a package that interfaces with R-Hub to allow you to check a package on the platform.

Other Web Services

Fitness Apps: fitbitScraper (GitHub ) retrieves Fitbit data. RGoogleFit provides similar functionality for Google Fit .

Push Notifications: RPushbullet provides an easy-to-use interface for the Pushbullet service which provides fast and efficient notifications between computers, phones and tablets. pushoverr (GitHub ) can sending push notifications to mobile devices (iOS and Android) and desktop using Pushover . notifyme (GitHub ) can control Phillips Hue lighting.

Reference/bibliography/citation management: RefManageR imports and manage BibTeX and BibLaTeX references with RefManager. rorcid (GitHub ) is a programmatic interface the Orcid.org API, which can be used for identifying scientific authors and their publications (e.g., by DOI). rdatacite connects to DataCite , which manages DOIs and metadata for scholarly datasets. scholar provides functions to extract citation data from Google Scholar. rscopus provides functions to extract citation data from Elsevier Scopus APIs. Convenience functions are also provided for comparing multiple scholars and predicting future h-index values. mathpix convert an image of a formula (typeset or handwritten) via Mathpix webservice to produce the LaTeX code. zen4R provides an Interface to Zenodo REST API, including management of depositions, attribution of DOIs by ‘Zenodo’ and upload of files.

Literature: rplos is a programmatic interface to the Web Service methods provided by the Public Library of Science journals for search. europepmc connects to the Europe PubMed Central service. pubmed.mineR is a package for text mining of PubMed Abstracts that supports fetching text and XML from PubMed. jstor provides functions and helpers to import metadata, ngrams and full-texts from Data for Research service by JSTOR. aRxiv is a client for the arXiv API, a repository of electronic preprints for computer science, mathematics, physics, quantitative biology, quantitative finance, and statistics. roadoi provides an interface to the Unpaywall API for finding free full-text versions of academic papers. rcoreoa is an interface to the CORE API , a search interface for open access scholarly articles. rcrossref is an interface to Crossref’s API, crminer extracts full text from scholarly articles retrieved via Crossref’s Text and Data Mining service; fulltext is a general purpose package to search for, retrieve and extract full text from scholarly articles; and rromeo (GitHub ) is an interface to the SHERPA/RoMEO API , a database of scientific journal archival policies regarding pre-, post-print, and accepted manuscript.

Automated Metadata Harvesting: oai and OAIHarvester harvest metadata using the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) standard. rresync is a client for the ResourceSync framework , a sort of replacement for OAI-PMH.

Wikipedia: WikipediR (GitHub ) is a wrapper for the MediaWiki API, aimed particularly at the Wikimedia ‘production’ wikis, such as Wikipedia. WikidataR (GitHub ) can request data from Wikidata.org , the free knowledgebase. wikipediatrend (GitHub ) provides access to Wikipedia page access statistics. WikidataQueryServiceR is a client for the Wikidata Query Service .

bigrquery (GitHub ): An interface to Google’s bigquery.

sparkbq (GitHub ): Google BigQuery support for sparklyr.

cymruservices queries Team Cymru web security services.

datamart: Provides an S4 infrastructure for unified handling of internal datasets and web based data sources. Examples include dbpedia, eurostat and sourceforge.

discgolf (GitHub ) provides a client to interact with the API for the Discourse web forum platform. The API is for an installed instance of Discourse, not for the Discourse site itself.

rdpla ((GitHub)[https://github.com/ropensci/rdpla\]) works with the Digital Public Library of America API.

factualR: Thin wrapper for the Factual.com server API.

internetarchive: API client for internet archive metadata.

jSonarR: Enables users to access MongoDB by running queries and returning their results in data.frames. jSonarR uses data processing and conversion capabilities in the jSonar Analytics Platform and the JSON Studio Gateway , to convert JSON to a tabular format.

livechatR is a client for the LiveChat API .

mockaRoo (not on CRAN) uses the MockaRoo API to generate mock or fake data based on an input schema.

pivotaltrackR provides an interface to the API for Pivotal Tracker , an agile project management tool.

randNames (GitHub ) generates random names and personal identifying information using the https://randomapi.com/ API.

Rblpapi (GitHub ) is a client for Bloomberg Finance L.P. ROpenFIGI (GitHub ) provides an interface to Bloomberg’s OpenFIGI API.

rerddap: A generic R client to interact with any ERDDAP instance, which is a special case of OPeNDAP ( https://en.wikipedia.org/wiki/OPeNDAP ), or Open-source Project for a Network Data Access Protocol . Allows user to swap out the base URL to use any ERDDAP instance.

restimizeapi provides an interface to trading website estimize.com .

RForcecom: RForcecom provides a connection to Force.com and Salesforce.com.

Two packages, owmr and ROpenWeatherMap, work with the Open Weather Map API .

RSauceLabs (GitHub ) connects to SauceLabs .

RStripe provides an interface to Stripe , an online payment processor.

RZabbix links with the Zabbix network monitoring service API .

slackr (GitHub ) is a client for Slack.com messaging platform.

shutterstock (GitHub ) is to access Shutterstock library from R.

stackr (not on CRAN): An unofficial wrapper for the read-only features of the Stack Exchange API .

telegram (GitHub ) connects with the Telegram Bot API.

trelloR (GitHub ) connects to the Trello API .

tuber is a YouTube API client and tubern is a client for the YouTube Analytics and Reporting API

udapi connects to Urban Dictionary.

useRsnap (not on CRAN) provides an interface to the API for Usersnap , a tool for collecting feedback from web application users.

yummlyr (GitHub ) provides an interface to the Yummly recipe database.

zendeskR: This package provides a wrapper for the Zendesk API.

ZillowR is a client for the Zillow real estate service.

docuSignr provides an interface to the DocuSign Rest API .

giphyr is an R interface to the Giphy API for GIF’s

duckduckr is an R interface DuckDuckGo’s Instant Answer API

CRAN packages:

Related links:

  
Maintainer:Scott Chamberlain, Thomas Leeper, Patrick Mair, Karthik Ram, Christopher Gandrud
Contact:myrmecocystus at gmail.com
Version:2020-06-08
URL:https://CRAN.R-project.org/view=WebTechnologies

Do not edit this README by hand. See CONTRIBUTING.md .

Download Details:

Author: Cran-task-views
Source Code: https://github.com/cran-task-views/WebTechnologies 

#r #web #technologies 

CRAN Task View: WebTechnologies

Kingworld: Fast, and Friendly Bun Web Framework

🦊 KingWorld

Fast, and friendly Bun web framework.

⚡️ Faster than Express.js by 8.5x on M1 Max

Named after my favorite VTuber (Shirakami Fubuki) and composer (Sasakure.UK) song KINGWORLD/白上フブキ(Original)

KingWorld is web framework built for Bun. It focuses on speed, and developer friendliness.

Borrowing concepts from many popular Node.js web frameworks, KingWorld has a very familiar API designed to easily get started while hiding complex abstractions, and embracing simplicity.

Ecosystem

KingWorld can be heavily customized with the use of plugins.

Currently, you can take a look at these:

  • Static for serving static file/folders
  • Cookie for reading/setting cookie
  • Schema for validating request declaratively
  • CORS for handling CORs request

Quick Start

KingWorld is a web framework based on Bun.

bun add kingworld

Now create index.ts, and place the following:

import KingWorld from 'kingworld'

new KingWorld()
    .get("/", () => "🦊 Now foxing")
    .listen(3000)

And run the server:

bun index.ts

Then simply open http://localhost:3000 in your browser.

Congrats! You have just create a new web server in KingWorld 🎉🎉

Routing

Common HTTP methods have a built-in methods for convenient usage:

app.get("/hi", () => "Hi")
    .post("/hi", () => "From Post")
    .put("/hi", () => "From Put")
    .on("M-SEARCH", async () => "Custom Method")
    .listen(3000)

// [GET] /hi => "Hi"
// [POST] /hi => "From Post"
// [PUT] /hi => "From Put"
// [M-SEARCH] /hi => "Custom Method"

To return JSON, simply return any serializable object:

app.get("/json", () => ({
    hi: 'KingWorld'
}))

// [GET] /json => {"hi": "KingWorld"}

All values returned from handler will be transformed into Response.

You can return Response if you want to declaratively control the response.

app
    .get("/number", () => 1)
    .get("/boolean", () => true)
    .get("/promise", () => new Promise((resovle) => resolve("Ok")))
    .get("/response", () => new Response("Hi", {
        status: 200
    }))

// [GET] /number => "1"
// [GET] /boolean => "true"
// [GET] /promise => "Ok"
// [GET] /response => "Hi"

Files are also transformed to response. Simply return Bun.file to serve static file.

app.get("/tako", () => Bun.file('./example/takodachi.png'))

To get path paramameters, prefix the path with a colon:

app.get("/id/:id", ({ params: { id } }) => id)

// [GET] /id/123 => 123

To ensure the type, simply pass a generic:

app.get<{
    params: {
        id: string
    }
}>("/id/:id", ({ params: { id } }) => id)

// [GET] /id/123 => 123

Wildcards are also acceptable:

app.get("/wildcard/*", () => "Hi")

// [GET] /wildcard/ok => "Hi"
// [GET] /wildcard/abc/def/ghi => "Hi"

For custom 404 page, use default:

app.get("/", () => "Hi")
    .default(() => new Response("Not stonk :(", {
        status: 404
    }))

// [GET] / => "Not stonk :("

You can group multiple route with a prefix with group:

app
    .get("/", () => "Hi")
    .group("/auth", app => {
        app
            .get("/", () => "Hi")
            .post("/sign-in", ({ body }) => body)
            .put("/sign-up", ({ body }) => body)
    })
    .listen(3000)

// [GET] /auth/sign-in => "Hi"
// [POST] /auth/sign-in => <body>
// [PUT] /auth/sign-up => <body>

Finally, you can decouple the route logic to a separate plugin.

import KingWorld, { type Plugin } from 'kingworld'

const hi: Plugin = (app) => app
    .get('/hi', () => 'Hi')

const app = new KingWorld()
    .use(hi)
    .get('/', () => 'KINGWORLD')
    .listen(3000)

// [GET] / => "KINGWORLD"
// [GET] /hi => "Hi"

Handler

Handler is a callback function that returns Response. Used in HTTP method handler.

new KingWorld()
    .get(
        '/', 
        // This is handler
        () => "KingWorld"
    )
    .listen(3000)

By default, handler will accepts two parameters: request and store.

// Simplified Handler
type Handler = (request: ParsedRequest, store: Instance['store']) => Response

const handler: Handler = (request: {
    request: Request
    query: ParsedUrlQuery
    params: Record<string, string>
    headers: Record<string, string>
    body: Promise<string | Object>
    responseHeaders: Headers
}, store: Record<any, unknown>)

Handler Request

Handler's request consists of

  • request [Request]
    • Native fetch Request
  • query [ParsedUrlQuery]
    • Parsed Query Parameters as Record<string, string>
    • Default: {}
    • Example:
      • path: /hi?name=fubuki&game=KingWorld
      • query: { "name": "fubuki", "game": "KingWorld" }
  • params [Record<string, string>]
    • Path paramters as object
    • Default: {}
    • Example:
      • Code: app.get("/id/:name/:game")
      • path: /id/kurokami/KingWorld
      • params: { "name": "kurokami", "game": "KingWorld" }
  • headers [Record<string, string>]
    • Function which returns request's headers
  • body [Promise<string | Object>]
    • Function which returns request's body
    • By default will return either string or Object
      • Will return Object if request's header contains Content-Type: application/json, and is deserializable
      • Otherwise, will return string
  • responseHeaders [Header]
    • Mutable object reference, will attached to response's header
    • For example, adding CORS to response as a plugin

Store

Store is a singleton store of the application.

Is recommended for local state, reference of database connection, and other things that need to be available to be used with handler.

Store value if of 2 types:

  • State: Assigned once at creation
  • Ref: Assign at every request
new KingWorld()
    .state('build', Math.random())
    .ref('random', () => Math.random())
    .get("/build", ({}, { build }) => build)
    .get("/random", ({}, { random }) => random)
    .listen(3000)

// [GET] /build => 0.5
// [GET] /build => 0.5 // Will have the same value as first request
// [GET] /date => 0.374
// [GET] /date => 0.785
// [GET] /date => 0.651

State will have any value assigned, eg. Function will be a function reference. However for ref, if a value is a function, it will be called once.

This is for convenient usage of complex logic assigning at the beginning of every request.

You can assign a function to ref by assigning another callback, however if you want to assign function, please use state instead because function should be static.

// ❌ Function is assigned on every request
new KingWorld()
    .ref('getRandom', () => () => Math.random())
    .get("/random", ({}, { getRandom }) => getRandom())

// ✅ Function is assigned once
new KingWorld()
    .state('getRandom', () => Math.random())
    .get("/random", ({}, { getRandom }) => getRandom())

Typed Store

KingWorld accepts generic to type a store globally.

new KingWorld<{
    store: {
        build: number
        random: number
    }
}>()
    .state('build', Math.random())
    .ref('random', () => Math.random())
    .get("/build", ({}, { build }) => build)
    .get("/random", ({}, { random }) => random)
    .listen(3000)

Lifecycle

KingWorld request's lifecycle can be illustrate as the following:

Request -> onRequest -> route -> transform -> preHandler -> Response

The callback that assigned to lifecycle is called hook.

Pre Handler

  • onRequest
    • Call on new request

Internal

  • router.find (route)
    • Find handler assigned to route

Post Handler

  • transform [Handler]
    • Called before validating request
    • Use to transform request's body, params, query before validation
  • preHandler [Handler]
    • Handle request before executing path handler
    • If value returned, will skip to Response process

Lifecycle can be assigned with app.<lifecycle name>():

For example, assigning transform to a request:

app
    // ? Transform params 'id' to number if available
    .transform(({ params }) => {
        if(params.id)
            params.id = +params.id
    })

Local Hook

There's 2 type of hook

  • Global Hook
    • Assign to every handler
  • Local Hook
    • Assigned by third parameters of Route Handler or app.<method>(path, handler, localHook)
    • Affected only scoped handler
app
    // ? Global Hook
    .transform(({ params }) => {
        if(params.id)
            params.id = +params.id + 1
    })
    .get(
        "/id/:id/:name", 
        ({ params: { id, name } }) => `${id} ${name}`,
        // ? Local hook
        {
            transform: ({ params }) => {
                if(params.name === "白上フブキ")
                    params.name = "Shirakami Fubuki"
            }
        }
    )
    .get("/new/:id", ({ params: { id, name } }) => `${id} ${name}`)
    .listen(3000)

// [GET] /id/2/kson => "3 kson"
// [GET] /id/1/白上フブキ => "2 Shirakami Fubuki"
// [GET] /new/1/白上フブキ => "2 白上フブキ"

You can have multiple local hooks as well by assigning it as array:

app
    .get(
        "/id/:id/:name", 
        ({ params: { id, name } }) => `${id} ${name}`,
        {
            transform: [
                ({ params }) => {
                    if(params.id)
                        params.id = +params.id + 1
                },
                ({ params }) => {
                    if(params.name === "白上フブキ")
                        params.name = "Shirakami Fubuki"
                }
            ]
        }
    )
    .listen(3000)

// [GET] /id/2/kson => "3 kson"
// [GET] /id/1/白上フブキ => "2 Shirakami Fubuki"
// [GET] /new/1/白上フブキ => "2 白上フブキ"

PreRequestHandler

Callback assign to lifecycle before routing.

As it's handle before routing, there's no params, query.

type PreRequestHandler = (request: Request, store: Store) => void

Lifecycle that assigned with PreRequestHandler:

  • onRequest

Handler (Event)

Callback assign to lifecycle after routing.

Accept same value as path handler, @see Handler

Lifecycle that assigned with Handler:

  • transform
  • preHandler

Transform

Use to modify request's body, params, query before validation.

app
    .get(
        "/gamer/:name", 
        ({ params: { name }, hi }) => hi(name),
        // ? Local hook
        {
            transform: ({ params }) => {
                if(params.name === "白上フブキ")
                    params.name = "Shirakami Fubuki"
                    
                params.hi = (name: string) => `Hi ${name}`
            }
        }
    )

// [GET] /gamer/白上フブキ => "Shirakami Fubuki"
// [GET] /gamer/Botan => "Botan"

As body is lazily execute as promise, simply use .then to modify body.

new KingWorld()
    .post<{
        body: {
            id: number
            username: string
        }
    }>(
        '/gamer',
        async ({ body }) => {
            const { username } = await body

            return `Hi ${username}`
        },
        {
            transform: (request) => {
                request.body = request.body.then((user) => {
                    user.id = +user.id

                    return user
                })
            }
        }
    )
    .listen(8080)

Schema Validation

Please use @kingworldjs/schema to handle typed-strict validation of incoming request.

Example

import KingWorld from 'kingworld'
import schema, { S } from '@kingworldjs/schema'

new KingWorld()
    .get('/id/:id', ({ request: { params: { id } } }) => id, {
        transform: (request, store) {
            request.params.id = +request.params.id
        },
        preHandler: schema({
            params: S.object().prop('id', S.number().minimum(1).maximum(100))
        })
    })
    .listen(3000)

// [GET] /id/2 => 2
// [GET] /id/500 => Invalid params
// [GET] /id/-3 => Invalid params

See @kingworldjs/schema for more detail about schema validation.

PreHandler

Handle request before executing path handler. If value is returned, the value will be the response instead and skip the path handler.

Schema validation is useful, but as it only validate the type sometime app require more complex logic than type validation.

For example: Checking value if value existed in database before executing the request.

import KingWorld, { S } from 'kingworld'

new KingWorld()
    .post<{
        body: {
            username: string
        }
    }>('/id/:id', ({ request: { body }) => {
            const { username } = await body

            return `Hi ${username}`
        }, {
        preHandler: async ({ body }) => {
            const { username } = await body
            const user = await database.find(username)

            if(user)
                return user.profile
            else
                return Response("User doesn't exists", {
                    status: 400
                })
        }
    })
    .listen(3000)

Plugin

Plugin is used to decouple logic into smaller function.

import KingWorld, { type Plugin } from 'kingworld'

const hi: Plugin = (app) => app
    .get('/hi', () => 'Hi')

const app = new KingWorld()
    .use(hi)
    .get('/', () => 'KINGWORLD')
    .listen(3000)

// [GET] / => "KINGWORLD"
// [GET] /hi => "Hi"

However, plugin can also be used for assigning new store, and hook making it very useful.

To register a plugin, simply add plugin into use.

use can accept 2 parameters:

  • plugin [Plugin]
  • config [Config?] (Optional)
const plugin: Plugin = (
    app, 
    // Config (2nd paramters of `use`)
    { prefix = '/fbk' } = {}
) => app
        .group(prefix, (app) => {
            app.get('/plugin', () => 'From Plugin')
        })

new KingWorld()
    .use(app, {
        prefix: '/fubuki'
    })

To develop plugin with type support, Plugin can accepts generic.

const plugin: Plugin<
    // ? Typed Config
    {
        prefix?: string
    },
    // ? Same as KingWorld<{}>(), will extends current instance
    {
        store: {
            fromPlugin: 'From Logger'
        }
        request: {
            log: () => void
        }
    }
> = (app, { prefix = '/fbk' } = {})  => 
    app
        .state('fromPlugin', 'From Logger')
        .transform(({ responseHeaders }) => {
            request.log = () => {
                console.log('From Logger')
            }

            responseHeaders.append('X-POWERED-BY', 'KINGWORLD')
        })
        .group(prefix, (app) => {
            app.get('/plugin', () => 'From Plugin')
        })

const app = new KingWorld<{
    Store: {
        build: number
        date: number
    }
}>()
    .use(plugin)
    .get('/', ({ log }) => {
        log()

        return 'KingWorld'
    })

// [GET] /fbk/plugin => "From Plugin"

Since Plugin have a type declaration, all request and store will be fully type and extended from plugin.

For example:

// Before plugin registration
new KingWorld<{
    Store: {
        build: number
        date: number
    }
}>()

// After plugin registration
new KingWorld<{
    Store: {
        build: number
        date: number
    } & {
        fromPlugin: 'From Logger'
    }
    Request: {
        log: () => void
    }
}>()

This will enforce type safety across codebase.

const app = new KingWorld<{
    Store: {
        build: number
        date: number
    }
}>()
    .use(plugin)
    .get('/', ({ log }) => {
        // `log` get type declaration reference from `plugin`
        log()

        return 'KingWorld'
    })

Local plugin custom type

Sometime, when you develop local plugin, type reference from main instance is need, but not available after separation.

const plugin: Plugin = (app)  => 
    app
        .get("/user/:id", ({ db, params: { id } }) => 
            // ❌ Type Error: db is not defined or smth like that
            db.find(id)
        )

const app = new KingWorld<{
    Store: {
        database: Database
    }
}>()
    .state('db', database)
    .use(plugin)

That's why plugin can accept the third generic for adding temporary local type but do not extend the main instance.

const plugin: Plugin<
    {},
    {},
    // Same as KingWorld<Instance>
    {
        store: {
            db: Database
        }
    }
> = (app)  => 
    app
        .get("/user/:id", ({ db, params: { id } }) => 
            // ✅ db is now typed
            db.find(id)
        )

const app = new KingWorld()
    .state('db', database)
    .use(plugin)

Async Plugin

To create an async plugin, simply create an async function the return plugin.

const plugin = async (): Plugin => {
    const db = await setupDatabase()

    return (app) => 
        app
            .state('db', database)
            .get("/user/:id", ({ db, params: { id } }) => 
                // ✅ db is now typed
                db.find(id)
            )
}

const app = new KingWorld()
    .state('db', database)
    .use(plugin)

KingWorld Instance

KingWorld can accepts named generic to type global instance.

For example, type-strict store.

const app = new KingWorld<{
    Store: {
        build: number
    }
}>()
    .state('build', 1)

KingWorld instance can accept generic of KingWorldInstance

export interface KingWorldInstance<
    Store extends Record<string, any> = {},
    Request extends Record<string, any> = {}
> {
    Request?: Request
    Store: Store
}

Test

KingWorld is designed to be serverless, only one simple handle is need to be assigned to serverless function.

This also be used to create simple test environment, by simply call handle function.

import { describe, expect, it } from "bun:test"

const req = (path: string) => new Request(path)

describe('Correctness', () => {
    it('[GET] /', async () => {
        const app = new KingWorld().get('/', () => 'Hi')
        const res = await app.handle(req('/'))

        expect(await res.text()).toBe('Hi')
    })
})

Caveat

Sometime KingWorld doesn't perform well in some situation or has some bug, which can be related to Bun.

Notable reference:

However, if you're sure that the bug is related to KingWorld, filing an issue is always welcome.

Optimization

For the current state of Bun, if you wants full speed of Bun, avoid using await in critical path.

As the state of Bun 0.1.3, KingWorld will be slowed down when using await which might occurs from the following:

  • Using await in handler
  • Using schema.body
  • Using request.body

The performance will be slowed down by around 1.75x - 3x vary on how powerful the machine is.

State of KingWorld

KingWorld is an experimental web framework based on bun.

A bleeding edge web framework focused on developer friendliness, and performance, but is not recommended for production.

As KingWorld is in 0.0.0-experimental.x, API is very unstable and will change in any point of time, at-least until 0.1.0 is release.

As bun is currently in early stage of development, some API might changed in the future which might also results KingWorld API to be changed to match the better speed and developer experience.

Author: SaltyAom
Source Code: https://github.com/SaltyAom/kingworld 
License: MIT license

#typescript #http #web #server 

Kingworld: Fast, and Friendly Bun Web Framework
Ivan Petrenko

Ivan Petrenko

1659711524

Top JavaScript Spreadsheet Components for Data Management Apps

Top JavaScript Spreadsheet Components for Data Management Apps

When developing an enterprise web app, you will probably need to complement it with a spreadsheet table. It is one of the most highly-demanded functions for doing business in digital space. It is not surprising that users, who often have to deal with large amounts of information, love this tool. This instrument helps to cover various aspects of data management that contribute to carefully weighed business decisions.

The internet is full of tips on how to quickly add a basic spreadsheet to a web page, but what if your project requires a mode advanced solution with a wider set of capabilities? Implementing this kind of functionality from scratch can be a tedious and time-consuming task. Therefore, you can consider using commercial JavaScript spreadsheet libraries. They make it much easier to embed a multifunctional Excel-like table into web apps written in pure JavaScript or with popular JS frameworks.

In this article, you have an opportunity to get acquainted with the most noteworthy stand-alone JS spreadsheets and UI widget libraries with this functionality in more detail and choose the right one for your project.

#JavaScript #web-development #spreadsheets 

Stand-Alone Spreadsheet Components

First of all, let us go through JS libraries that are primarily intended to contribute to faster integration of a feature-packed spreadsheet into a web app.

DHTMLX Spreadsheet

DHTMLX Spreadsheet is a fully customizable JavaScript spreadsheet component with a convenient API for processing tabular data of any complexity. It has an intuitive Excel-like interface adaptable to touch devices. With this library in your dev stack, you get a lot of features for productive working in spreadsheet tables. Besides common grid operations, you can enable users to freeze rows & columns, lock/unlock cells, create drop-down lists with validation, autofill cells with data, apply text-wrapping, sort columns and data ranges, and much more. A user-friendly toolbar (ribbon) with multiple controls will aid users in formatting and styling specific cells.
DHTMLX Spreadsheet with Data Validation
There is a regularly updated collection of more than 250 built-in functions compatible with Excel and Google Sheets that can be used to manipulate string and numeric data within a cell or range. It is also possible to arrange data in separate worksheets and, when needed, combine data from various sheets into one with the help of cross-referencing via available formulas. DHTMLX Spreadsheet supports a range of predefined number formats with adjustable settings and allows creating custom ones. The spreadsheet interface is also adaptable to various languages. The table navigation is facilitated with a large list of hotkeys. The final version of your document can be converted to Excel format.
spreadsheet based on DHTMLX

All functional benefits of this library are backed up by flexible licensing with affordable prices varying from $599 to $2899. Thus, you can consider DHTMLX Spreadsheet as a product with a good price/quality ratio. Apart from that, you may be interested in reviewing other DHTMLX components for implementing various functionalities required in business apps.
Useful resources: documentation, samples
Free trial version: DHTMLX Spreadsheet

Handsontable

Handsontable is primarily known as a JavaScript grid library for business apps but the decision to include it in this article is not accidental. Although this tool doesn't offer as many text formatting options as Excel, it includes quite a lot of features for manipulating data typical of spreadsheets. The list of the most widely used Handsontable features includes operations on rows & columns (moving, hiding, resizing, freezing), CRUD actions, non-contiguous selection, data validation, export to file, and merging cells. You can also take advantage of more advanced capabilities such as multi-column sorting, data summaries, trimming rows, and nested headers.
Handsontable data grid

With numerous supported cell types such as date, time, dropdown, password, you’ll give users more freedom for working with cell content. To specify the way users will edit data, you can utilize a predefined cell editor or create a custom one. But how does Handsontable address one of the key functions of any business spreadsheet, namely math calculations? To enable users to perform different kinds of calculating operations in your app, you can utilize a separate Formulas plugin based on the HyperFormula calculation engine specifically designed by the Handsontable team. If you have your own ideas on how to make the Handsontable grid tool more effective, you can create custom plugins.

The Handsontable library has a lot of good qualities in terms of functionality that certainly boosts its price. To obtain this tool for web development, you’ll have to fork out from $899 per developer.
Useful resources: documentation, samples
Free trial version: Handsontable

Jspreadsheet PRO

Jspreadsheet PRO (previously known as Jexcel) is an Excel-like spreadsheet plugin notable for its real-time collaboration feature. It offers such essential functionality as text formatting, frozen columns, data filtering, sorting, and searching.
spreadsheet based on Jspreadsheet PRO

Similar to popular spreadsheet software, Jspreadsheet allows utilizing built-in and custom formulas and performing cross-worksheet calculations. The plugin supports several native editors that can be complemented with custom options to enhance the user experience in data editing. Jspreadsheet lacks some of the common built-in features like keyboard navigation, data validation, and value formatting. However, you can put into action third-party extensions or set up your own add-ons to achieve your goals.
spreadsheet based on Jspreadsheet PRO

This JavaScript tool is provided in Base and Premium editions via monthly ($39/$119), yearly ($390/$1,190) subscription or one-time purchase (3699$). The premium edition includes extra features, plugins, extensions, priority tech support, and other goodies.
Useful resources: documentation, samples
Free trial version: Jspreadsheet

SpreadJS

SpreadJS is a multifaceted Excel-inspired solution designed and maintained by GrapeCity to become a strong response to most of modern data handling demands on the web. When exploring its extensive feature-set, it may be hard for many to immediately name any Excel feature that is not supported by this widget.
spreadsheet based on SpreadJS

First of all, SpreadJS is known for its powerful calculation engine with a large collection of functions of different types for performing complex computations when preparing various business documents. There are many features that will help you to expand users’ abilities in working with cells. For example, you can add cell buttons, barcodes, apply rich text, rotate text in cells, auto merge cells, and much more. To offer a deeper insight into your spreadsheet data, it is possible to add pivot tables, charts, sparklines, tables. An auxiliary desktop design tool named SpreadJS Designer offers a code-free way to quickly create a complex spreadsheet layout or load your existing Excel .xlsx template files and get down to work with data right away.
spreadsheet based on SpreadJS

Everything seems to be great in SpreadJS but there is one important factor that may discourage you to buy this product. It is about high associated expenditures. The thing is that you’ll have to pay not only for a developer license but also for a hostname deployment license. Such a bundle will cost you at least 1499$ per developer. GrapeCity also provides the Wijmo library that can be used in combination with SpreadJS for building a consistent business app much faster.
Useful resources: documentation, samples
Free trial version: SpreadJS

Webix Spreadsheet

Webix Spreadsheet is a JavaScript UI widget designed to speed up the integration of a multifunctional spreadsheet table into your web project. It is probably one of the most advanced tools included in the Webix UI library. Spreadsheets built with Webix can be equipped with features to which many users have grown accustomed in the desktop Excel. They are built-in math functions, manipulations with rows and columns, sorting and filtering, editing, various types of data, etc.
spreadsheet based on Webix

The widget allows enriching the cell content with sparklines, images, as placeholders. The ability to leave comments in cells will help to avoid any ambiguities. You can also enable such useful functions as cell merging and conditional cells formatting. At times, users may need to focus just on the presented information, that is where the capabilities to hide the headers of rows and columns and greedlines will come in handy. If none of the built-in functions can help you to perform a required operation, it is possible to create a custom one for this purpose. After users finish using the spreadsheet, they export it to PDF/PNG and Excel or activate the printing function to get the hard copy of the table.

If you decide that the Webix Spreadsheet widget is the right tool for your project, then you can consider several licenses with different pricing options floating from $798 to $9,499. Moreover, Webix also offers a pack of UI controls that will be a great addition to the spreadsheet widget.
Useful resources: documentation, samples
Free trial version: Webix Spreadsheet

Top JavaScript Spreadsheet Components for Data Management Apps
Hunter  Krajcik

Hunter Krajcik

1659705540

Oauth_webauth: A Plugin Provides an Alternative to AppAuth Use WebView

OAuth WebAuth

This plugin offers a WebView implementation approach for OAuth authorization/authentication with identity providers.

Compatibility

PlatformCompatibility
Android
iOS
WebWork in progress

Preamble

Other plugins like Flutter AppAuth uses native implementation of AppAuth which in turn uses SFAuthenticationSession and CustomTabs for iOS and Android respectively. When using SFAuthenticationSession and CustomTabs your app will/could present some problems like:

  • UI: users will notice a breaking UI difference when system browser opens to handle the identity provider authentication process.
  • In iOS an annoying system dialog shows up every time the user tries to authenticate, indicating that the current app and browser could share their information.
  • Your system browser cache is shared with your app which is good and bad, bad because any cache problem due to your every day navigation use could affect your app authentication and the only way to clean cache it's by cleaning system browser cache at operating system level.
  • Authentication page will use the locale from System Browser which in fact uses the Operating System locale, this means if your app uses different language than the Operating System then authentication page will show different internationalization than your app.

Features

With this plugin you will get:

  • Full control over the UI, WebView will run inside your app so Theme and Color Scheme will be yours to choose, in fact you can add AppBar or FloatingActionButton or whatever you thinks it's necessary to your UI.
  • No system dialog will be shown when users tries to authenticate.
  • Users will not be affected by any system browser problem cache and also will be able to clean app browser cache from the authentication screen itself.
  • Authentication page locale can be set from app using the contentLocale property to ensure the same locale. By default Operating System locale will be used if no contentLocale is specified.

Note:

  • contentLocale will apply only if the authentication page supports the specified Locale('...') and accepts the header: 'Accept-Language': 'es-ES'

Migration from ^1.0.0 to ^2.0.0

  • Static constants key for tooltips, message and hero tags were moved from OAuthWebView to BaseWebView
  • OAuthWebView renamed onSuccess function to onSuccessAuth.

Getting started

As stated before this plugin uses WebView implementation specifically the plugin flutter_inappwebview. For any WebView related problem please check the documentation of that plugin at docs.

Android setup

Just add the internet permission to your AndroidManifest

<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.example.example">  
    <uses-permission android:name="android.permission.INTERNET"/>  
 <application>  
    ...
 </application>

iOS setup

Just add this to your Info.plist

<plist version="1.0">  
<dict>
    ....
    ....
    <key>NSAppTransportSecurity</key>
    <dict>
        <key>NSAllowsArbitraryLoads</key>
        <true/>
    </dict>
    ....
    ....
</dict>  
</plist>

Usage

  • This plugin offers a widget OAuthWebView which handles all the authorization/authentication and navigation logic; this widget can be used in any widget tree of your current app or as an individual authentication screen. For individual authentication screen it offers the widget OAuthWebScreen which can be started as a new route and also handles the Android back button to navigate backward when applies.
  • In addition, this plugin offers a simple widget BaseWebView which may be useful for cases in which you need to handle a link to your Auth server, let's say for email confirmation, or password reset, etc. This widget will handle the web UI and automatically get back to you when loaded any of the specified redirectUrls. The BaseWebView widget works very similar to OAuthWebView, it can be used in any widget tree of your current app or as an individual screen. For individual screen it offers the widget BaseWebScreen which can be started as a new route and also handles the Android back button to navigate backward when applies.

NOTE: all widgets can be extended to change/improve its features.

OAuthWebView

An authorization/authentication process can get 3 outputs.

  1. User successfully authenticates, which returns an Oauth2 Credentials object with the access-token and refresh-token.
  2. An error occurred during authorization/authentication, or maybe certificate validation failed.
  3. User canceled authentication.

This plugin offers two variants to handle these outputs.

Variant 1

Awaiting response from navigator route Future.

void loginV1() async {
  final result = await OAuthWebScreen.start(
      context: context,
      authorizationEndpointUrl: authorizationEndpointUrl,
      tokenEndpointUrl: tokenEndpointUrl,
      clientSecret: clientSecret,
      clientId: clientId,
      redirectUrl: redirectUrl,
      baseUrl: baseUrl, //Optional, if you want to get back to the application when the web view gets redirected to baseUrl. This will be like an onCancel callback
      scopes: scopes,
      promptValues: const ['login'],
      loginHint: 'johndoe@mail.com',
      onCertificateValidate: (certificate) {
        ///This is recommended
        /// Do certificate validations here
        /// If false is returned then a CertificateException() will be thrown
        return true;
      },
      contentLocale: Locale('es'),
      refreshBtnVisible: false,
      clearCacheBtnVisible: false,
      textLocales: {
        ///Optionally texts can be localized
        OAuthWebView.backButtonTooltipKey: 'Ir atrás',
        OAuthWebView.forwardButtonTooltipKey: 'Ir adelante',
        OAuthWebView.reloadButtonTooltipKey: 'Recargar',
        OAuthWebView.clearCacheButtonTooltipKey: 'Limpiar caché',
        OAuthWebView.closeButtonTooltipKey: 'Cerrar',
        OAuthWebView.clearCacheWarningMessageKey:
        '¿Está seguro que desea limpiar la caché?',
      });
  if (result != null) {
    if (result is Credentials) {
      authResponse = getPrettyCredentialsJson(result);
    } else {
      authResponse = result.toString();
    }
  } else {
    authResponse = 'User cancelled authentication';
  }
  setState(() {});
}

Variant2

Using callbacks

void loginV2() {
 OAuthWebScreen.start(
     context: context,
     authorizationEndpointUrl: authorizationEndpointUrl,
     tokenEndpointUrl: tokenEndpointUrl,
     clientSecret: clientSecret,
     clientId: clientId,
     redirectUrl: redirectUrl,
     baseUrl: baseUrl, //Optional, if you want to get back to the application when the web view gets redirected to baseUrl. This will be like an onCancel callback
     scopes: scopes,
     promptValues: const ['login'],
     loginHint: 'johndoe@mail.com',
     onCertificateValidate: (certificate) {
       ///This is recommended
       /// Do certificate validations here
       /// If false is returned then a CertificateException() will be thrown
       return true;
     },
     contentLocale: Locale('es'),
     refreshBtnVisible: false,
     clearCacheBtnVisible: false,
     textLocales: {
       ///Optionally text can be localized
       OAuthWebView.backButtonTooltipKey: 'Ir atrás',
       OAuthWebView.forwardButtonTooltipKey: 'Ir adelante',
       OAuthWebView.reloadButtonTooltipKey: 'Recargar',
       OAuthWebView.clearCacheButtonTooltipKey: 'Limpiar caché',
       OAuthWebView.closeButtonTooltipKey: 'Cerrar',
       OAuthWebView.clearCacheWarningMessageKey:
       '¿Está seguro que desea limpiar la caché?',
     },
     onSuccess: (credentials) {
       setState(() {
         authResponse = getPrettyCredentialsJson(credentials);
       });
     },
     onError: (error) {
       setState(() {
         authResponse = error.toString();
       });
     },
     onCancel: () {
       setState(() {
         authResponse = 'User cancelled authentication';
       });
     });
}

BaseWebView

3 possible outputs.

  1. User is successfully redirected, which returns the full redirect url.
  2. An error occurred during navigation.
  3. User canceled web view.

This plugin offers two variants to handle these outputs.

Variant 1

Awaiting response from navigator route Future.

void baseRedirectV1() async {
  final result = await BaseWebScreen.start(
    context: context,
    initialUrl: initialUrl,
    redirectUrls: [redirectUrl, baseUrl],
    onCertificateValidate: (certificate) {
      ///This is recommended
      /// Do certificate validations here
      /// If false is returned then a CertificateException() will be thrown
      return true;
    },
    textLocales: {
      ///Optionally texts can be localized
      BaseWebView.backButtonTooltipKey: 'Ir atrás',
      BaseWebView.forwardButtonTooltipKey: 'Ir adelante',
      BaseWebView.reloadButtonTooltipKey: 'Recargar',
      BaseWebView.clearCacheButtonTooltipKey: 'Limpiar caché',
      BaseWebView.closeButtonTooltipKey: 'Cerrar',
      BaseWebView.clearCacheWarningMessageKey:
      '¿Está seguro que desea limpiar la caché?',
    },
    contentLocale: contentLocale,
    refreshBtnVisible: false,
    clearCacheBtnVisible: false,
  );
  if (result != null) {
    if (result is String) {
      /// If result is String it means redirected successful
      response = 'User redirected to: $result';
    } else {
      /// If result is not String then some error occurred
      response = result.toString();
    }
  } else {
    /// If no result means user cancelled
    response = 'User cancelled';
  }
  setState(() {});
}

Variant2

Using callbacks

void baseRedirectV2() {
 BaseWebScreen.start(
     context: context,
     initialUrl: initialUrl,
     redirectUrls: [redirectUrl, baseUrl],
     onCertificateValidate: (certificate) {
       ///This is recommended
       /// Do certificate validations here
       /// If false is returned then a CertificateException() will be thrown
       return true;
     },
     textLocales: {
       ///Optionally text can be localized
       BaseWebView.backButtonTooltipKey: 'Ir atrás',
       BaseWebView.forwardButtonTooltipKey: 'Ir adelante',
       BaseWebView.reloadButtonTooltipKey: 'Recargar',
       BaseWebView.clearCacheButtonTooltipKey: 'Limpiar caché',
       BaseWebView.closeButtonTooltipKey: 'Cerrar',
       BaseWebView.clearCacheWarningMessageKey:
       '¿Está seguro que desea limpiar la caché?',
     },
     contentLocale: contentLocale,
     refreshBtnVisible: false,
     clearCacheBtnVisible: false,
     onSuccess: () {
       setState(() {
         response = 'User redirected';
       });
     },
     onError: (error) {
       setState(() {
         response = error.toString();
       });
     },
     onCancel: () {
       setState(() {
         response = 'User cancelled';
       });
     });
}

Important notes

  • goBackBtnVisible, goForwardBtnVisible, refreshBtnVisible, clearCacheBtnVisible, closeBtnVisible allows you to show/hide buttons from toolbar, if you want to completely hide toolbar, set all buttons to false.
  • Use urlStream when you need to asynchronously navigate to a specific url, like when user registered using OauthWebAuth and the web view waits for user email verification; in this case when the user opens the email verification link you can navigate to this link by emitting the new url to the stream you previously set in the urlStream instead of creating a new OautHWebAuth or BaseWebView.
  • For more details on how to use check the sample project of this plugin.

Installing

Use this package as a library

Depend on it

Run this command:

With Flutter:

 $ flutter pub add oauth_webauth

This will add a line like this to your package's pubspec.yaml (and run an implicit flutter pub get):

dependencies:
  oauth_webauth: ^2.2.0+9

Alternatively, your editor might support flutter pub get. Check the docs for your editor to learn more.

Import it

Now in your Dart code, you can use:

import 'package:oauth_webauth/oauth_webauth.dart';

example/lib/main.dart

import 'package:example/src/auth_sample_screen.dart';
import 'package:example/src/base_redirect_sample_screen.dart';
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: const MyHomePage(title: 'Oauth WebAuth samples'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({Key? key, required this.title}) : super(key: key);

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
        systemOverlayStyle: SystemUiOverlayStyle.dark,
      ),
      body: Center(
        child: SingleChildScrollView(
          child: Column(
            mainAxisAlignment: MainAxisAlignment.center,
            children: <Widget>[
              ElevatedButton(
                onPressed: () {
                  Navigator.push(
                      context,
                      MaterialPageRoute(
                          builder: (context) => const AuthSampleScreen()));
                },
                child: const Text('Oauth login samples'),
                style: ButtonStyle(
                    backgroundColor: MaterialStateProperty.all(Colors.green)),
              ),
              const SizedBox(height: 4),
              ElevatedButton(
                onPressed: () {
                  Navigator.push(
                      context,
                      MaterialPageRoute(
                          builder: (context) =>
                              const BaseRedirectSampleScreen()));
                },
                child: const Text('Base redirect samples'),
              ),
            ],
          ),
        ),
      ),
    );
  }
}

Author: luis901101
Source Code: https://github.com/luis901101/oauth_webauth 
License: BSD-3-Clause license

#flutter #dart #oauth #web 

Oauth_webauth: A Plugin Provides an Alternative to AppAuth Use WebView
Kishan Pujara

Kishan Pujara

1659694917

ReactJS Web Development Company in India & USA

MultiQoS is a top-class ReactJS Web Development Company in India & USA that provides end-to-end ReactJS Web Development Services to create innovative and dynamic web applications.

We are inclined towards offering robust ReactJS Development Services which consist of all aspects of development, integration, support, and maintenance. Your search for a professional ReactJS Development Company in India ends here.

https://multiqos.com/reactjs-development
 

#React #react #reactjs #web-development 

ReactJS Web Development Company in India & USA
Alexis Gilbert

Alexis Gilbert

1659694087

What perks are you getting when you hire Vue js Developer!

#vue  #web-development 

Giving an exceptional User experience is the prime goal of any digital product owner. And what technology can give an as immense and exquisite experience rather than Vue js Development? However, if you consider current trends hiring a Vue.js Developer is giving the global players an edge over their competitors when it comes to quality UI/UX

Best Vue.JS Development Company | Hire Vue.JS Developer

Services for Upgrading and Maintenance

Hire external Vue JS developers to help your company with cutting-edge technologies that offer all adequate assistance and timely updates.

Advancement of Real-Time Applications

Employ Nmap's VueJS programmers to create real-time processing solutions that will enhance user experience.

Personalized Vue JS application

To customize your mobile and website project, we hire talented and committed VueJS developers from outside.

Integration of a simple app

Hire Vue JS developers to improve your company's understanding by adding new features to your existing applications in a simple manner.

Low-effort optimization

Hire Vue JS developers to provide quick, effective, fit, and finish work that will improve the UI/UX of your project.

Pre-study for Vue.js

The following are just a few fundamental abilities that a Vue.js developer should possess:

  1. Basic Coding functions in Vue.js
  2. Npm or Yarn package management for JavaScript
  3. Command-line interface for Vue (CLI)

Standard basic required Vue.js Knowledge

  • Developers of Vue.js need to have some familiarity with the following:
  • using single-page applications for routing
  • programming in object-oriented style (OOP) and web design
  • Page analysis and the developer dashboard of the web browser
  • Making and utilizing RESTful APIs
  • Projects that are commercial, enterprise, and open-source
  • AJAX's asynchronous site processing
  • analyzing and fixing Vue.js application bugs
  • rendering on the client and server sides
  • knowledge of HTML and CSS code for websites
  • JSON data representation of data

Get your Vue.js project developed without wasting your time or resources and Hire vue js developer in just 48 hours!


 

Monty  Boehm

Monty Boehm

1659677220

WebIO.jl: A bridge between Julia and the Web

WebIO

WebIO provides a simple abstraction for displaying and interacting with web content. It works with:

  • Juno - The hottest Julia IDE
  • IJulia - Jupyter notebooks for Julia
  • Blink - An Electron wrapper to make desktop apps
  • Mux - A web server framework

People using WebIO

This is a non-comprehensive list of projects using WebIO.

Web Artifacts bundle.tar.gz

BuildCoverageDocs
CIcodecov

Author: JuliaGizmos
Source Code: https://github.com/JuliaGizmos/WebIO.jl 
License: View license

#julia #web

WebIO.jl: A bridge between Julia and the Web
Monty  Boehm

Monty Boehm

1659669720

TableView.jl: A Tables.jl Compatible Table Viewer Based on Ag-grid

TableView

TableView.jl is an ag-grid based table viewer built on WebIO.jl. It can display arbitrarily large tables by lazy-loading additional data when scrolling (this is the default for datasets with more than 10k rows).

demo

Usage

showtable(yourtable) returns a WebIO.Scope which can be displayed with multiple frontends (e.g. IJulia, Blink, Juno...). See the WebIO readme for information on that.

Limitations

When trying to display big tables (>10k rows) we switch to lazy-loading additional rows while scrolling, which disables the filtering/sorting that's possible for smaller datasets. It's possible (but not trivial) to write proper backend support for those operations -- PRs would be very welcome.

ag-grid Enterprise

Setting the AG_GRID_LICENSE_KEY environment variable at build and run time will use the enterprise distribution instead of the normal community distribution.

Author: JuliaComputing
Source Code: https://github.com/JuliaComputing/TableView.jl 
License: View license

#julia #data #web #table 

TableView.jl: A Tables.jl Compatible Table Viewer Based on Ag-grid