Got: Nicer interface to The Built-in Http Module

Human-friendly and powerful HTTP request library for Node.js

For browser usage, we recommend Ky by the same people.

Support questions should be asked here.

Install

npm install got

Warning: This package is native ESM and no longer provides a CommonJS export. If your project uses CommonJS, you'll have to convert to ESM or use the dynamic import() function. Please don't open issues for questions regarding CommonJS / ESM. You can also use Got v11 instead which is pretty stable.

Take a peek

JSON mode

Got has a dedicated option for handling JSON payload.
Furthermore, the promise exposes a .json<T>() function that returns Promise<T>.

import got from 'got';

const {data} = await got.post('https://httpbin.org/anything', {
	json: {
		hello: 'world'
	}
}).json();

console.log(data);
//=> {"hello": "world"}

For advanced JSON usage, check out the parseJson and stringifyJson options.

For more useful tips like this, visit the Tips page.

Highlights

Documentation

Main API

Timeouts and retries

Advanced creation

Cache, Proxy and UNIX sockets

Integration

Migration guides

Got plugins

got4aws - Got convenience wrapper to interact with AWS v4 signed APIs
gh-got - Got convenience wrapper to interact with the GitHub API
gl-got - Got convenience wrapper to interact with the GitLab API
gotql - Got convenience wrapper to interact with GraphQL using JSON-parsed queries instead of strings
got-fetch - Got with a fetch interface
got-scraping - Got wrapper specifically designed for web scraping purposes
got-ssrf - Got wrapper to protect server-side requests against SSRF attacks

Legacy

travis-got - Got convenience wrapper to interact with the Travis API
graphql-got - Got convenience wrapper to interact with GraphQL

Comparison

	`got`	`request`	`node-fetch`	`ky`	`axios`	`superagent`
HTTP/2 support	✔️¹	❌	❌	❌	❌	✔️**
Browser support	❌	❌	✔️*	✔️	✔️	✔️
Promise API	✔️	✔️	✔️	✔️	✔️	✔️
Stream API	✔️	✔️	Node.js only	❌	❌	✔️
Pagination API	✔️	❌	❌	❌	❌	❌
Request cancelation	✔️	❌	✔️	✔️	✔️	✔️
RFC compliant caching	✔️	❌	❌	❌	❌	❌
Cookies (out-of-box)	✔️	✔️	❌	❌	❌	❌
Follows redirects	✔️	✔️	✔️	✔️	✔️	✔️
Retries on failure	✔️	❌	❌	✔️	❌	✔️
Progress events	✔️	❌	❌	✔️***	Browser only	✔️
Handles gzip/deflate	✔️	✔️	✔️	✔️	✔️	✔️
Advanced timeouts	✔️	❌	❌	❌	❌	❌
Timings	✔️	✔️	❌	❌	❌	❌
Errors with metadata	✔️	❌	❌	✔️	✔️	❌
JSON mode	✔️	✔️	✔️	✔️	✔️	✔️
Custom defaults	✔️	✔️	❌	✔️	✔️	❌
Composable	✔️	❌	❌	❌	❌	✔️
Hooks	✔️	❌	❌	✔️	✔️	❌
Issues open
Issues closed
Downloads
Coverage	TBD
Build
Bugs
Dependents
Install size
GitHub stars
TypeScript support
Last commit

* It's almost API compatible with the browser fetch API.
** Need to switch the protocol manually. Doesn't accept PUSH streams and doesn't reuse HTTP/2 sessions.
*** Currently, only DownloadProgress event is supported, UploadProgress event is not supported.
¹ Requires Node.js 15.10.0 or above.
❇️ Almost-stable feature, but the API may change. Don't hesitate to try it out!
❔ Feature in early stage of development. Very experimental.

Click here to see the install size of the Got dependencies.

Maintainers


Sindre Sorhus	Szymon Marczak

Former

These amazing companies are using Got

Segment is a happy user of Got! Got powers the main backend API that our app talks to. It's used by our in-house RPC client that we use to communicate with all microservices.
— Vadim Demedes

Antora, a static site generator for creating documentation sites, uses Got to download the UI bundle. In Antora, the UI bundle (aka theme) is maintained as a separate project. That project exports the UI as a zip file we call the UI bundle. The main site generator downloads that UI from a URL using Got and streams it to vinyl-zip to extract the files. Those files go on to be used to create the HTML pages and supporting assets.
— Dan Allen

GetVoIP is happily using Got in production. One of the unique capabilities of Got is the ability to handle Unix sockets which enables us to build a full control interfaces for our docker stack.
— Daniel Kalen

We're using Got inside of Exoframe to handle all the communication between CLI and server. Exoframe is a self-hosted tool that allows simple one-command deployments using Docker.
— Tim Ermilov

Karaoke Mugen uses Got to fetch content updates from its online server.
— Axel Terizaki

Renovate uses Got, gh-got and gl-got to send millions of queries per day to GitHub, GitLab, npmjs, PyPi, Packagist, Docker Hub, Terraform, CircleCI, and more.
— Rhys Arkins

Resistbot uses Got to communicate from the API frontend where all correspondence ingresses to the officials lookup database in back.
— Chris Erickson

Natural Cycles is using Got to communicate with all kinds of 3rd-party REST APIs (over 9000!).
— Kirill Groshkov

Microlink is a cloud browser as an API service that uses Got widely as the main HTTP client, serving ~22M requests a month, every time a network call needs to be performed.
— Kiko Beats

We’re using Got at Radity. Thanks for such an amazing work!
— Mirzayev Farid

Got has been a crucial component of Apify's scraping for years. We use it to extract data from billions of web pages every month, and we really appreciate the powerful API and extensibility, which allowed us to build our own specialized HTTP client on top of Got. The support has always been stellar too.
— Ondra Urban

For enterprise

Available as part of the Tidelift Subscription.

The maintainers of got and thousands of other packages are working with Tidelift to deliver commercial support and maintenance for the open source dependencies you use to build your applications. Save time, reduce risk, and improve code health, while paying the maintainers of the exact dependencies you use. Learn more.

Author: Sindresorhus
Source Code: https://github.com/sindresorhus/got
License: MIT License

#node #http

What is GEEK

Buddha Community

Got: Nicer interface to The Built-in Http Module

Bongani Ngema

1644841620

Got: Nicer interface to The Built-in Http Module

Human-friendly and powerful HTTP request library for Node.js

For browser usage, we recommend Ky by the same people.

Support questions should be asked here.

Install

npm install got

Warning: This package is native ESM and no longer provides a CommonJS export. If your project uses CommonJS, you'll have to convert to ESM or use the dynamic import() function. Please don't open issues for questions regarding CommonJS / ESM. You can also use Got v11 instead which is pretty stable.

Take a peek

JSON mode

Got has a dedicated option for handling JSON payload.
Furthermore, the promise exposes a .json<T>() function that returns Promise<T>.

import got from 'got';

const {data} = await got.post('https://httpbin.org/anything', {
	json: {
		hello: 'world'
	}
}).json();

console.log(data);
//=> {"hello": "world"}

For advanced JSON usage, check out the parseJson and stringifyJson options.

For more useful tips like this, visit the Tips page.

Highlights

Documentation

Main API

Timeouts and retries

Advanced creation

Cache, Proxy and UNIX sockets

Integration

Migration guides

Got plugins

got4aws - Got convenience wrapper to interact with AWS v4 signed APIs
gh-got - Got convenience wrapper to interact with the GitHub API
gl-got - Got convenience wrapper to interact with the GitLab API
gotql - Got convenience wrapper to interact with GraphQL using JSON-parsed queries instead of strings
got-fetch - Got with a fetch interface
got-scraping - Got wrapper specifically designed for web scraping purposes
got-ssrf - Got wrapper to protect server-side requests against SSRF attacks

Legacy

travis-got - Got convenience wrapper to interact with the Travis API
graphql-got - Got convenience wrapper to interact with GraphQL

Comparison

	`got`	`request`	`node-fetch`	`ky`	`axios`	`superagent`
HTTP/2 support	✔️¹	❌	❌	❌	❌	✔️**
Browser support	❌	❌	✔️*	✔️	✔️	✔️
Promise API	✔️	✔️	✔️	✔️	✔️	✔️
Stream API	✔️	✔️	Node.js only	❌	❌	✔️
Pagination API	✔️	❌	❌	❌	❌	❌
Request cancelation	✔️	❌	✔️	✔️	✔️	✔️
RFC compliant caching	✔️	❌	❌	❌	❌	❌
Cookies (out-of-box)	✔️	✔️	❌	❌	❌	❌
Follows redirects	✔️	✔️	✔️	✔️	✔️	✔️
Retries on failure	✔️	❌	❌	✔️	❌	✔️
Progress events	✔️	❌	❌	✔️***	Browser only	✔️
Handles gzip/deflate	✔️	✔️	✔️	✔️	✔️	✔️
Advanced timeouts	✔️	❌	❌	❌	❌	❌
Timings	✔️	✔️	❌	❌	❌	❌
Errors with metadata	✔️	❌	❌	✔️	✔️	❌
JSON mode	✔️	✔️	✔️	✔️	✔️	✔️
Custom defaults	✔️	✔️	❌	✔️	✔️	❌
Composable	✔️	❌	❌	❌	❌	✔️
Hooks	✔️	❌	❌	✔️	✔️	❌
Issues open
Issues closed
Downloads
Coverage	TBD
Build
Bugs
Dependents
Install size
GitHub stars
TypeScript support
Last commit

* It's almost API compatible with the browser fetch API.
** Need to switch the protocol manually. Doesn't accept PUSH streams and doesn't reuse HTTP/2 sessions.
*** Currently, only DownloadProgress event is supported, UploadProgress event is not supported.
¹ Requires Node.js 15.10.0 or above.
❇️ Almost-stable feature, but the API may change. Don't hesitate to try it out!
❔ Feature in early stage of development. Very experimental.

Click here to see the install size of the Got dependencies.

Maintainers


Sindre Sorhus	Szymon Marczak

Former

These amazing companies are using Got

Segment is a happy user of Got! Got powers the main backend API that our app talks to. It's used by our in-house RPC client that we use to communicate with all microservices.
— Vadim Demedes

Antora, a static site generator for creating documentation sites, uses Got to download the UI bundle. In Antora, the UI bundle (aka theme) is maintained as a separate project. That project exports the UI as a zip file we call the UI bundle. The main site generator downloads that UI from a URL using Got and streams it to vinyl-zip to extract the files. Those files go on to be used to create the HTML pages and supporting assets.
— Dan Allen

GetVoIP is happily using Got in production. One of the unique capabilities of Got is the ability to handle Unix sockets which enables us to build a full control interfaces for our docker stack.
— Daniel Kalen

We're using Got inside of Exoframe to handle all the communication between CLI and server. Exoframe is a self-hosted tool that allows simple one-command deployments using Docker.
— Tim Ermilov

Karaoke Mugen uses Got to fetch content updates from its online server.
— Axel Terizaki

Renovate uses Got, gh-got and gl-got to send millions of queries per day to GitHub, GitLab, npmjs, PyPi, Packagist, Docker Hub, Terraform, CircleCI, and more.
— Rhys Arkins

Resistbot uses Got to communicate from the API frontend where all correspondence ingresses to the officials lookup database in back.
— Chris Erickson

Natural Cycles is using Got to communicate with all kinds of 3rd-party REST APIs (over 9000!).
— Kirill Groshkov

Microlink is a cloud browser as an API service that uses Got widely as the main HTTP client, serving ~22M requests a month, every time a network call needs to be performed.
— Kiko Beats

We’re using Got at Radity. Thanks for such an amazing work!
— Mirzayev Farid

Got has been a crucial component of Apify's scraping for years. We use it to extract data from billions of web pages every month, and we really appreciate the powerful API and extensibility, which allowed us to build our own specialized HTTP client on top of Got. The support has always been stellar too.
— Ondra Urban

For enterprise

Available as part of the Tidelift Subscription.

The maintainers of got and thousands of other packages are working with Tidelift to deliver commercial support and maintenance for the open source dependencies you use to build your applications. Save time, reduce risk, and improve code health, while paying the maintainers of the exact dependencies you use. Learn more.

Author: Sindresorhus
Source Code: https://github.com/sindresorhus/got
License: MIT License

#node #http

Ray Patel

1619571780

Top 20 Most Useful Python Modules or Packages

March 25, 2021 Deepak@321 0 Comments

Welcome to my blog, In this article, we will learn the top 20 most useful python modules or packages and these modules every Python developer should know.

Hello everybody and welcome back so in this article I’m going to be sharing with you 20 Python modules you need to know. Now I’ve split these python modules into four different categories to make little bit easier for us and the categories are:

Web Development
Data Science
Machine Learning
AI and graphical user interfaces.

Near the end of the article, I also share my personal favorite Python module so make sure you stay tuned to see what that is also make sure to share with me in the comments down below your favorite Python module.

#python #packages or libraries #python 20 modules #python 20 most usefull modules #python intersting modules #top 20 python libraries #top 20 python modules #top 20 python packages

Antwon Lindgren

1627077240

Go net/http Starter

In this edition of Stripe Developer Office Hours, follow along as CJ Avilla walks through the fundamentals of setting up a basic web application with the net/http go lang package.

#net/http starter #net/http #http #net

Hertha Mayer

1596255360

Super easy trick to bypass Http Interceptors in Angular

Searching for a better folder structure for angular projects? Check this article out, you can thank me later.

Image for post

Photo by Amir samoh on unsplash

Recently, I have been working on a task, regarding AWS S3 bucket file uploading using Angular and preSigned URLs. In that project, I have used HTTP interceptor to handle my request to bind all header parameters. My problem arose while I was trying upload files and found that I wasn’t able to since there were “two Authorization headers” in that request. They were the usual request header with token and the header which auto binds by the presigned Url. To solve this issue, I found a super-easy solution that allowed me to bypass my HTTP interceptor and BOOM! Issue fixed. Let’s dive into what I did.

Wait… What is an Interceptor?

Before starting the explanation of my trick, let’s get to know what interceptors are and how we can use them.

Angular is one of the most popular front-end development frameworks in the developer community. One of the main reasons for that is that Angular provides many built-in tools that help to scale industry level JavaScript applications. Interceptors are one of the tools in the list capable of handling HTTP requests globally. They allow us to intercept incoming and outgoing HTTP requests using the HttpClient. By intercepting the request we can modify or change any parameter of the request.

Before diving any deeper I suggest that you have a basic knowledge of Angular HTTP Client and RxJS Observable.

Here is the trick

As I mentioned above, there are occasions where we need to allow for a custom header (or, in other words, to skip the interceptor action in http requests).

BONUS POINT:_ Usually in Angular best practices, it is better to keep our services separate from modules, components and models etc. Please checkout the below folder structure. It is better if you can refer to this structure for your future implementations._

src
 ┣ app
 ┃ ┣ common
 ┃ ┣ customer
 ┃ ┣ models
 ┃ ┣ public
 ┃ ┣ services
 ┃ ┃ ┣ authentication
 ┃ ┃ ┃ ┣ guards
 ┃ ┃ ┃ ┃ ┣ auth.guard.ts
 ┃ ┃ ┃ ┣ interceptors
 ┃ ┃ ┃ ┃ ┣ response.interceptor.ts
 ┃ ┃ ┃ ┗ auth.service.ts
 ┃ ┃ ┣ directives
 ┃ ┃ ┣ pipes
 ┃ ┃ ┣ resolvers
 ┃ ┃ ┣ services-api
 ┃ ┃ ┣ services-inter
 ┃ ┣ app.component.css
 ┃ ┣ app.component.html
 ┃ ┣ app.component.spec.ts
 ┃ ┣ app.component.ts
 ┃ ┣ app.module.ts
 ┃ ┣ app.routing.ts
 ┃ ┗ app.server.module.ts
 ┣ assets
 ┣ environments
 ┣ favicon.ico
 ┣ index.html

As per the file tree, I kept my interceptors inside the _src/services/interceptors _folder.

Image for post

#http-interceptors #angular #http-client #programming #http-request

Nigel Uys

1651119540

RQ: A Nicer interface for Golang Stdlib HTTP Client

rq

A nicer interface for golang stdlib HTTP client

Why?

Because golang HTTP client is a pain in the a...

Features

Compatible with golang http stdlib: http.Request, http.Response and http.Cookie
Step by step to build your request
Better HTTP client
Better cookie jar
Import/export allow we save/transfer requests in JSON
Default setting: example default User-Agent or Accept-Language

Installation

go get -u github.com/ddo/rq

Getting started

Simple

import "net/http"
import "github.com/ddo/rq"

r := rq.Get("https://httpbin.org/get")

// query https://httpbin.org/get?q=1&q=2&q=3&_=123456
r.Qs("q", "1", "2")
r.Qs("q", "3")
r.Qs("_", "123456")

// send with golang default HTTP client
res, err := http.DefaultClient.Do(r.ParseRequest())
defer res.Body.Close()

Custom client

In case you did not know that golang default http.Client has no timeout. use rq/client which has 180s timeout by default

import "github.com/ddo/rq"
import "github.com/ddo/rq/client"

r := rq.Post("https://httpbin.org/post")

// query
r.Qs("_", "123456")

// Form
r.Send("data", "data value")
r.Send("extra", "extra value")

// use default rq client
// true to tell #Send to read all the response boby when return
data, res, err := client.Send(r, true)
// no need to close res.Body
// read = false -> you need to call res.Body when done reading

Headers

r := rq.Post("https://httpbin.org/post")

r.Set("Content-Type", "application/json")
r.Set("User-Agent", "ddo/rq")

Raw body

r := rq.Post("https://httpbin.org/post")

r.SendRaw(strings.NewReader("raw data binary or json"))

Client

Default

// by default timeout = 3min
// no cookie jar
// and stops after 10 consecutive requests (10 redirects)
customClient := client.New(nil)

Custom Options

import "github.com/ddo/rq/client/jar"

cookieJar := jar.New()

// custom timeout = 10s and cookie jar
customClient := client.New(&Option{
    Timeout: time.Second * 10,
    jar: cookieJar,
})

Default settings

// set default User-Agent
defaultRq := rq.Get("")
defaultRq.Set("User-Agent", "github.com/ddo/rq")

customClient := client.New(&Option{
    DefaultRq: defaultRq,
})

// from now all the requests called via this customClient
// gonna have the User-Agent header = "github.com/ddo/rq"
// if User-Agent header in request is not set

Redirect

Default client stops after 10 consecutive requests
Or you can use client.NoRedirect to disable redirect

client.New(&Option{
    CheckRedirect: client.NoCheckRedirect,
})

Cookies

import "github.com/ddo/rq/client/jar"

cookieJar := jar.New()

customClient := client.New(&client.Option{
    Jar: cookieJar,
})

// get all cookies by hostname
cookies, err := cookieJar.Get("httpbin.org")

// get a cookie by hostname and name
cookie, err := cookieJar.GetByName("httpbin.org", "cookiename").

// set cookies
err := cookieJar.Set("httpbin.org", cookies)

// set a cookie
err := cookieJar.SetOne("httpbin.org", cookie)

// clear the cookie jar
err := cookieJar.Clear("httpbin.org")

// delete a cookie by it's name
err := cookieJar.Delete("httpbin.org", "cookiename")

Documents

rq: here
client: here
jar: here

Debug

Set env DLOG=* to enable logger to see request activities

TODO

List here #1

Author: ddo
Source Code: https://github.com/ddo/rq
License: Apache-2.0 License

#go #golang #client #http