Kubo: IPFS Implementation in Go

kubo

the oldest IPFS implementation, previously known as "go-ipfs" 

What is Kubo?

Kubo (go-ipfs) the earliest and most widely used implementation of IPFS.

It includes:

• an IPFS daemon server
• extensive command line tooling
• an HTTP Gateway (/ipfs/, /ipns/) for serving content to HTTP browsers
• an HTTP RPC API (/api/v0) for controlling the daemon node

Note: other implementations exist.

What is IPFS?

IPFS is a global, versioned, peer-to-peer filesystem. It combines good ideas from previous systems such as Git, BitTorrent, Kademlia, SFS, and the Web. It is like a single BitTorrent swarm, exchanging git objects. IPFS provides an interface as simple as the HTTP web, but with permanence built-in. You can also mount the world at /ipfs.

For more info see: https://docs.ipfs.tech/concepts/what-is-ipfs/

Before opening an issue, consider using one of the following locations to ensure you are opening your thread in the right place:

• kubo (previously named go-ipfs) implementation bugs in this repo.
• Documentation issues in ipfs/docs issues.
• IPFS design in ipfs/specs issues.
• Exploration of new ideas in ipfs/notes issues.
• Ask questions and meet the rest of the community at the IPFS Forum.
• Or chat with us.

Next milestones

Milestones on GitHub

Security Issues

Please follow SECURITY.md.

Install

The canonical download instructions for IPFS are over at: https://docs.ipfs.tech/install/. It is highly recommended you follow those instructions if you are not interested in working on IPFS development.

System Requirements

IPFS can run on most Linux, macOS, and Windows systems. We recommend running it on a machine with at least 2 GB of RAM and 2 CPU cores (kubo is highly parallel). On systems with less memory, it may not be completely stable.

If your system is resource-constrained, we recommend:

1. Installing OpenSSL and rebuilding kubo manually with make build GOTAGS=openssl. See the download and compile section for more information on compiling kubo.
2. Initializing your daemon with ipfs init --profile=lowpower

Docker

More info on how to run kubo (go-ipfs) inside Docker can be found here.

Native Linux package managers

• Arch Linux
• Nix
• Solus
• openSUSE

Arch Linux

# pacman -S kubo

Nix

With the purely functional package manager Nix you can install kubo (go-ipfs) like this:

$ nix-env -i ipfs

You can also install the Package by using its attribute name, which is also ipfs.

Solus

In solus, kubo (go-ipfs) is available in the main repository as go-ipfs.

$ sudo eopkg install go-ipfs

You can also install it through the Solus software center.

openSUSE

Community Package for go-ipfs

Other package managers

• Guix
• Snap

Guix

GNU's functional package manager, Guix, also provides a go-ipfs package:

$ guix package -i go-ipfs

Snap

With snap, in any of the supported Linux distributions:

$ sudo snap install ipfs

The snap sets IPFS_PATH to SNAP_USER_COMMON, which is usually ~/snap/ipfs/common. If you want to use ~/.ipfs instead, you can bind-mount it to ~/snap/ipfs/common like this:

$ sudo mount --bind ~/.ipfs ~/snap/ipfs/common

If you want something more sophisticated to escape the snap confinement, we recommend using a different method to install kubo so that it is not subject to snap confinement.

macOS package managers

• MacPorts
• Nix
• Homebrew

MacPorts

The package ipfs currently points to kubo (go-ipfs) and is being maintained.

$ sudo port install ipfs

Nix

In macOS you can use the purely functional package manager Nix:

$ nix-env -i ipfs

You can also install the Package by using its attribute name, which is also ipfs.

Homebrew

A Homebrew formula ipfs is maintained too.

$ brew install --formula ipfs

Windows package managers

• Chocolatey
• Scoop

Chocolatey

PS> choco install ipfs

Scoop

Scoop provides kubo as go-ipfs in its 'extras' bucket.

PS> scoop bucket add extras
PS> scoop install go-ipfs

Install prebuilt binaries

From there:

• Click the blue "Download kubo" on the right side of the page.
• Open/extract the archive.
• Move kubo (ipfs) to your path (install.sh can do it for you).

If you are unable to access dist.ipfs.io, you can also download kubo (go-ipfs) from:

• this project's GitHub releases page
• /ipns/dist.ipfs.io at dweb.link gateway

Build from Source

kubo's build system requires Go and some standard POSIX build tools:

• GNU make
• Git
• GCC (or some other go compatible C Compiler) (optional)

To build without GCC, build with CGO_ENABLED=0 (e.g., make build CGO_ENABLED=0).

Install Go

If you need to update: Download latest version of Go.

You'll need to add Go's bin directories to your $PATH environment variable e.g., by adding these lines to your /etc/profile (for a system-wide installation) or $HOME/.profile:

export PATH=$PATH:/usr/local/go/bin
export PATH=$PATH:$GOPATH/bin

(If you run into trouble, see the Go install instructions).

Download and Compile IPFS

$ git clone https://github.com/ipfs/kubo.git

$ cd kubo
$ make install

Alternatively, you can run make build to build the go-ipfs binary (storing it in cmd/ipfs/ipfs) without installing it.

NOTE: If you get an error along the lines of "fatal error: stdlib.h: No such file or directory", you're missing a C compiler. Either re-run make with CGO_ENABLED=0 or install GCC.

Cross Compiling

Compiling for a different platform is as simple as running:

make build GOOS=myTargetOS GOARCH=myTargetArchitecture

OpenSSL

To build go-ipfs with OpenSSL support, append GOTAGS=openssl to your make invocation. Building with OpenSSL should significantly reduce the background CPU usage on nodes that frequently make or receive new connections.

Note: OpenSSL requires CGO support and, by default, CGO is disabled when cross-compiling. To cross-compile with OpenSSL support, you must:

1. Install a compiler toolchain for the target platform.
2. Set the CGO_ENABLED=1 environment variable.

Troubleshooting

• Separate instructions are available for building on Windows.
• git is required in order for go get to fetch all dependencies.
• Package managers often contain out-of-date golang packages. Ensure that go version reports at least 1.10. See above for how to install go.
• If you are interested in development, please install the development dependencies as well.
• Shell command completions can be generated with one of the ipfs commands completion subcommands. Read docs/command-completion.md to learn more.
• See the misc folder for how to connect IPFS to systemd or whatever init system your distro uses.

Updating

Using ipfs-update

IPFS has an updating tool that can be accessed through ipfs update. The tool is not installed alongside IPFS in order to keep that logic independent of the main codebase. To install ipfs update, download it here.

Downloading builds using IPFS

List the available versions of kubo (go-ipfs) implementation:

$ ipfs cat /ipns/dist.ipfs.io/go-ipfs/versions

Then, to view available builds for a version from the previous command ($VERSION):

$ ipfs ls /ipns/dist.ipfs.io/go-ipfs/$VERSION

To download a given build of a version:

$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_darwin-386.tar.gz # darwin 32-bit build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_darwin-amd64.tar.gz # darwin 64-bit build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_freebsd-amd64.tar.gz # freebsd 64-bit build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_linux-386.tar.gz # linux 32-bit build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_linux-amd64.tar.gz # linux 64-bit build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_linux-arm.tar.gz # linux arm build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_windows-amd64.zip # windows 64-bit build

Getting Started

Usage

To start using IPFS, you must first initialize IPFS's config files on your system, this is done with ipfs init. See ipfs init --help for information on the optional arguments it takes. After initialization is complete, you can use ipfs mount, ipfs add and any of the other commands to explore!

Some things to try

Basic proof of 'ipfs working' locally:

echo "hello world" > hello
ipfs add hello
# This should output a hash string that looks something like:
# QmT78zSuBmuS4z925WZfrqQ1qHaJ56DQaTfyMUF7F8ff5o
ipfs cat <that hash>

Troubleshooting

If you have previously installed IPFS before and you are running into problems getting a newer version to work, try deleting (or backing up somewhere else) your IPFS config directory (~/.ipfs by default) and rerunning ipfs init. This will reinitialize the config file to its defaults and clear out the local datastore of any bad entries.

Please direct general questions and help requests to our forum or our IRC channel (freenode #ipfs).

If you believe you've found a bug, check the issues list and, if you don't see your problem there, either come talk to us on Matrix chat, or file an issue of your own!

Packages

See IPFS in GO documentation.

Development

Some places to get you started on the codebase:

• Main file: ./cmd/ipfs/main.go
• CLI Commands: ./core/commands/
• Bitswap (the data trading engine): go-bitswap
• libp2p
  • libp2p: https://github.com/libp2p/go-libp2p
  • DHT: https://github.com/libp2p/go-libp2p-kad-dht
  • PubSub: https://github.com/libp2p/go-libp2p-pubsub
• IPFS : The Add command demystified

Map of Implemented Subsystems

WIP: This is a high-level architecture diagram of the various sub-systems of this specific implementation. To be updated with how they interact. Anyone who has suggestions is welcome to comment here on how we can improve this!

CLI, HTTP-API, Architecture Diagram

Origin

Description: Dotted means "likely going away". The "Legacy" parts are thin wrappers around some commands to translate between the new system and the old system. The grayed-out parts on the "daemon" diagram are there to show that the code is all the same, it's just that we turn some pieces on and some pieces off depending on whether we're running on the client or the server.

Testing

make test

Development Dependencies

If you make changes to the protocol buffers, you will need to install the protoc compiler.

Developer Notes

Find more documentation for developers on docs

Maintainer Info

• Project Board for active and upcoming work
• Release Process
• Additional PL EngRes Kubo maintainer info

Contributing

We ❤️ all our contributors; this project wouldn’t be what it is without you! If you want to help out, please see CONTRIBUTING.md.

This repository falls under the IPFS Code of Conduct.

Please reach out to us in one chat rooms.

Author: ipfs
Source Code: https://github.com/ipfs/kubo 
License: Unknown and 2 other licenses found

#go #golang #ipfs 

What is GEEK

Buddha Community

Kubo: IPFS Implementation in Go

kubo

the oldest IPFS implementation, previously known as "go-ipfs" 

What is Kubo?

Kubo (go-ipfs) the earliest and most widely used implementation of IPFS.

It includes:

• an IPFS daemon server
• extensive command line tooling
• an HTTP Gateway (/ipfs/, /ipns/) for serving content to HTTP browsers
• an HTTP RPC API (/api/v0) for controlling the daemon node

Note: other implementations exist.

What is IPFS?

IPFS is a global, versioned, peer-to-peer filesystem. It combines good ideas from previous systems such as Git, BitTorrent, Kademlia, SFS, and the Web. It is like a single BitTorrent swarm, exchanging git objects. IPFS provides an interface as simple as the HTTP web, but with permanence built-in. You can also mount the world at /ipfs.

For more info see: https://docs.ipfs.tech/concepts/what-is-ipfs/

Before opening an issue, consider using one of the following locations to ensure you are opening your thread in the right place:

• kubo (previously named go-ipfs) implementation bugs in this repo.
• Documentation issues in ipfs/docs issues.
• IPFS design in ipfs/specs issues.
• Exploration of new ideas in ipfs/notes issues.
• Ask questions and meet the rest of the community at the IPFS Forum.
• Or chat with us.

Next milestones

Milestones on GitHub

Security Issues

Please follow SECURITY.md.

Install

The canonical download instructions for IPFS are over at: https://docs.ipfs.tech/install/. It is highly recommended you follow those instructions if you are not interested in working on IPFS development.

System Requirements

IPFS can run on most Linux, macOS, and Windows systems. We recommend running it on a machine with at least 2 GB of RAM and 2 CPU cores (kubo is highly parallel). On systems with less memory, it may not be completely stable.

If your system is resource-constrained, we recommend:

1. Installing OpenSSL and rebuilding kubo manually with make build GOTAGS=openssl. See the download and compile section for more information on compiling kubo.
2. Initializing your daemon with ipfs init --profile=lowpower

Docker

More info on how to run kubo (go-ipfs) inside Docker can be found here.

Native Linux package managers

• Arch Linux
• Nix
• Solus
• openSUSE

Arch Linux

# pacman -S kubo

Nix

With the purely functional package manager Nix you can install kubo (go-ipfs) like this:

$ nix-env -i ipfs

You can also install the Package by using its attribute name, which is also ipfs.

Solus

In solus, kubo (go-ipfs) is available in the main repository as go-ipfs.

$ sudo eopkg install go-ipfs

You can also install it through the Solus software center.

openSUSE

Community Package for go-ipfs

Other package managers

• Guix
• Snap

Guix

GNU's functional package manager, Guix, also provides a go-ipfs package:

$ guix package -i go-ipfs

Snap

With snap, in any of the supported Linux distributions:

$ sudo snap install ipfs

The snap sets IPFS_PATH to SNAP_USER_COMMON, which is usually ~/snap/ipfs/common. If you want to use ~/.ipfs instead, you can bind-mount it to ~/snap/ipfs/common like this:

$ sudo mount --bind ~/.ipfs ~/snap/ipfs/common

If you want something more sophisticated to escape the snap confinement, we recommend using a different method to install kubo so that it is not subject to snap confinement.

macOS package managers

• MacPorts
• Nix
• Homebrew

MacPorts

The package ipfs currently points to kubo (go-ipfs) and is being maintained.

$ sudo port install ipfs

Nix

In macOS you can use the purely functional package manager Nix:

$ nix-env -i ipfs

You can also install the Package by using its attribute name, which is also ipfs.

Homebrew

A Homebrew formula ipfs is maintained too.

$ brew install --formula ipfs

Windows package managers

• Chocolatey
• Scoop

Chocolatey

PS> choco install ipfs

Scoop

Scoop provides kubo as go-ipfs in its 'extras' bucket.

PS> scoop bucket add extras
PS> scoop install go-ipfs

Install prebuilt binaries

From there:

• Click the blue "Download kubo" on the right side of the page.
• Open/extract the archive.
• Move kubo (ipfs) to your path (install.sh can do it for you).

If you are unable to access dist.ipfs.io, you can also download kubo (go-ipfs) from:

• this project's GitHub releases page
• /ipns/dist.ipfs.io at dweb.link gateway

Build from Source

kubo's build system requires Go and some standard POSIX build tools:

• GNU make
• Git
• GCC (or some other go compatible C Compiler) (optional)

To build without GCC, build with CGO_ENABLED=0 (e.g., make build CGO_ENABLED=0).

Install Go

If you need to update: Download latest version of Go.

You'll need to add Go's bin directories to your $PATH environment variable e.g., by adding these lines to your /etc/profile (for a system-wide installation) or $HOME/.profile:

export PATH=$PATH:/usr/local/go/bin
export PATH=$PATH:$GOPATH/bin

(If you run into trouble, see the Go install instructions).

Download and Compile IPFS

$ git clone https://github.com/ipfs/kubo.git

$ cd kubo
$ make install

Alternatively, you can run make build to build the go-ipfs binary (storing it in cmd/ipfs/ipfs) without installing it.

NOTE: If you get an error along the lines of "fatal error: stdlib.h: No such file or directory", you're missing a C compiler. Either re-run make with CGO_ENABLED=0 or install GCC.

Cross Compiling

Compiling for a different platform is as simple as running:

make build GOOS=myTargetOS GOARCH=myTargetArchitecture

OpenSSL

To build go-ipfs with OpenSSL support, append GOTAGS=openssl to your make invocation. Building with OpenSSL should significantly reduce the background CPU usage on nodes that frequently make or receive new connections.

Note: OpenSSL requires CGO support and, by default, CGO is disabled when cross-compiling. To cross-compile with OpenSSL support, you must:

1. Install a compiler toolchain for the target platform.
2. Set the CGO_ENABLED=1 environment variable.

Troubleshooting

• Separate instructions are available for building on Windows.
• git is required in order for go get to fetch all dependencies.
• Package managers often contain out-of-date golang packages. Ensure that go version reports at least 1.10. See above for how to install go.
• If you are interested in development, please install the development dependencies as well.
• Shell command completions can be generated with one of the ipfs commands completion subcommands. Read docs/command-completion.md to learn more.
• See the misc folder for how to connect IPFS to systemd or whatever init system your distro uses.

Updating

Using ipfs-update

IPFS has an updating tool that can be accessed through ipfs update. The tool is not installed alongside IPFS in order to keep that logic independent of the main codebase. To install ipfs update, download it here.

Downloading builds using IPFS

List the available versions of kubo (go-ipfs) implementation:

$ ipfs cat /ipns/dist.ipfs.io/go-ipfs/versions

Then, to view available builds for a version from the previous command ($VERSION):

$ ipfs ls /ipns/dist.ipfs.io/go-ipfs/$VERSION

To download a given build of a version:

$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_darwin-386.tar.gz # darwin 32-bit build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_darwin-amd64.tar.gz # darwin 64-bit build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_freebsd-amd64.tar.gz # freebsd 64-bit build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_linux-386.tar.gz # linux 32-bit build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_linux-amd64.tar.gz # linux 64-bit build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_linux-arm.tar.gz # linux arm build
$ ipfs get /ipns/dist.ipfs.io/go-ipfs/$VERSION/go-ipfs_$VERSION_windows-amd64.zip # windows 64-bit build

Getting Started

Usage

To start using IPFS, you must first initialize IPFS's config files on your system, this is done with ipfs init. See ipfs init --help for information on the optional arguments it takes. After initialization is complete, you can use ipfs mount, ipfs add and any of the other commands to explore!

Some things to try

Basic proof of 'ipfs working' locally:

echo "hello world" > hello
ipfs add hello
# This should output a hash string that looks something like:
# QmT78zSuBmuS4z925WZfrqQ1qHaJ56DQaTfyMUF7F8ff5o
ipfs cat <that hash>

Troubleshooting

If you have previously installed IPFS before and you are running into problems getting a newer version to work, try deleting (or backing up somewhere else) your IPFS config directory (~/.ipfs by default) and rerunning ipfs init. This will reinitialize the config file to its defaults and clear out the local datastore of any bad entries.

Please direct general questions and help requests to our forum or our IRC channel (freenode #ipfs).

If you believe you've found a bug, check the issues list and, if you don't see your problem there, either come talk to us on Matrix chat, or file an issue of your own!

Packages

See IPFS in GO documentation.

Development

Some places to get you started on the codebase:

• Main file: ./cmd/ipfs/main.go
• CLI Commands: ./core/commands/
• Bitswap (the data trading engine): go-bitswap
• libp2p
  • libp2p: https://github.com/libp2p/go-libp2p
  • DHT: https://github.com/libp2p/go-libp2p-kad-dht
  • PubSub: https://github.com/libp2p/go-libp2p-pubsub
• IPFS : The Add command demystified

Map of Implemented Subsystems

WIP: This is a high-level architecture diagram of the various sub-systems of this specific implementation. To be updated with how they interact. Anyone who has suggestions is welcome to comment here on how we can improve this!

CLI, HTTP-API, Architecture Diagram

Origin

Description: Dotted means "likely going away". The "Legacy" parts are thin wrappers around some commands to translate between the new system and the old system. The grayed-out parts on the "daemon" diagram are there to show that the code is all the same, it's just that we turn some pieces on and some pieces off depending on whether we're running on the client or the server.

Testing

make test

Development Dependencies

If you make changes to the protocol buffers, you will need to install the protoc compiler.

Developer Notes

Find more documentation for developers on docs

Maintainer Info

• Project Board for active and upcoming work
• Release Process
• Additional PL EngRes Kubo maintainer info

Contributing

We ❤️ all our contributors; this project wouldn’t be what it is without you! If you want to help out, please see CONTRIBUTING.md.

This repository falls under the IPFS Code of Conduct.

Please reach out to us in one chat rooms.

Author: ipfs
Source Code: https://github.com/ipfs/kubo 
License: Unknown and 2 other licenses found

#go #golang #ipfs 

What's new in the go 1.15

Go announced Go 1.15 version on 11 Aug 2020. Highlighted updates and features include Substantial improvements to the Go linker, Improved allocation for small objects at high core counts, X.509 CommonName deprecation, GOPROXY supports skipping proxies that return errors, New embedded tzdata package, Several Core Library improvements and more.

As Go promise for maintaining backward compatibility. After upgrading to the latest Go 1.15 version, almost all existing Golang applications or programs continue to compile and run as older Golang version.

#go #golang #go 1.15 #go features #go improvement #go package #go new features

WTH is IPFS? InterPlanetary File Systems To Rescue The Internet

Introduced in February 2015, TechCrunch Magazine noted IPFS was “quickly spreading by word of mouth.”

It’s possibly a key component to solving deep-seated but largely unknown problems that exist in today’s internet usage.

Some believe IPFS, a new tongue-twisting acronym, is a tool that’ll finally evolve the internet from central entities to a world wide web of shared information, as our online founders always envisioned.

The Simple Breakdown:

IPFS = Git + BitTorrent

To understand IPFS or an Interplanetary File System, one can envision a file system that stores files and tracks versions over time, like the project Git……

Photo Source.

Git is a distributed version control system, or VCS. Developers use it to track changes to their code . When text is added, edited or deleted in any piece of code, Git tracks the changes line-by-line It’s distributed because every user has all the source code on their computer and can act as a server. Secondly, Git is backed by a content-addressable database, meaning the content in the database is immutable.

But… IPFS also incorporates how files move across a network, making it a distributed file system, such as BitTorrent.

Photo Source.

_BitTorrent allows users to quickly download large files using minimum internet bandwidth. For this reason it is free to use and includes no spyware or pop-up advertising. BitTorrent’s protocol maximizes transfer speed by gathering small pieces of files desired by the user and downloads the content pieces simultaneously from other users that retain seekers content. BitTorrent is popular for freely sharing videos, programs, books, music, legal/medical records, and more. Not to mention, the downloads are much faster than other protocols. _

Photo Source.

Git + BitTorrent = BFF

IPFS uses BitTorrent’s approach, but applies Git’s concept and creates a new type of file system that tracks the respective versions of files from all the users in the network.

By utilizing both characteristics of these two entities, IPFS birthed a new permanent web that challenges existing internet protocols, such as HTTP.

WTF Is Wrong With HTTP? It Seems Fine To Me, TYVM.

Well, first, I’m not sure if you realized you had options. So while the thought: “The World Wide Web is actually wider…” sinks into your head, here’s a brief summary of what you’ve likely been using for some time.

Sit back for a quick lesson on HTTP, ASAP. (lol, you love the free jokes in my articles).

The internet is a collection of protocols that describe how data moves through a network. Developers adopted these protocols over time as they built applications on top of existing infrastructure. The protocol that serves as the backbone of the web is the HyperText Transfer Protocol.

HTTP, or HyperText Transfer Protocol is an application layer protocol for distributed, collaborative hypermedia systems created by Tim Berners Lee at CERN in 1989. HTTP is the foundation for data communication using hypertext files. It is currently used for most of the data transfer on the internet.

HTTP is a request-response protocol.

Since the internet boasts a vast array of resources hosted on different servers. To access these resources, a browser needs to be able to send a request to the server and display the resources. HTTP is the underlying format for structuring requests and responses for communication between client and host.

The message that is sent by a client to a server is what is known as an HTTP request. When these requests are being sent, clients can use various methods to make this request.

HTTP request methods are the assets that indicate a specific action to be performed on a given resource. Each method implements a distinct semantic, but there are some shared features.

Common Example: the Google homepage-back to the “client” or browser. This is a location-addressed protocol which means when google.com is inserted into a browser, it gets translated into an IP address belonging to a Google server, initiating a request-response cycle with that server.

WTF, We Have a 404 On HTTP! SOS!

Internet savvy, or not, I believe we have all fallen victim to an HTTP meltdown at least once in our lives.

Photo Source.

Do you recall a time in your life when you and a large group of people went to the same website at the same time?

Each individual participating in this action types the request into their online device and sends a request to that website, where a response is given.

each person is sent the same data, individually. If there’s 10,000 people trying to access a site, on the backend, there are 10k requests, and 10k responses. This sounds great, right? Problem is it’s pretty inefficient.

In a perfect world, participants should be able to leverage physical proximity to more effectively retrieve requested information.

HTTP presents another significant issue if there is a problem in the network’s line of communication, leaving the client unable to connect with the server.

This occurs if:

• A country is blocking some content
• An ISP has an outage
• Content was merely moved or deleted.

These types of broken links exist everywhere on the HTTP web.

A location-based addressing model, like HTTP, encourages the centralization of information.

It is convenient to trust a handful of applications with all our online data, but there’s a great sense of power and responsibility that comes with placing centralized providers with our precious personal and public information.

#ipfs #blockchain #bittorrent #women-in-blockchain #women-in-tech #what-is-ipfs #how-does-ipfs-work #hackernoon-top-story

Secure HTTPS servers in Go

In this article, we are going to look at some of the basic APIs of the http package to create and initialize HTTPS servers in Go.

(source: unsplash.com)

In the “Simple Hello World Server” lesson, we learned about net/http package, how to create routes and how [ServeMux](https://golang.org/pkg/net/http/#ServeMux) works. In the “Running multiple HTTP servers” lesson, we learned about [Server](https://golang.org/pkg/net/http/#Server) structure and how to run multiple HTTP servers concurrently.

In this lesson, we are going to create an HTTPS server using both Go’s standard server configuration and custom configuration (using [_Server_](https://golang.org/pkg/net/http/#Server) structure). But before this, we need to know what HTTPS really is?

HTTPS is a big topic of discussion in itself. Hence while writing this lesson, I published an article just on “How HTTPS works?”. I advise you to read this lesson first before continuing this article. In this article, I’ve also described the encryption paradigm and SSL certificates generation process.

---

If we recall the simplest HTTP server example from previous lessons, we only need http.``[ListenAndServe](https://golang.org/pkg/net/http/#ListenAndServe) function to start an HTTP server and http.``[HandleFunc](https://golang.org/pkg/net/http/#HandleFunc) to register a response handler for a particular endpoint.

(https://play.golang.org/p/t3sOenOYAzS)

In the example above, when we run the command go run server.go , it will start an HTTP server on port 9000. By visiting http://localhost:9000 URL in a browser, you will be able to see a Hello World! message on the screen.

(http://localhost:9000)

As we know, the nil argument to ListenAndServe() call invokes Go to use the [DefaultServeMux](https://golang.org/pkg/net/http/#DefaultServeMux) response multiplexer, which is the default instance of ServeMux structure provided globally by the Go. The HandleFunc() call adds a response handler for a specific route on the multiplexer instance.

The http.ListenAndServe() call uses the Go’s standard HTTP server configuration, however, in the previous lesson, how we can customize a server using [Server](https://golang.org/pkg/net/http/#Server) structure type.

To start an HTTPS server, all we need do is to call ServerAndListenTLS method with some configuration. Just like ServeAndListen method, this method is available on both the http package and the Server structure.

The http.``[ServeAndListenTLS](https://golang.org/pkg/net/http/#ListenAndServeTLS) method uses the Go’s standard server implementation, however, both [Server](https://golang.org/pkg/net/http/#Server) instance and Server.``[ServeAndListenTLS](https://golang.org/pkg/net/http/#Server.ListenAndServeTLS) method can be configured for our needs.

#go-programming-language #go #golang-tutorial #go-programming #golang

Go-callvis: Visualize Call Graph Of A Go Program using Graphviz

go-callvis

go-callvis is a development tool to help visualize call graph of a Go program using interactive view.

---

Introduction

The purpose of this tool is to provide developers with a visual overview of a Go program using data from call graph and its relations with packages and types. This is especially useful in larger projects where the complexity of the code much higher or when you are just simply trying to understand code of somebody else.

Features

• 🆕 support for Go modules! :boom:
• focus specific package in the program
• click on package to quickly switch the focus using interactive viewer
• group functions by package and/or methods by type
• filter packages to specific import path prefixes
• ignore funcs from standard library
• omit various types of function calls

Output preview

Check out the source code for the above image.

How it works

It runs pointer analysis to construct the call graph of the program and uses the data to generate output in dot format, which can be rendered with Graphviz tools.

Reference guide

Here you can find descriptions for various types of output.

Packages / Types

Represents	Style
focused	blue color
stdlib	green color
other	yellow color

Functions / Methods

Represents	Style
exported	bold border
unexported	normal border
anonymous	dotted border

Calls

Represents	Style
internal	black color
external	brown color
static	solid line
dynamic	dashed line
regular	simple arrow
concurrent	arrow with circle
deferred	arrow with diamond

Quick start

Requirements

• Go 1.13+
• Graphviz (optional, required only with -graphviz flag)

Installation

go get -u github.com/ofabry/go-callvis
# or
git clone https://github.com/ofabry/go-callvis.git
cd go-callvis && make install

Usage

Interactive viewer

To use the interactive view provided by a web server that serves SVG images of focused packages, you can simply run:

go-callvis <target package>

HTTP server is listening on http://localhost:7878/ by default, use option -http="ADDR:PORT" to change HTTP server address.

Render static output

To generate a single output file use option -file=<file path> to choose output file destination.

The output format defaults to svg, use option -format=<svg|png|jpg|...> to pick a different output format.

Options

Usage of go-callvis:
  -debug
        Enable verbose log.
  -file string
        output filename - omit to use server mode
  -cacheDir string
        Enable caching to avoid unnecessary re-rendering.
  -focus string
        Focus specific package using name or import path. (default "main")
  -format string
        output file format [svg | png | jpg | ...] (default "svg")
  -graphviz
        Use Graphviz's dot program to render images.
  -group string
        Grouping functions by packages and/or types [pkg, type] (separated by comma) (default "pkg")
  -http string
        HTTP service address. (default ":7878")
  -ignore string
        Ignore package paths containing given prefixes (separated by comma)
  -include string
        Include package paths with given prefixes (separated by comma)
  -limit string
        Limit package paths to given prefixes (separated by comma)
  -minlen uint
        Minimum edge length (for wider output). (default 2)
  -nodesep float
        Minimum space between two adjacent nodes in the same rank (for taller output). (default 0.35)
  -nointer
        Omit calls to unexported functions.
  -nostd
        Omit calls to/from packages in standard library.
  -rankdir
        Direction of graph layout [LR | RL | TB | BT] (default "LR")
  -skipbrowser
        Skip opening browser.
  -tags build tags
        a list of build tags to consider satisfied during the build. For more information about build tags, see the description of build constraints in the documentation for the go/build package
  -tests
        Include test code.
  -version
        Show version and exit.

Run go-callvis -h to list all supported options.

Examples

Here is an example for the project syncthing.

Check out more examples and used command options.

Community

Join #go-callvis channel at gophers.slack.com. (not a member yet? get invitation)

How to help

Did you find any bugs or have some suggestions?

• Feel free to open new issue or start discussion in the slack channel.

Do you want to contribute to the project?

• Fork the repository and open a pull request. Here you can find TODO features.

---

Roadmap

The interactive tool described below has been published as a separate project called goexplorer!

Ideal goal of this project is to make web app that would locally store the call graph data and then provide quick access of the call graphs for any package of your dependency tree. At first it would show an interactive map of overall dependencies between packages and then by selecting particular package it would show the call graph and provide various options to alter the output dynamically.

Author: Ofabry
Source Code: https://github.com/ofabry/go-callvis 
License: MIT license

#go #golang #visualization