Morioh
Login
Sofia Kelly

Sofia Kelly

4 years ago

Building A Blockchain Network From scratch With Python

In today’s tutorial we will go from the first steps to building a simple blockchain network in python language. Giving us the best overview of how a blockchain works.

Using microframework Flask to create endpoints, then run on multiple machines to create a decentralized network. We will learn how to build a simple user interface to interact with blockchain and store information for every use case, such as peer-to-peer payments, chat, or e-commerce.

The goal is to build an application that allows users to share information by posting posts. Content will be stored on the blockchain, so it will not be changed and exist forever. Users will interact with the application through a simple web interface.

We will use the bottom-up approach. Start by defining the data structure that we will store in the blockchain. Each post will include three essential elements:

Here we will go into the steps:

Store transactions in blocks

Here we will store in the widely used format that is JSON. And here is what a post will be stored in the blockchain:

{ 
  "author": "some_author_name", 
  "content": "Some thoughts that author wants to share", 
  "timestamp": "The time at which the content was created"
}

The term datain blockchain is often replaced by transaction. So here we are unified to use the term transactionto refer datato this application.

Transactions will be packaged into blocks. A block may contain one or more transactions. Blocks containing transactions are generated regularly and added to the blockchain. Because there are many blocks, each block will have a IDunique:

Block class:
     def __init __ (self, index, transactions, timestamp):
         "" "
         Constructor for a `Block` class.
         : param index: The unique ID number of a block.
         : param transactions: List of transactions.
         : param timestamp: Block creation time.
         "" "
         self.index = index
         self.transactions = transactions
         self.timestamp = timestamp

Add digital signatures to blocks

To prevent fake data from being stored inside the blocks and to detect this we will use the hash function.

A hash function is a function that takes data of any size and creates data of a fixed size, often used to determine input data. Ideal hash functions often have the following characteristics:

  • Easy to calculate.
  • The same data will always result in the same hash value.
  • There must be uniform consistency which means that even a single bit change in data will change the hash value significantly.

The results of these properties are:

  • It is almost impossible to predict the hashed input. (The only way is to try all possible input cases)
  • If you know both the input and the hash value, you only need to pass the input via the hash function to verify the provided hash value is correct.

There are various common hash functions. Here is an example in Python we use the hash function SHA-256:

>>> from hashlib import sha256
>>> data = b "Some variable length data"
>>> sha256 (data) .hexdigest ()
'b919fbbcae38e2bdaebb6c04ed4098e5c70563d2dc51e085f784c058ff208516'
>>> sha256 (data) .hexdigest () # no matter how many times you run it, the result will be the same string of 256 characters.
'b919fbbcae38e2bdaebb6c04ed4098e5c70563d2dc51e085f784c058ff208516'
>>> data = b "Some variable length data2" # Added a character at the end.
'9fcaab521baf8e83f07512a7de7a0f567f6eef2688e8b9490694ada0a3ddeec8'

# Note: the hash value has changed completely!

Note: giá trị băm đã thay đổi hoàn toàn !

We will store the block’s hash value into a field inside the object Blockand it will act like the digital signature (or signature) of the data contained therein:

from hashlib import sha256
import json

def compute_hash (block):
     "" "
     Converts the block object into a JSON string and then returns the hash value.
     "" "
     block_string = json.dumps (self .__ dict__, sort_keys = True)
     return sha256 (block_string.encode ()). hexdigest ()

Join blocks into chains

We need a way to ensure that any changes in the previous blocks will invalidate the entire chain. The way Bitcoin works is to create dependencies between successive blocks by stringing them with the block’s hash value immediately. This means saving the hash of the previous block in the current block when adding a new field to the name previous_hash.

There will be a question: what about the first block? That block is called the genesis block and it can be created manually or through some logic.

from hashlib import sha256
import json
import time


Block class:
    def__init __ (self, index, transactions, timestamp, previous_hash):
       "" "
        Constructor for a `Block` class.
        : param index: The unique ID number of a block.
        : param transactions: List of transactions.
        : param timestamp: Block creation time.
        : param previous_hash: Contains the hash value of the preceding block in the chain.
        "" "
        self.index = index
        self.transactions = transactions
        self.timestamp = timestamp
        self.previous_hash = previous_hash # add previous_hash field

    def compute_hash (self):
        "" "
        Converts the block object into a JSON string and then returns the hash value.
        "" "
        block_string = json.dumps (self .__ dict__, sort_keys = True)
        return sha256 (block_string.encode ()). hexdigest ()

Blockchain class:

    def __init __ (self):
        "" "
        Constructor of the `Blockchain` class.
        "" "
        self.chain = []
        self.create_genesis_block ()

    def create_genesis_block (self):
        "" "
        The function generates `block genesis` and adds it to the chain. This block has an index
        equals to 0, previous_hash is 0 and a valid hash value.
        "" "
        genesis_block = Block (0, [], time.time (), "0")
        genesis_block.hash = genesis_block.compute_hash ()
        self.chain.append (genesis_block)

    @property
    def last_block (self):
        "" "
        A simple way to get the last block in a chain. Note the chain will always
        There is one block which is the genesis block
        "" "
        return self.chain [-1]

Now, if the contents of any previous blocks change:

  • The hash of the previous block will change.
  • This will result in mismatch with the previous_hash field in the next block.
  • Because the input data to calculate the hash value of any block also includes the previous_hash field, the hash value of the next block will also change.

In the end, the whole chain as a replacement block is disabled and the only way to fix it is to recalculate the entire chain.

Implementing the Proof-Of-Work algorithm (Proof of Work)

The problem, though, is that the hash value of all subsequent blocks can be easily recalculated to create another valid blockchain. To prevent this, we can exploit the asymmetry of the hash function we discussed above to make the task of calculating the hash value more difficult and random. What this means: Instead of accepting any hash value for the block, we add some constraints to it. Let’s add a constraint that our hash value will start with n leading zeros where n is a positive integer.

Here we will add some dummy data that we can change. We will add a new field which is the nonce field . The nonce number is a number that we can keep changing until we have a constrained hash function. The fact that the nonce satisfies the constraints acts as evidence that some calculations have been performed. This technique is a simplified version of the Hashcash algorithm used in Bitcoin. The number of zeroes specified in the constraint determines the difficulty of the PoW algorithm (the greater the number of zeroes, the harder it is to find nonce).

Also due to the asymmetry, Proof-of-work (PoW) is difficult to calculate but very easy to verify once you find the nonce (you just need to run the hash function again):

Blockchain class:
     # Difficulty level of the POW algorithm
     difficulty = 2

     ......

     def proof_of_work (self, block):
         "" "
         The function tests different values of nonce to get a satisfied hash value
         "" "
         block.nonce = 0

         computed_hash = block.compute_hash ()
         while not computed_hash.startswith ('0' * Blockchain.difficulty):
             block.nonce + = 1
             computed_hash = block.compute_hash ()

         return computed_hash

Add blocks to the Chain

To add a block to the chain, we must first verify that:

  • The data is not tampered with (proof of work provided is accurate).
  • The order of transactions is kept the same.
Blockchain class:

    .....

     def add_block (self, block, proof):
         "" "
         Function to add blocks to the chain after verification. Verification includes:
         * Check if proof is right.
         * The previous_hash field is correct and is the hash value of the block
         Newest in chain.
         "" "
         previous_hash = self.last_block.hash

         if previous_hash! = block.previous_hash:
             return False

         if not Blockchain.is_valid_proof (block, proof):
             return False

         block.hash = proof
         self.chain.append (block)
         return True

     def is_valid_proof (self, block, block_hash):
         "" "
         Checks whether block_hash is the valid hash value of the block or not
         and meet the difficulty criteria.
         "" "
         return (block_hash.startswith ('0' * Blockchain.difficulty) and
                 block_hash == block.compute_hash ())

Mining

The original transactions will be stored as a group of unconfirmed transactions. The process of putting unconfirmed transactions into a block and calculating the POW is called Mining blocks. When the nonce that meets the constraints is found, we can say that a block has been mined and it can be put into the blockchain.

In most cryptocurrencies (including Bitcoin), miners can be given a number of cryptocurrencies as a reward for using their computational power to calculate POW. Here is the function mining:

Blockchain class:

     def __init __ (self):
         self.unconfirmed_transactions = [] # list of transactions not yet confirmed
         self.chain = []
         self.create_genesis_block ()

     ....

     def add_new_transaction (self, transaction):
         self.unconfirmed_transactions.append (transaction)

     def mine (self):
         "" "
         The function adds transactions pending
          into the blockchain by adding them to the block and releasing PoW.
         "" "
         if not self.unconfirmed_transactions:
             return False

         last_block = self.last_block

         new_block = Block (index = last_block.index + 1,
                           transactions = self.unconfirmed_transactions,
                           timestamp = time.time (),
                           previous_hash = last_block.hash)

         proof = self.proof_of_work (new_block)
         self.add_block (new_block, proof)
         self.unconfirmed_transactions = []
         return new_block.index

Ok this is the code of the processing: source code

Create the interface

In this section, we will create the interface for the blockchain node to interact with the application we have just built. Here will use a popular Python microframework called Flask to create interactive REST APIs and invoke various actions in our blockchain node. If you’ve worked with any previous web framework, the code below won’t be hard to follow.

from flask import Flask, request
import requests

# Run at flask application
app = Flask (__ name__)

# Initialize the Blockchain object
blockchain = Blockchain ()

We need a route for the application to create a new transaction and this is the post

@ app.route ('/ new_transaction', methods = ['POST'])
def new_transaction ():
     tx_data = request.get_json ()
     required_fields = ["author", "content"]

     for field in required_fields:
         if not tx_data.get (field):
             return "Invalid transaction data", 404

     tx_data ["timestamp"] = time.time ()

     blockchain.add_new_transaction (tx_data)

     return "Success", 201

There will be a route that returns a copy of the chain. And this route the application will use to query all data to display:

@ app.route('/ chain', methods = ['GET'])
def get_chain():
     chain_data = []
     for block in blockchain.chain:
         chain_data.append(block .__ dict__)
     return json.dumps({"length": len(chain_data),
                        "chain": chain_data})

And here is the route to send a mine request - validating unverified transactions (if any). We will use it to start a mine command from our own application:

@app.route('/mine', methods=['GET'])
def mine_unconfirmed_transactions():
    result = blockchain.mine()
    if not result:
        return "No transactions to mine"
    return "Block #{} is mined.".format(result)

@app.route('/pending_tx')
def get_pending_tx():
    return json.dumps(blockchain.unconfirmed_transactions)

Establish a consensus and dispersion mechanism

There is a blockchain problem that we have deployed running on a computer. Although we have linked the blocks with the hash value and applied POW, it is still not possible to trust a single entity (in this case, a single machine). We need distributed data or multiple nodes to maintain the blockchain. So, to move from a single node to a peer-to-peer network, let’s first create a mechanism so that a new node can learn about other peers in the network:

# Contains the host address of other network participants
peers = set()

# route adds a new peer to the network
@app.route('/register_node', methods=['POST'])
def register_new_peers():
    # Host address to peer nodes 
    node_address = request.get_json()["node_address"]
    if not node_address:
        return "Invalid data", 400

    # Add node addresses to the list
    peers.add(node_address)

    # Return the new blockchain
    return get_chain()

@app.route('/register_with', methods=['POST'])
def register_with_existing_node():
    """
    Internally call the route `register_node` to
    Register the current node with the remote node specified in
    request and update the blockchain network
    """
    node_address = request.get_json()["node_address"]
    if not node_address:
        return "Invalid data", 400

    data = {"node_address": request.host_url}
    headers = {'Content-Type': "application/json"}

    # Reuqest register with the remote node and get information
    response = requests.post(node_address + "/register_node",
                             data=json.dumps(data), headers=headers)

    if response.status_code == 200:
        global blockchain
        global peers
        # update chain and peers
        chain_dump = response.json()['chain']
        blockchain = create_chain_from_dump(chain_dump)
        peers.update(response.json()['peers'])
        return "Registration successful", 200
    else:
        # If an error occurs, the API will return a response
        return response.content, response.status_code

def create_chain_from_dump(chain_dump):
    blockchain = Blockchain ()
    for idx, block_data in enumerate(chain_dump):
        block = Block(block_data["index"],
                      block_data["transactions"],
                      block_data["timestamp"],
                      block_data["previous_hash"])
        proof = block_data['hash']
        if idx > 0:
            added = blockchain.add_block(block, proof)
            if not added:
                raise Exception("The chain dump is tampered!!")
        else: # this block is a genesis block so no verification is needed
            blockchain.chain.append(block)
    return blockchain

A new node joining the network can call the function register_with_existing_node to register with existing nodes in the network. This will help the following:

  • Request the remote node to add a new peer to the existing peer list.
  • Easily Initialize blockchain of new node by retrieving remote node.
  • Re-sync the blockchain to the network if that node is no longer connected to the network.

However, there is a problem with many nodes due to intentional or unintentional reasons (such as network latency), the chain copy of several nodes may be different. In that case the nodes need to agree with some version of the chain to maintain the integrity of the entire system. In other words, we need to reach a consensus.

A simple consensus algorithm can agree with the longest valid chain when the chains of participating nodes in the network appear forked. The reason behind this method is that the longest chain proves the most work done (remember PoW is very fastidious):

class Blockchain

   ....

    def check_chain_validity(cls, chain):
        """
        A helper function to check if the entire blockchain is correct
        """
        result = True
        previous_hash = "0"

        # Repeat over all the blocks
        for block in chain:
            block_hash = block.hash
            # delete the hashed field to recalculate the hash value
            # use the `compute_hash` method.
            delattr(block, "hash")

            if not cls.is_valid_proof(block, block.hash) or \
                    previous_hash != block.previous_hash:
                result = False
                break

            block.hash, previous_hash = block_hash, block_hash

        return result

def consensus():
    """
   The simple consensus algorithm there is if a valid string is longer than possible
   Found, our chain will replace it.
    """
    global blockchain

    longest_chain = None
    current_len = len(blockchain.chain)

    for node in peers:
        response = requests.get('{}/chain'.format(node))
        length = response.json()['length']
        chain = response.json()['chain']
        if length > current_len and blockchain.check_chain_validity(chain):
              # Longer valid chain found!
            current_len = length
            longest_chain = chain

    if longest_chain:
        blockchain = longest_chain
        return True

    return False

Next we need to develop a way for any node to inform the network that it has mined a block so that everyone can update their blockchain and move on to mining other blocks. Other nodes may only need to verify the PoW and add the newly mined block to their respective chain (remember that verification is easy when knowing the nonce):

# This route to add another block just mined.
# The block needs to be verified first and then added to the chain
@app.route('/add_block', methods=['POST'])
def verify_and_add_block():
    block_data = request.get_json()
    block = Block(block_data["index"],
                  block_data["transactions"],
                  block_data["timestamp"],
                  block_data["previous_hash"])

    proof = block_data['hash']
    added = blockchain.add_block(block, proof)

    if not added:
        return "The block was discarded by the node", 400

    return "Block added to the chain", 201

def announce_new_block(block):
    """
    A function informs the network after a block has been mined.
    Other blocks can only verify PoW and add it
    corresponding string.
    """
    for peer in peers:
        url = "{}add_block".format(peer)
        requests.post(url, data=json.dumps(block.__dict__, sort_keys=True))

The function announce_new_block should be called after each block is mined by nodes so other peers can add it to their chain.

@app.route('/mine', methods=['GET'])
def mine_unconfirmed_transactions():
    result = blockchain.mine()
    if not result:
        return "No transactions to mine"
    else:
        # Make sure we have the longest chain before communicating with the network
        chain_length = len(blockchain.chain)
        consensus()
        if chain_length == len(blockchain.chain):
            # recently announced block notifications to the network
            announce_new_block(blockchain.last_block)
        return "Block #{} is mined.".format(blockchain.last_block.index

Build web application

So the blockchain server is set up. You can see the source code here .

Now it’s time to start developing the web application interface. We have used the Jinja2 template to display views and some CSS to make things look good.

The application needs to connect to a node in the blockchain network to fetch data and also to send new data.

import datetime
import json

import requests
from flask import render_template, redirect, request

from app import app

# Node that we will connect to fetch data about
CONNECTED_NODE_ADDRESS = "http://127.0.0.1:8000"

posts = []

Function fetch_posts:

def fetch_posts():
    """
    Get chain function from node, data analysis and local storage
    """
    get_chain_address = "{}/chain".format(CONNECTED_NODE_ADDRESS)
    response = requests.get(get_chain_address)
    if response.status_code == 200:
        content = []
        chain = json.loads(response.content)
        for block in chain["chain"]:
            for tx in block["transactions"]:
                tx["index"] = block["index"]
                tx["hash"] = block["previous_hash"]
                content.append(tx)

        global posts
        posts = sorted(content,
                       key=lambda k: k['timestamp'],
                       reverse=True)

The application has an HTML form to enter the user input and then makes a POST request to the connected node to add transactions to the unconfirmed transactions group. The transaction is then mined by the network and finally fetched after reloading the page:

@app.route('/submit', methods=['POST'])
def submit_textarea():
    """
    Route to create a new transaction
    """
    post_content = request.form["content"]
    author = request.form["author"]

    post_object = {
        'author': author,
        'content': post_content,
    }

    # Submit transaction
    new_tx_address = "{}/new_transaction".format(CONNECTED_NODE_ADDRESS)

    requests.post(new_tx_address,
                  json=post_object,
                  headers={'Content-type': 'application/json'})

    # Return to homepage
    return redirect('/')

Run the application

Here we have it! source code

Place to place:

$ cd python_blockchain_app
$ pip install -r requirements.txt

Run a blockchain node on port 8000

$ export FLASK_APP=node_server.py
$ flask run --port 8000

run the web application:

$ python run_app.py

The application will run at http: // localhost: 5000 .

Run multiple nodes

We will run multiple nodes by running on different ports. Use it register_withto register as a peer network:

# already running
$ flask run --port 8000 &
# spinning up new nodes
$ flask run --port 8001 &
$ flask run --port 8002 &
$ curl -X POST \
  http://127.0.0.1:8001/register_with \
  -H 'Content-Type: application/json' \
  -d '{"node_address": "http://127.0.0.1:8000"}'

$ curl -X POST \
  http://127.0.0.1:8002/register_with \
  -H 'Content-Type: application/json' \
  -d '{"node_address": "http://127.0.0.1:8000"}'

You can run the application (python run_app.py) and create transactions (post via the web interface) and when you mined transactions, all nodes in the network will update the chain. And nodes can also be checked by calling with cURL or Postman.

$ curl -X GET http://localhost:8001/chain
$ curl -X GET http://localhost:8002/chain

Authenticate transactions

You may have noticed an application flaw: Anyone can change the name and post any content. Also, posts are easily fake while sending transactions on the blockchain network. One way to solve this is to create a user account, using the public key cryptography. Each new user needs a public key and a private key to post in the application:

  • Each newly sent transaction (post post) is signed with the user’s private key. This signature is added to the transaction data along with the user information.
  • During the verification phase when mining transactions, we can verify that the requested owner of the post is the same as the owner specified in the transaction data and the message is not modified if incorrect. This can be done using the signature and the public key of the owner who submits the post.

conclude

This tutorial covers the fundamentals of a public blockchain. If you’ve followed that up, you can now deploy a blockchain from scratch and build a simple application that allows users to share information on the blockchain. Hopefully the article will bring you useful and very interesting knowledge and see you in the next article.

#python #programming #blockchain

Building A Blockchain Network From scratch With Python
49.20 GEEK