Stracke  Kiana

Stracke Kiana

1598754113

How to Build a Kubeflow Pipeline

Want to learn how to create an ML application from Kubeflow Pipelines? In this episode of Kubeflow 101, we show you how to build a Kubeflow Pipeline from the ML model we explored in the last episode. Moreover, we give you a walkthrough of how to create, test and deploy your ML application in Kubeflow Pipelines. Watch to learn how Kubeflow Pipelines can bring orchestration to complex workflows when working with ML applications.

Last episode → https://goo.gle/2FKcLNF

Timestamps:

0:00 - Intro

0:40 - Pipeline steps overview

1:09 - Import dependencies, define constants

1:45 - Download trainer data

2:10 - Train the model

3:04 - Deploy the model

3:22 - Define and submit the Kubeflow pipeline

3:56 - Compile and share the pipeline

4:26 - Conclusion

Watch more episodes of Kubeflow 101 → https://goo.gle/3cqY2lR

Subscribe to the GCP Channel → https://goo.gle/GCP

#kubeflow #programming

What is GEEK

Buddha Community

How to Build a Kubeflow Pipeline
Anthony  Dach

Anthony Dach

1623649980

Building An Automated Testing Pipeline with GoCD [Tutorial]

CI/CD enables developers, engineers and DevOps team to create a fast and effective process of packaging the product to market, thereby allowing them to stay ahead of the competition. When Selenium automation testing joins force with an effective CI/CD tool, it does wonders for the product delivery. GoCD is one such open-source Continuous Integration (CI) and Continuous Delivery (CD) tool developed by ThoughtWorks that supports the software development life cycle by enabling automation for the entire process. Right from development –. test –> deployment, GoCD ensures that your delivery cycles are on time, reliable, and efficient.

Ok. I know what you are thinking!

We have Jenkins for CI & CD ! Why a new tool ?

In this GoCD pipeline tutorial, we will deep dive into all the information you would need to set up a GoCD pipeline and in conjunction with the underlying concepts. You will also learn how to perform automation testing using Selenium in GoCD Pipeline through an Online Selenium Grid .

Why GoCD ?

Setting up GoCD

Setting up GoCD

Setting Up the GoCD Pipeline

#selenium testing #ci/cd #building an automated testing pipeline with gocd #gocd pipeline #build-test #gocd

Martin  Soit

Martin Soit

1600410108

Machine Learning Pipelines with Kubeflow

Why Machine Learning Pipelines?

A lot of attention is being given now to the idea of Machine Learning Pipelines, which are meant to automate and orchestrate the various steps involved in training a machine learning model; however, it’s not always made clear what the benefits are of modeling machine learning workflows as automated pipelines.

When tasked with training a new ML model, most Data Scientists and ML Engineers will probably start by developing some new Python scripts or interactive notebooks that perform the data extraction and preprocessing necessary to construct a clean set of data on which to train the model. Then, they might create several additional scripts or notebooks to try out different types of models or different machine learning frameworks. And finally, they’ll gather and explore metrics to evaluate how each model performed on a test dataset, and then determine which model to deploy to production.

#kubeflow-pipelines #kubernetes #kubeflow #machine-learning

The Best Way to Build a Chatbot in 2021

A useful tool several businesses implement for answering questions that potential customers may have is a chatbot. Many programming languages give web designers several ways on how to make a chatbot for their websites. They are capable of answering basic questions for visitors and offer innovation for businesses.

With the help of programming languages, it is possible to create a chatbot from the ground up to satisfy someone’s needs.

Plan Out the Chatbot’s Purpose

Before building a chatbot, it is ideal for web designers to determine how it will function on a website. Several chatbot duties center around fulfilling customers’ needs and questions or compiling and optimizing data via transactions.

Some benefits of implementing chatbots include:

  • Generating leads for marketing products and services
  • Improve work capacity when employees cannot answer questions or during non-business hours
  • Reducing errors while providing accurate information to customers or visitors
  • Meeting customer demands through instant communication
  • Alerting customers about their online transactions

Some programmers may choose to design a chatbox to function through predefined answers based on the questions customers may input or function by adapting and learning via human input.

#chatbots #latest news #the best way to build a chatbot in 2021 #build #build a chatbot #best way to build a chatbot

Riyad Amin

Riyad Amin

1571046022

Build Your Own Cryptocurrency Blockchain in Python

Cryptocurrency is a decentralized digital currency that uses encryption techniques to regulate the generation of currency units and to verify the transfer of funds. Anonymity, decentralization, and security are among its main features. Cryptocurrency is not regulated or tracked by any centralized authority, government, or bank.

Blockchain, a decentralized peer-to-peer (P2P) network, which is comprised of data blocks, is an integral part of cryptocurrency. These blocks chronologically store information about transactions and adhere to a protocol for inter-node communication and validating new blocks. The data recorded in blocks cannot be altered without the alteration of all subsequent blocks.

In this article, we are going to explain how you can create a simple blockchain using the Python programming language.

Here is the basic blueprint of the Python class we’ll use for creating the blockchain:

class Block(object):
    def __init__():
        pass
    #initial structure of the block class 
    def compute_hash():
        pass
    #producing the cryptographic hash of each block 
  class BlockChain(object):
    def __init__(self):
    #building the chain
    def build_genesis(self):
        pass
    #creating the initial block
    def build_block(self, proof_number, previous_hash):
        pass
    #builds new block and adds to the chain
   @staticmethod
    def confirm_validity(block, previous_block):
        pass
    #checks whether the blockchain is valid
    def get_data(self, sender, receiver, amount):
        pass
    # declares data of transactions
    @staticmethod
    def proof_of_work(last_proof):
        pass
    #adds to the security of the blockchain
    @property
    def latest_block(self):
        pass
    #returns the last block in the chain

Now, let’s explain how the blockchain class works.

Initial Structure of the Block Class

Here is the code for our initial block class:

import hashlib
import time
class Block(object):
    def __init__(self, index, proof_number, previous_hash, data, timestamp=None):
        self.index = index
        self.proof_number = proof_number
        self.previous_hash = previous_hash
        self.data = data
        self.timestamp = timestamp or time.time()
    @property
    def compute_hash(self):
        string_block = "{}{}{}{}{}".format(self.index, self.proof_number, self.previous_hash, self.data, self.timestamp)
        return hashlib.sha256(string_block.encode()).hexdigest()

As you can see above, the class constructor or initiation method ( init()) above takes the following parameters:

self — just like any other Python class, this parameter is used to refer to the class itself. Any variable associated with the class can be accessed using it.

index — it’s used to track the position of a block within the blockchain.

previous_hash — it used to reference the hash of the previous block within the blockchain.

data—it gives details of the transactions done, for example, the amount bought.

timestamp—it inserts a timestamp for all the transactions performed.

The second method in the class, compute_hash , is used to produce the cryptographic hash of each block based on the above values.

As you can see, we imported the SHA-256 algorithm into the cryptocurrency blockchain project to help in getting the hashes of the blocks.

Once the values have been placed inside the hashing module, the algorithm will return a 256-bit string denoting the contents of the block.

So, this is what gives the blockchain immutability. Since each block will be represented by a hash, which will be computed from the hash of the previous block, corrupting any block in the chain will make the other blocks have invalid hashes, resulting in breakage of the whole blockchain network.

Building the Chain

The whole concept of a blockchain is based on the fact that the blocks are “chained” to each other. Now, we’ll create a blockchain class that will play the critical role of managing the entire chain.

It will keep the transactions data and include other helper methods for completing various roles, such as adding new blocks.

Let’s talk about the helper methods.

Adding the Constructor Method

Here is the code:

class BlockChain(object):
    def __init__(self):
        self.chain = []
        self.current_data = []
        self.nodes = set()
        self.build_genesis()

The init() constructor method is what instantiates the blockchain.

Here are the roles of its attributes:

self.chain — this variable stores all the blocks.

self.current_data — this variable stores information about the transactions in the block.

self.build_genesis() — this method is used to create the initial block in the chain.

Building the Genesis Block

The build_genesis() method is used for creating the initial block in the chain, that is, a block without any predecessors. The genesis block is what represents the beginning of the blockchain.

To create it, we’ll call the build_block() method and give it some default values. The parameters proof_number and previous_hash are both given a value of zero, though you can give them any value you desire.

Here is the code:

def build_genesis(self):
        self.build_block(proof_number=0, previous_hash=0)
 def build_block(self, proof_number, previous_hash):
        block = Block(
            index=len(self.chain),
            proof_number=proof_number,
            previous_hash=previous_hash,
            data=self.current_data
        )
        self.current_data = []  
        self.chain.append(block)
        return block

Confirming Validity of the Blockchain

The confirm_validity method is critical in examining the integrity of the blockchain and making sure inconsistencies are lacking.

As explained earlier, hashes are pivotal for realizing the security of the cryptocurrency blockchain, because any slight alteration in an object will result in the creation of an entirely different hash.

Thus, the confirm_validity method utilizes a series of if statements to assess whether the hash of each block has been compromised.

Furthermore, it also compares the hash values of every two successive blocks to identify any anomalies. If the chain is working properly, it returns true; otherwise, it returns false.

Here is the code:

def confirm_validity(block, previous_block):
        if previous_block.index + 1 != block.index:
            return False
        elif previous_block.compute_hash != block.previous_hash:
            return False
        elif block.timestamp <= previous_block.timestamp:
            return False
        return True

Declaring Data of Transactions

The get_data method is important in declaring the data of transactions on a block. This method takes three parameters (sender’s information, receiver’s information, and amount) and adds the transaction data to the self.current_data list.

Here is the code:

def get_data(self, sender, receiver, amount):
        self.current_data.append({
            'sender': sender,
            'receiver': receiver,
            'amount': amount
        })
        return True

Effecting the Proof of Work

In blockchain technology, Proof of Work (PoW) refers to the complexity involved in mining or generating new blocks on the blockchain.

For example, the PoW can be implemented by identifying a number that solves a problem whenever a user completes some computing work. Anyone on the blockchain network should find the number complex to identify but easy to verify — this is the main concept of PoW.

This way, it discourages spamming and compromising the integrity of the network.

In this article, we’ll illustrate how to include a Proof of Work algorithm in a blockchain cryptocurrency project.

Finalizing With the Last Block

Finally, the latest_block() helper method is used for retrieving the last block on the network, which is actually the current block.

Here is the code:

def latest_block(self):
        return self.chain[-1]

Implementing Blockchain Mining

Now, this is the most exciting section!

Initially, the transactions are kept in a list of unverified transactions. Mining refers to the process of placing the unverified transactions in a block and solving the PoW problem. It can be referred to as the computing work involved in verifying the transactions.

If everything has been figured out correctly, a block is created or mined and joined together with the others in the blockchain. If users have successfully mined a block, they are often rewarded for using their computing resources to solve the PoW problem.

Here is the mining method in this simple cryptocurrency blockchain project:

def block_mining(self, details_miner):
            self.get_data(
            sender="0", #it implies that this node has created a new block
            receiver=details_miner,
            quantity=1, #creating a new block (or identifying the proof number) is awarded with 1
        )
        last_block = self.latest_block
        last_proof_number = last_block.proof_number
        proof_number = self.proof_of_work(last_proof_number)
        last_hash = last_block.compute_hash
        block = self.build_block(proof_number, last_hash)
        return vars(block)

Summary

Here is the whole code for our crypto blockchain class in Python:

import hashlib
import time
class Block(object):
    def __init__(self, index, proof_number, previous_hash, data, timestamp=None):
        self.index = index
        self.proof_number = proof_number
        self.previous_hash = previous_hash
        self.data = data
        self.timestamp = timestamp or time.time()
    @property
    def compute_hash(self):
        string_block = "{}{}{}{}{}".format(self.index, self.proof_number, self.previous_hash, self.data, self.timestamp)
        return hashlib.sha256(string_block.encode()).hexdigest()
    def __repr__(self):
        return "{} - {} - {} - {} - {}".format(self.index, self.proof_number, self.previous_hash, self.data, self.timestamp)
class BlockChain(object):
    def __init__(self):
        self.chain = []
        self.current_data = []
        self.nodes = set()
        self.build_genesis()
    def build_genesis(self):
        self.build_block(proof_number=0, previous_hash=0)
    def build_block(self, proof_number, previous_hash):
        block = Block(
            index=len(self.chain),
            proof_number=proof_number,
            previous_hash=previous_hash,
            data=self.current_data
        )
        self.current_data = []  
        self.chain.append(block)
        return block
    @staticmethod
    def confirm_validity(block, previous_block):
        if previous_block.index + 1 != block.index:
            return False
        elif previous_block.compute_hash != block.previous_hash:
            return False
        elif block.timestamp <= previous_block.timestamp:
            return False
        return True
    def get_data(self, sender, receiver, amount):
        self.current_data.append({
            'sender': sender,
            'receiver': receiver,
            'amount': amount
        })
        return True        
    @staticmethod
    def proof_of_work(last_proof):
        pass
    @property
    def latest_block(self):
        return self.chain[-1]
    def chain_validity(self):
        pass        
    def block_mining(self, details_miner):       
        self.get_data(
            sender="0", #it implies that this node has created a new block
            receiver=details_miner,
            quantity=1, #creating a new block (or identifying the proof number) is awared with 1
        )
        last_block = self.latest_block
        last_proof_number = last_block.proof_number
        proof_number = self.proof_of_work(last_proof_number)
        last_hash = last_block.compute_hash
        block = self.build_block(proof_number, last_hash)
        return vars(block)  
    def create_node(self, address):
        self.nodes.add(address)
        return True
    @staticmethod
    def get_block_object(block_data):        
        return Block(
            block_data['index'],
            block_data['proof_number'],
            block_data['previous_hash'],
            block_data['data'],
            timestamp=block_data['timestamp']
        )
blockchain = BlockChain()
print("GET READY MINING ABOUT TO START")
print(blockchain.chain)
last_block = blockchain.latest_block
last_proof_number = last_block.proof_number
proof_number = blockchain.proof_of_work(last_proof_number)
blockchain.get_data(
    sender="0", #this means that this node has constructed another block
    receiver="LiveEdu.tv", 
    amount=1, #building a new block (or figuring out the proof number) is awarded with 1
)
last_hash = last_block.compute_hash
block = blockchain.build_block(proof_number, last_hash)
print("WOW, MINING HAS BEEN SUCCESSFUL!")
print(blockchain.chain)

Now, let’s try to run our code to see if we can generate some digital coins…

Wow, it worked!

Conclusion

That is it!

We hope that this article has assisted you to understand the underlying technology that powers cryptocurrencies such as Bitcoin and Ethereum.

We just illustrated the basic ideas for making your feet wet in the innovative blockchain technology. The project above can still be enhanced by incorporating other features to make it more useful and robust.

Learn More

Thanks for reading !

Do you have any comments or questions? Please share them below.

#python #cryptocurrency

Franz  Becker

Franz Becker

1620057780

How to Create Custom Azure Pipelines Build Agent Fully Automated

Currently, most software teams that are using Azure DevOps to build their software choose to use Azure-hosted agents as their build server. This comes with a lot of benefits. Many libraries are already installed on these agents so it’s quite compatible with most projects’ specifications. For every build, you get a new agent that is provisioned and after the build will be destroyed. However, this has its own limitations. Azure Hosted Agents are using a specific type of VM so it is not possible to increase hardware specifications. If your projects also have libraries that are not supported by Azure Hosted agents, you need to install them on the agents. This might not be an issue for projects that have started in recent years, but if you have a legacy code that takes an hour to build then you might start to think about how you can speed up the build process. For such a situation, there are a lot of solutions like reconsidering the architecture of software and so on which is not the scope of this article. In this article, I am going to focus on how you can create a custom self-hosted agent.

#azure #azure devops #cicd pipeline #build pipeline