MuleSoft (Event-Based Process) With Enhanced Execution Engine - DZone Microservices

To opt benefits of asynchronous processes, MuleSoft has introduced thread management which has no dependency on the configuration. It works based on available resources on platforms.

MuleSoft follows and recommends an asynchronous process hence came with an event-based approach along with an asynchronous approach. It allows the application to be more responsive by not waiting for long processes like your I/O processes to finish.

Below is a major difference to follow event-based (Reactive Programming) pattern:

• Reactive Programming — Focusing on computation through ephemeral data streams, tend to be event-driven.
• Messages have a unique clear destination, while Events are made for others to observe.

With definition**:**

Reactive Systems: Defined by the Reactive Manifesto — is a set of architectural design principles for building modern systems that are well prepared to meet the increasing demands that applications face today.

**Proactor Pattern: **A fully asynchronous design pattern made for handling events. It’s very much like multi-threaded programming, without the need for you to think of thread management.

• Pros and Cons (Proactor Pattern)
• The obvious benefit is the asynchronous execution. As mentioned, it allows the application to be more responsive by not waiting for long processes like your I/O processes to finish. Also, with a centralized thread pooling and dispatching mechanism, there is no need for the user of the Proactor implementation to deal with thread management directly. The implementation can be further improved so that the thread pool can grow and shrink dynamically based on the available physical resources and the number of pending tasks. Tasks can also be made to queue based on priority.
• As for the downsides, like many asynchronous or multi-threaded paradigms, debugging can be quite a hassle. Although usually not needed, you may have to consider thread synchronization when multiple tasks use a shared resource (and this increases the risk of getting deadlocks, starving away your thread pool). For a small application, I feel that the Proactor pattern may add unnecessary complexity to the overall architecture.

**Reactive Programming: **It has a major benefit in event-driven programming… as follows:

It is the flow of data rather than the flow of control;

• This is information that drives the logic forward rather than having control flow driven by a thread-of-execution.
• It supports decomposing to execute an asynchronous and non-blocking fashion.
• Allows for non-blocking execution.

The Benefits (And Limitations) Of Reactive Programming;

• Increased utilization of computing resources on multi-core and multi-CPU hardware and increased performance by reducing serialization points.
• Developer productivity as a straightforward and maintainable approach to dealing with asynchronous and non-blocking computation and IO.
• Inclusion of back-pressure is crucial to avoid over-utilization or rather unbounded consumption of resources

The primary benefits of Reactive Programming are:

• Increased utilization of computing resources on multi-core and multi-CPU hardware; and increased performance by reducing serialization points.
• Developer productivity as a straightforward and maintainable approach to dealing with asynchronous and non-blocking computation and IO.
• Inclusion of back-pressure is crucial to avoid over-utilization or rather unbounded consumption of resources

MuleSoft Threads — Uber vs. Dedicated

MuleSoft introduced an approach to self manages approach for thread availability and executions. In the current release, MuleSoft has two major thread pooling approach as below.

UBER: It is a unified scheduling strategy to manage threads and its allocation to process events. All individual thread pool (cpu_light, cpu_intensive, and I/O) backed by Uber thread pool. In Uber, type of threads same as previous processes, it only follows an algorithm to support the optimum performance of threads in processing.

DEDICATED: It is a self thread managed thread pool and allocates based on component process requirements.

NOTE: Both approaches follow the strategy to best utilization of thread management in Mulesoft runtime. This strategy works on existing threads (cpu_light, cpu_intensive, and I/O).

Recommended: Always run mule using the UBER strategy.

#microservices #mulesoft #reactive programming #mule 4 #proactor pattern #reactive system

What is GEEK

Buddha Community

PySQL Tutorial: A Database Framework for Python

PySQL 

PySQL is database framework for Python (v3.x) Language, Which is based on Python module mysql.connector, this module can help you to make your code more short and more easier. Before using this framework you must have knowledge about list, tuple, set, dictionary because all codes are designed using it. It's totally free and open source.

Tutorial Video in English (Watch Now)

Installation

Before we said that this framework is based on mysql.connector so you have to install mysql.connector first on your system. Then you can import pysql and enjoy coding!

python -m pip install mysql-connector-python

After Install mysql.connector successfully create Python file download/install pysql on the same dir where you want to create program. You can clone is using git or npm command, and you can also downlaod manually from repository site.

PyPi Command

Go to https://pypi.org/project/pysql-framework/ or use command

pip install pysql-framework

Git Command

git clone https://github.com/rohit-chouhan/pysql

Npm Command

Go to https://www.npmjs.com/package/pysql or use command

$ npm i pysql

Snippet Extention for VS Code

Install From Here https://marketplace.visualstudio.com/items?itemName=rohit-chouhan.pysql

Table of contents

• Connecting a Server
• Create a Database in Server
• Drop Database
• Connecting a Database
• Creating Table in Database
• Drop Table in Database
• Selecting data from Table
• Add New Column to Table
• Modify Column to Table
• Drop Column from Table
• Manual Execute Query
• Inserting data
• Updating data
• Deleting data

Connecting a Server

---

To connect a database with localhost server or phpmyadmin, use connect method to establish your python with database server.

import pysql

db = pysql.connect(
    "host",
    "username",
    "password"
 )

Create a Database in Server

---

Creating database in server, to use this method

import pysql

db = pysql.connect(
    "host",
    "username",
    "password"
 )
 pysql.createDb(db,"demo")
 #execute: CREATE DATABASE demo

Drop Database

---

To drop database use this method .

Syntex Code -

pysql.dropDb([connect_obj,"table_name"])

Example Code -

pysql.dropDb([db,"demo"])
#execute:DROP DATABASE demo

Connecting a Database

---

To connect a database with localhost server or phpmyadmin, use connect method to establish your python with database server.

import pysql

db = pysql.connect(
    "host",
    "username",
    "password",
    "database"
 )

Creating Table in Database

---

To create table in database use this method to pass column name as key and data type as value.

Syntex Code -

pysql.createTable([db,"table_name_to_create"],{
    "column_name":"data_type", 
    "column_name":"data_type"
})

Example Code -

pysql.createTable([db,"details"],{
    "id":"int(11) primary", 
     "name":"text", 
    "email":"varchar(50)",
    "address":"varchar(500)"
})

2nd Example Code -

Use can use any Constraint with Data Value

pysql.createTable([db,"details"],{
    "id":"int NOT NULL PRIMARY KEY", 
     "name":"varchar(20) NOT NULL", 
    "email":"varchar(50)",
    "address":"varchar(500)"
})

Drop Table in Database

---

To drop table in database use this method .

Syntex Code -

pysql.dropTable([connect_obj,"table_name"])

Example Code -

pysql.dropTable([db,"users"])
#execute:DROP TABLE users

Selecting data from Table

---

For Select data from table, you have to mention the connector object with table name. pass column names in set.

Syntex For All Data (*)-

records = pysql.selectAll([db,"table_name"])
for x in records:
  print(x)

Example - -

records = pysql.selectAll([db,"details"])
for x in records:
  print(x)
#execute: SELECT * FROM details

Syntex For Specific Column-

records = pysql.select([db,"table_name"],{"column","column"})
for x in records:
  print(x)

Example - -

records = pysql.select([db,"details"],{"name","email"})
for x in records:
  print(x)
#execute: SELECT name, email FROM details

Syntex Where and Where Not-

#For Where Column=Data
records = pysql.selectWhere([db,"table_name"],{"column","column"},("column","data"))

#For Where Not Column=Data (use ! with column)
records = pysql.selectWhere([db,"table_name"],{"column","column"},("column!","data"))
for x in records:
  print(x)

Example - -

records = pysql.selectWhere([db,"details"],{"name","email"},("county","india"))
for x in records:
  print(x)
#execute: SELECT name, email FROM details WHERE country='india'

Add New Column to Table

---

To add column in table, use this method to pass column name as key and data type as value. Note: you can only add one column only one call

Syntex Code -

pysql.addColumn([db,"table_name"],{
    "column_name":"data_type"
})

Example Code -

pysql.addColumn([db,"details"],{
    "email":"varchar(50)"
})
#execute: ALTER TABLE details ADD email varchar(50);

Modify Column to Table

---

To modify data type of column table, use this method to pass column name as key and data type as value.

Syntex Code -

pysql.modifyColumn([db,"table_name"],{
    "column_name":"new_data_type"
})

Example Code -

pysql.modifyColumn([db,"details"],{
    "email":"text"
})
#execute: ALTER TABLE details MODIFY COLUMN email text;

Drop Column from Table

---

Note: you can only add one column only one call

Syntex Code -

pysql.dropColumn([db,"table_name"],"column_name")

Example Code -

pysql.dropColumn([db,"details"],"name")
#execute: ALTER TABLE details DROP COLUMN name

Manual Execute Query

---

To execute manual SQL Query to use this method.

Syntex Code -

pysql.query(connector_object,your_query)

Example Code -

pysql.query(db,"INSERT INTO users (name) VALUES ('Rohit')")

Inserting data

---

For Inserting data in database, you have to mention the connector object with table name, and data as sets.

Syntex -

data =     {
    "db_column":"Data for Insert",
    "db_column":"Data for Insert"
}
pysql.insert([db,"table_name"],data)

Example Code -

data =     {
    "name":"Komal Sharma",
    "contry":"India"
}
pysql.insert([db,"users"],data)

Updating data

---

For Update data in database, you have to mention the connector object with table name, and data as tuple.

Syntex For Updating All Data-

data = ("column","data to update")
pysql.updateAll([db,"users"],data)

Example - -

data = ("name","Rohit")
pysql.updateAll([db,"users"],data)
#execute: UPDATE users SET name='Rohit'

Syntex For Updating Data (Where and Where Not)-

data = ("column","data to update")
#For Where Column=Data
where = ("column","data")

#For Where Not Column=Data (use ! with column)
where = ("column!","data")
pysql.update([db,"users"],data,where)

Example -

data = ("name","Rohit")
where = ("id",1)
pysql.update([db,"users"],data,where)
#execute: UPDATE users SET name='Rohit' WHERE id=1

Deleting data

---

For Delete data in database, you have to mention the connector object with table name.

Syntex For Delete All Data-

pysql.deleteAll([db,"table_name"])

Example - -

pysql.deleteAll([db,"users"])
#execute: DELETE FROM users

Syntex For Deleting Data (Where and Where Not)-

where = ("column","data")

pysql.delete([db,"table_name"],where)

Example -

#For Where Column=Data
where = ("id",1)

#For Where Not Column=Data (use ! with column)
where = ("id!",1)
pysql.delete([db,"users"],where)
#execute: DELETE FROM users WHERE id=1

--- Finish ---

Change Logs

[19/06/2021]
 - ConnectSever() removed and merged to Connect()
 - deleteAll() [Fixed]
 - dropTable() [Added]
 - dropDb() [Added]
 
[20/06/2021]
 - Where Not Docs [Added]

The module is designed by Rohit Chouhan, contact us for any bug report, feature or business inquiry.

Author: rohit-chouhan
Source Code: https://github.com/rohit-chouhan/pysql
License: Apache-2.0 License

#python 

MuleSoft (Event-Based Process) With Enhanced Execution Engine - DZone Microservices

To opt benefits of asynchronous processes, MuleSoft has introduced thread management which has no dependency on the configuration. It works based on available resources on platforms.

MuleSoft follows and recommends an asynchronous process hence came with an event-based approach along with an asynchronous approach. It allows the application to be more responsive by not waiting for long processes like your I/O processes to finish.

Below is a major difference to follow event-based (Reactive Programming) pattern:

• Reactive Programming — Focusing on computation through ephemeral data streams, tend to be event-driven.
• Messages have a unique clear destination, while Events are made for others to observe.

With definition**:**

Reactive Systems: Defined by the Reactive Manifesto — is a set of architectural design principles for building modern systems that are well prepared to meet the increasing demands that applications face today.

**Proactor Pattern: **A fully asynchronous design pattern made for handling events. It’s very much like multi-threaded programming, without the need for you to think of thread management.

• Pros and Cons (Proactor Pattern)
• The obvious benefit is the asynchronous execution. As mentioned, it allows the application to be more responsive by not waiting for long processes like your I/O processes to finish. Also, with a centralized thread pooling and dispatching mechanism, there is no need for the user of the Proactor implementation to deal with thread management directly. The implementation can be further improved so that the thread pool can grow and shrink dynamically based on the available physical resources and the number of pending tasks. Tasks can also be made to queue based on priority.
• As for the downsides, like many asynchronous or multi-threaded paradigms, debugging can be quite a hassle. Although usually not needed, you may have to consider thread synchronization when multiple tasks use a shared resource (and this increases the risk of getting deadlocks, starving away your thread pool). For a small application, I feel that the Proactor pattern may add unnecessary complexity to the overall architecture.

**Reactive Programming: **It has a major benefit in event-driven programming… as follows:

It is the flow of data rather than the flow of control;

• This is information that drives the logic forward rather than having control flow driven by a thread-of-execution.
• It supports decomposing to execute an asynchronous and non-blocking fashion.
• Allows for non-blocking execution.

The Benefits (And Limitations) Of Reactive Programming;

• Increased utilization of computing resources on multi-core and multi-CPU hardware and increased performance by reducing serialization points.
• Developer productivity as a straightforward and maintainable approach to dealing with asynchronous and non-blocking computation and IO.
• Inclusion of back-pressure is crucial to avoid over-utilization or rather unbounded consumption of resources

The primary benefits of Reactive Programming are:

• Increased utilization of computing resources on multi-core and multi-CPU hardware; and increased performance by reducing serialization points.
• Developer productivity as a straightforward and maintainable approach to dealing with asynchronous and non-blocking computation and IO.
• Inclusion of back-pressure is crucial to avoid over-utilization or rather unbounded consumption of resources

MuleSoft Threads — Uber vs. Dedicated

MuleSoft introduced an approach to self manages approach for thread availability and executions. In the current release, MuleSoft has two major thread pooling approach as below.

UBER: It is a unified scheduling strategy to manage threads and its allocation to process events. All individual thread pool (cpu_light, cpu_intensive, and I/O) backed by Uber thread pool. In Uber, type of threads same as previous processes, it only follows an algorithm to support the optimum performance of threads in processing.

DEDICATED: It is a self thread managed thread pool and allocates based on component process requirements.

NOTE: Both approaches follow the strategy to best utilization of thread management in Mulesoft runtime. This strategy works on existing threads (cpu_light, cpu_intensive, and I/O).

Recommended: Always run mule using the UBER strategy.

#microservices #mulesoft #reactive programming #mule 4 #proactor pattern #reactive system

MuleSoft (Event-Based Process) With Enhanced Execution Engine

To opt benefits of asynchronous processes, MuleSoft has introduced thread management which has no dependency on the configuration. It works based on available resources on platforms.

MuleSoft follows and recommends an asynchronous process hence came with an event-based approach along with an asynchronous approach. It allows the application to be more responsive by not waiting for long processes like your I/O processes to finish.

Below is a major difference to follow event-based (Reactive Programming) pattern:

• Reactive Programming — Focusing on computation through ephemeral data streams, tend to be event-driven.
• Messages have a unique clear destination, while Events are made for others to observe.

With definition**:**

Reactive Systems: Defined by the Reactive Manifesto — is a set of architectural design principles for building modern systems that are well prepared to meet the increasing demands that applications face today.

**Proactor Pattern: **A fully asynchronous design pattern made for handling events. It’s very much like multi-threaded programming, without the need for you to think of thread management.

• Pros and Cons (Proactor Pattern)
• The obvious benefit is the asynchronous execution. As mentioned, it allows the application to be more responsive by not waiting for long processes like your I/O processes to finish. Also, with a centralized thread pooling and dispatching mechanism, there is no need for the user of the Proactor implementation to deal with thread management directly. The implementation can be further improved so that the thread pool can grow and shrink dynamically based on the available physical resources and the number of pending tasks. Tasks can also be made to queue based on priority.
• As for the downsides, like many asynchronous or multi-threaded paradigms, debugging can be quite a hassle. Although usually not needed, you may have to consider thread synchronization when multiple tasks use a shared resource (and this increases the risk of getting deadlocks, starving away your thread pool). For a small application, I feel that the Proactor pattern may add unnecessary complexity to the overall architecture.

**Reactive Programming: **It has a major benefit in event-driven programming… as follows:

It is the flow of data rather than the flow of control;

• This is information that drives the logic forward rather than having control flow driven by a thread-of-execution.
• It supports decomposing to execute an asynchronous and non-blocking fashion.
• Allows for non-blocking execution.

The Benefits (And Limitations) Of Reactive Programming;

• Increased utilization of computing resources on multi-core and multi-CPU hardware and increased performance by reducing serialization points.
• Developer productivity as a straightforward and maintainable approach to dealing with asynchronous and non-blocking computation and IO.
• Inclusion of back-pressure is crucial to avoid over-utilization or rather unbounded consumption of resources

The primary benefits of Reactive Programming are:

• Increased utilization of computing resources on multi-core and multi-CPU hardware; and increased performance by reducing serialization points.
• Developer productivity as a straightforward and maintainable approach to dealing with asynchronous and non-blocking computation and IO.
• Inclusion of back-pressure is crucial to avoid over-utilization or rather unbounded consumption of resources

MuleSoft Threads — Uber vs. Dedicated

MuleSoft introduced an approach to self manages approach for thread availability and executions. In the current release, MuleSoft has two major thread pooling approach as below.

UBER: It is a unified scheduling strategy to manage threads and its allocation to process events. All individual thread pool (cpu_light, cpu_intensive, and I/O) backed by Uber thread pool. In Uber, type of threads same as previous processes, it only follows an algorithm to support the optimum performance of threads in processing.

DEDICATED: It is a self thread managed thread pool and allocates based on component process requirements.

NOTE: Both approaches follow the strategy to best utilization of thread management in Mulesoft runtime. This strategy works on existing threads (cpu_light, cpu_intensive, and I/O).

Recommended: Always run mule using the UBER strategy.

#microservices #mulesoft #reactive programming #mule 4 #reactive system #proactor pattern

MuleSoft (Event-Based Process) With Enhanced Execution Engine

To opt benefits of asynchronous processes, MuleSoft has introduced thread management which has no dependency on the configuration. It works based on available resources on platforms.

MuleSoft follows and recommends an asynchronous process hence came with an event-based approach along with an asynchronous approach. It allows the application to be more responsive by not waiting for long processes like your I/O processes to finish.

Below is a major difference to follow event-based (Reactive Programming) pattern:

• Reactive Programming — Focusing on computation through ephemeral data streams, tend to be event-driven.
• Messages have a unique clear destination, while Events are made for others to observe.

With definition**:**

Reactive Systems: Defined by the Reactive Manifesto — is a set of architectural design principles for building modern systems that are well prepared to meet the increasing demands that applications face today.

**Proactor Pattern: **A fully asynchronous design pattern made for handling events. It’s very much like multi-threaded programming, without the need for you to think of thread management.

• Pros and Cons (Proactor Pattern)
• The obvious benefit is the asynchronous execution. As mentioned, it allows the application to be more responsive by not waiting for long processes like your I/O processes to finish. Also, with a centralized thread pooling and dispatching mechanism, there is no need for the user of the Proactor implementation to deal with thread management directly. The implementation can be further improved so that the thread pool can grow and shrink dynamically based on the available physical resources and the number of pending tasks. Tasks can also be made to queue based on priority.
• As for the downsides, like many asynchronous or multi-threaded paradigms, debugging can be quite a hassle. Although usually not needed, you may have to consider thread synchronization when multiple tasks use a shared resource (and this increases the risk of getting deadlocks, starving away your thread pool). For a small application, I feel that the Proactor pattern may add unnecessary complexity to the overall architecture.

#microservices #mulesoft #reactive programming #mule 4 #proactor pattern

The Principles of Chaos Engineering

Resilience is something those who use Kubernetes to run apps and microservices in containers aim for. When a system is resilient, it can handle losing a portion of its microservices and components without the entire system becoming inaccessible.

Resilience is achieved by integrating loosely coupled microservices. When a system is resilient, microservices can be updated or taken down without having to bring the entire system down. Scaling becomes easier too, since you don’t have to scale the whole cloud environment at once.

That said, resilience is not without its challenges. Building microservices that are independent yet work well together is not easy. You also have to create and maintain a reliable system with high fault tolerance. This is where Chaos Engineering comes into play.

What Is Chaos Engineering?

Chaos Engineering has been around for almost a decade now but it is still a relevent and useful concept to incorporate into improving your whole systems architecture. In essence, Chaos Engineering is the process of triggering and injecting faults into a system deliberately. Instead of waiting for errors to occur, engineers can take deliberate steps to cause (or simulate) errors in a controlled environment.

Chaos Engineering allows for better, more advanced resilience testing. Developers can now experiment in cloud-native distributed systems. Experiments involve testing both the physical infrastructure and the cloud ecosystem.

Chaos Engineering is not a new approach. In fact, companies like Netflix have been using resilience testing through Chaos Monkey, an in-house Chaos Engineering framework designed to improve the strength of cloud infrastructure for years now.

When dealing with a large-scale distributed system, Chaos Engineering provides an empirical way of building confidence by anticipating faults instead of reacting to them. The chaotic condition is triggered intentionally for this purpose.

There are a lot of analogies depicting how Chaos Engineering works, but the traffic light analogy represents the concept best. Conventional testing is similar to testing traffic lights individually to make sure that they work.

Chaos Engineering, on the other hand, means closing out a busy array of intersections to see how traffic reacts to the chaos of losing traffic lights. Since the test is run deliberately, more insights can be collected from the process.

#devops #chaos engineering #high fault tolerance #microservice-based architecture #microservices #microservices architecture #resilience engineering