1671438340
Python Tutorial: Easier Experimenting
In this Python article, we will learn about Easier Experimenting in Python. When we work on a machine learning project, we quite often need to experiment with multiple alternatives. Some features in Python allow us to try out different options without much effort. In this tutorial, we are going to see some tips to make our experiments faster.
After finishing this tutorial, you will learn:
- How to leverage a duck-typing feature to easily swap functions and objects
- How making components into drop-in replacements for each other can help experiments run faster
Kick-start your project with my new book Python for Machine Learning, including step-by-step tutorials and the Python source code files for all examples.
Let’s get started.
Overview
This tutorial is in three parts; they are:
- Workflow of a machine learning project
- Functions as objects
- Caveats
Workflow of a Machine Learning Project
Consider a very simple machine learning project as follows:
from pandas import read_csv
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
# Load dataset
url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/iris.csv"
names = ['sepal-length', 'sepal-width', 'petal-length', 'petal-width', 'class']
dataset = read_csv(url, names=names)
# Split-out validation dataset
array = dataset.values
X = array[:,0:4]
y = array[:,4]
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.20, random_state=1, shuffle=True)
# Train
clf = SVC()
clf.fit(X_train, y_train)
# Test
score = clf.score(X_val, y_val)
print("Validation accuracy", score)This is a typical machine learning project workflow. We have a stage of preprocessing the data, then training a model, and afterward, evaluating our result. But in each step, we may want to try something different. For example, we may wonder if normalizing the data would make it better. So we may rewrite the code above into the following:
from pandas import read_csv
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
# Load dataset
url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/iris.csv"
names = ['sepal-length', 'sepal-width', 'petal-length', 'petal-width', 'class']
dataset = read_csv(url, names=names)
# Split-out validation dataset
array = dataset.values
X = array[:,0:4]
y = array[:,4]
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.20, random_state=1, shuffle=True)
# Train
clf = Pipeline([('scaler',StandardScaler()), ('classifier',SVC())])
clf.fit(X_train, y_train)
# Test
score = clf.score(X_val, y_val)
print("Validation accuracy", score)So far, so good. But what if we keep experimenting with different datasets, different models, or different score functions? Each time, we keep flipping between using a scaler and not would mean a lot of code change, and it would be quite easy to make mistakes.
Because Python supports duck typing, we can see that the following two classifier models implemented the same interface:
clf = SVC()
clf = Pipeline([('scaler',StandardScaler()), ('classifier',SVC())])Therefore, we can simply select between these two version and keep everything intact. We can say these two models are drop-in replacements for each other.
Making use of this property, we can create a toggle variable to control the design choice we make:
USE_SCALER = True
if USE_SCALER:
clf = Pipeline([('scaler',StandardScaler()), ('classifier',SVC())])
else:
clf = SVC()By toggling the variable USE_SCALER between True and False, we can select whether a scaler should be applied. A more complex example would be to select among different scaler and the classifier models, such as:
SCALER = "standard"
CLASSIFIER = "svc"
if CLASSIFIER == "svc":
model = SVC()
elif CLASSIFIER == "cart":
model = DecisionTreeClassifier()
else:
raise NotImplementedError
if SCALER == "standard":
clf = Pipeline([('scaler',StandardScaler()), ('classifier',model)])
elif SCALER == "maxmin":
clf = Pipeline([('scaler',MaxMinScaler()), ('classifier',model)])
elif SCALER == None:
clf = model
else:
raise NotImplementedErrorA complete example is as follows:
from pandas import read_csv
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, MinMaxScaler
# toggle between options
SCALER = "maxmin" # "standard", "maxmin", or None
CLASSIFIER = "cart" # "svc" or "cart"
# Load dataset
url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/iris.csv"
names = ['sepal-length', 'sepal-width', 'petal-length', 'petal-width', 'class']
dataset = read_csv(url, names=names)
# Split-out validation dataset
array = dataset.values
X = array[:,0:4]
y = array[:,4]
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.20, random_state=1, shuffle=True)
# Create model
if CLASSIFIER == "svc":
model = SVC()
elif CLASSIFIER == "cart":
model = DecisionTreeClassifier()
else:
raise NotImplementedError
if SCALER == "standard":
clf = Pipeline([('scaler',StandardScaler()), ('classifier',model)])
elif SCALER == "maxmin":
clf = Pipeline([('scaler',MinMaxScaler()), ('classifier',model)])
elif SCALER == None:
clf = model
else:
raise NotImplementedError
# Train
clf.fit(X_train, y_train)
# Test
score = clf.score(X_val, y_val)
print("Validation accuracy", score)If you go one step further, you may even skip the toggle variable and use a string directly for a quick experiment:
import numpy as np
import scipy.stats as stats
# Covariance matrix and Cholesky decomposition
cov = np.array([[1, 0.8], [0.8, 1]])
L = np.linalg.cholesky(cov)
# Generate 100 pairs of bi-variate Gaussian random numbers
if not "USE SCIPY":
z = np.random.randn(100,2)
x = z @ L.T
else:
x = stats.multivariate_normal(mean=[0, 0], cov=cov).rvs(100)
...Functions as Objects
In Python, functions are first-class citizens. You can assign functions to a variable. Indeed, functions are objects in Python, as are classes (the classes themselves, not only incarnations of classes). Therefore, we can use the same technique as above to experiment with similar functions.
import numpy as np
DIST = "normal"
if DIST == "normal":
rangen = np.random.normal
elif DIST == "uniform":
rangen = np.random.uniform
else:
raise NotImplementedError
random_data = rangen(size=(10,5))
print(random_data)The above is similar to calling np.random.normal(size=(10,5)), but we hold the function in a variable for the convenience of swapping one function with another. Note that since we call the functions with the same argument, we have to make sure all variations will accept it. In case it is not, we may need some additional lines of code to make a wrapper. For example, in the case of generating Student’s t distribution, we need an additional parameter for the degree of freedom:
import numpy as np
DIST = "t"
if DIST == "normal":
rangen = np.random.normal
elif DIST == "uniform":
rangen = np.random.uniform
elif DIST == "t":
def t_wrapper(size):
# Student's t distribution with 3 degree of freedom
return np.random.standard_t(df=3, size=size)
rangen = t_wrapper
else:
raise NotImplementedError
random_data = rangen(size=(10,5))
print(random_data)This works because in the above, np.random.normal, np.random.uniform, and t_wrapper as we defined, are all drop-in replacements of each other.
Caveats
Machine learning differs from other programming projects because there are more uncertainties in the workflow. When you build a web page or build a game, you have a picture in your mind of what to achieve. But there is some exploratory work in machine learning projects.
You will probably use some source code control system like git or Mercurial to manage your source code development history in other projects. In machine learning projects, however, we are trying out different combinations of many steps. Using git to manage the different variations may not fit, not to say sometimes may be overkill. Therefore, using a toggle variable to control the flow should allow us to try out different things faster. This is especially handy when we are working on our projects in Jupyter notebooks.
However, as we put multiple versions of code together, we made the program clumsy and less readable. It is better to do some clean-up after we confirm what to do. This will help us with maintenance in the future.
Original article sourced at: https://machinelearningmastery.com
What is GEEK
Buddha Community
1626775355
Why use Python for Software Development
No programming language is pretty much as diverse as Python. It enables building cutting edge applications effortlessly. Developers are as yet investigating the full capability of end-to-end Python development services in various areas.
By areas, we mean FinTech, HealthTech, InsureTech, Cybersecurity, and that's just the beginning. These are New Economy areas, and Python has the ability to serve every one of them. The vast majority of them require massive computational abilities. Python's code is dynamic and powerful - equipped for taking care of the heavy traffic and substantial algorithmic capacities.
Programming advancement is multidimensional today. Endeavor programming requires an intelligent application with AI and ML capacities. Shopper based applications require information examination to convey a superior client experience. Netflix, Trello, and Amazon are genuine instances of such applications. Python assists with building them effortlessly.
5 Reasons to Utilize Python for Programming Web Apps
Python can do such numerous things that developers can't discover enough reasons to admire it. Python application development isn't restricted to web and enterprise applications. It is exceptionally adaptable and superb for a wide range of uses.
Robust frameworks
Python is known for its tools and frameworks. There's a structure for everything. Django is helpful for building web applications, venture applications, logical applications, and mathematical processing. Flask is another web improvement framework with no conditions.
Web2Py, CherryPy, and Falcon offer incredible capabilities to customize Python development services. A large portion of them are open-source frameworks that allow quick turn of events.
Simple to read and compose
Python has an improved sentence structure - one that is like the English language. New engineers for Python can undoubtedly understand where they stand in the development process. The simplicity of composing allows quick application building.
The motivation behind building Python, as said by its maker Guido Van Rossum, was to empower even beginner engineers to comprehend the programming language. The simple coding likewise permits developers to roll out speedy improvements without getting confused by pointless subtleties.
Utilized by the best
Alright - Python isn't simply one more programming language. It should have something, which is the reason the business giants use it. Furthermore, that too for different purposes. Developers at Google use Python to assemble framework organization systems, parallel information pusher, code audit, testing and QA, and substantially more. Netflix utilizes Python web development services for its recommendation algorithm and media player.
Massive community support
Python has a steadily developing community that offers enormous help. From amateurs to specialists, there's everybody. There are a lot of instructional exercises, documentation, and guides accessible for Python web development solutions.
Today, numerous universities start with Python, adding to the quantity of individuals in the community. Frequently, Python designers team up on various tasks and help each other with algorithmic, utilitarian, and application critical thinking.
Progressive applications
Python is the greatest supporter of data science, Machine Learning, and Artificial Intelligence at any enterprise software development company. Its utilization cases in cutting edge applications are the most compelling motivation for its prosperity. Python is the second most well known tool after R for data analytics.
The simplicity of getting sorted out, overseeing, and visualizing information through unique libraries makes it ideal for data based applications. TensorFlow for neural networks and OpenCV for computer vision are two of Python's most well known use cases for Machine learning applications.
Summary
Thinking about the advances in programming and innovation, Python is a YES for an assorted scope of utilizations. Game development, web application development services, GUI advancement, ML and AI improvement, Enterprise and customer applications - every one of them uses Python to its full potential.
The disadvantages of Python web improvement arrangements are regularly disregarded by developers and organizations because of the advantages it gives. They focus on quality over speed and performance over blunders. That is the reason it's a good idea to utilize Python for building the applications of the future.
#python development services #python development company #python app development #python development #python in web development #python software development
1602968400
Python Tricks Every Developer Should Know
Python is awesome, it’s one of the easiest languages with simple and intuitive syntax but wait, have you ever thought that there might ways to write your python code simpler?
In this tutorial, you’re going to learn a variety of Python tricks that you can use to write your Python code in a more readable and efficient way like a pro.
Let’s get started
Swapping value in Python
Instead of creating a temporary variable to hold the value of the one while swapping, you can do this instead
>>> FirstName = "kalebu"
>>> LastName = "Jordan"
>>> FirstName, LastName = LastName, FirstName
>>> print(FirstName, LastName)
('Jordan', 'kalebu')
#python #python-programming #python3 #python-tutorials #learn-python #python-tips #python-skills #python-development
1602666000
How to Remove all Duplicate Files on your Drive via Python
Today you’re going to learn how to use Python programming in a way that can ultimately save a lot of space on your drive by removing all the duplicates.
Intro
In many situations you may find yourself having duplicates files on your disk and but when it comes to tracking and checking them manually it can tedious.
Heres a solution
Instead of tracking throughout your disk to see if there is a duplicate, you can automate the process using coding, by writing a program to recursively track through the disk and remove all the found duplicates and that’s what this article is about.
But How do we do it?
If we were to read the whole file and then compare it to the rest of the files recursively through the given directory it will take a very long time, then how do we do it?
The answer is hashing, with hashing can generate a given string of letters and numbers which act as the identity of a given file and if we find any other file with the same identity we gonna delete it.
There’s a variety of hashing algorithms out there such as
- md5
- sha1
- sha224, sha256, sha384 and sha512
#python-programming #python-tutorials #learn-python #python-project #python3 #python #python-skills #python-tips
1597751700
How To Compare Tesla and Ford Company By Using Magic Methods in Python
Magic Methods are the special methods which gives us the ability to access built in syntactical features such as ‘<’, ‘>’, ‘==’, ‘+’ etc…
You must have worked with such methods without knowing them to be as magic methods. Magic methods can be identified with their names which start with __ and ends with __ like init, call, str etc. These methods are also called Dunder Methods, because of their name starting and ending with Double Underscore (Dunder).
Now there are a number of such special methods, which you might have come across too, in Python. We will just be taking an example of a few of them to understand how they work and how we can use them.
1. init
class AnyClass:
def __init__():
print("Init called on its own")
obj = AnyClass()
The first example is _init, _and as the name suggests, it is used for initializing objects. Init method is called on its own, ie. whenever an object is created for the class, the init method is called on its own.
The output of the above code will be given below. Note how we did not call the init method and it got invoked as we created an object for class AnyClass.
Init called on its own
2. add
Let’s move to some other example, add gives us the ability to access the built in syntax feature of the character +. Let’s see how,
class AnyClass:
def __init__(self, var):
self.some_var = var
def __add__(self, other_obj):
print("Calling the add method")
return self.some_var + other_obj.some_var
obj1 = AnyClass(5)
obj2 = AnyClass(6)
obj1 + obj2
#python3 #python #python-programming #python-web-development #python-tutorials #python-top-story #python-tips #learn-python
1624291780
Learn Python - Full Course for Beginners [Tutorial]
This course will give you a full introduction into all of the core concepts in python. Follow along with the videos and you’ll be a python programmer in no time!
⭐️ Contents ⭐
⌨️ (0:00) Introduction
⌨️ (1:45) Installing Python & PyCharm
⌨️ (6:40) Setup & Hello World
⌨️ (10:23) Drawing a Shape
⌨️ (15:06) Variables & Data Types
⌨️ (27:03) Working With Strings
⌨️ (38:18) Working With Numbers
⌨️ (48:26) Getting Input From Users
⌨️ (52:37) Building a Basic Calculator
⌨️ (58:27) Mad Libs Game
⌨️ (1:03:10) Lists
⌨️ (1:10:44) List Functions
⌨️ (1:18:57) Tuples
⌨️ (1:24:15) Functions
⌨️ (1:34:11) Return Statement
⌨️ (1:40:06) If Statements
⌨️ (1:54:07) If Statements & Comparisons
⌨️ (2:00:37) Building a better Calculator
⌨️ (2:07:17) Dictionaries
⌨️ (2:14:13) While Loop
⌨️ (2:20:21) Building a Guessing Game
⌨️ (2:32:44) For Loops
⌨️ (2:41:20) Exponent Function
⌨️ (2:47:13) 2D Lists & Nested Loops
⌨️ (2:52:41) Building a Translator
⌨️ (3:00:18) Comments
⌨️ (3:04:17) Try / Except
⌨️ (3:12:41) Reading Files
⌨️ (3:21:26) Writing to Files
⌨️ (3:28:13) Modules & Pip
⌨️ (3:43:56) Classes & Objects
⌨️ (3:57:37) Building a Multiple Choice Quiz
⌨️ (4:08:28) Object Functions
⌨️ (4:12:37) Inheritance
⌨️ (4:20:43) Python Interpreter
📺 The video in this post was made by freeCodeCamp.org
The origin of the article: https://www.youtube.com/watch?v=rfscVS0vtbw&list=PLWKjhJtqVAblfum5WiQblKPwIbqYXkDoC&index=3
🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
⭐ ⭐ ⭐The project is of interest to the community. Join to Get free ‘GEEK coin’ (GEEKCASH coin)!
☞ **-----CLICK HERE-----**⭐ ⭐ ⭐
Thanks for visiting and watching! Please don’t forget to leave a like, comment and share!
#python #learn python #learn python for beginners #learn python - full course for beginners [tutorial] #python programmer #concepts in python