1592543549
In this PyQt5 article we are going to learn How to Make PyQt5 GUI Calculator, OK before coding let’s just have a simple introduction to PyQt5.
#pyqt5 #gui #programming
1675304280
We are back with another exciting and much-talked-about Rails tutorial on how to use Hotwire with the Rails application. This Hotwire Rails tutorial is an alternate method for building modern web applications that consume a pinch of JavaScript.
Rails 7 Hotwire is the default front-end framework shipped with Rails 7 after it was launched. It is used to represent HTML over the wire in the Rails application. Previously, we used to add a hotwire-rails gem in our gem file and then run rails hotwire: install. However, with the introduction of Rails 7, the gem got deprecated. Now, we use turbo-rails and stimulus rails directly, which work as Hotwire’s SPA-like page accelerator and Hotwire’s modest JavaScript framework.
Hotwire is a package of different frameworks that help to build applications. It simplifies the developer’s work for writing web pages without the need to write JavaScript, and instead sending HTML code over the wire.
Introduction to The Hotwire Framework:
It uses simplified techniques to build web applications while decreasing the usage of JavaScript in the application. Turbo offers numerous handling methods for the HTML data sent over the wire and displaying the application’s data without actually loading the entire page. It helps to maintain the simplicity of web applications without destroying the single-page application experience by using the below techniques:
Turbo Frames: Turbo Frames help to load the different sections of our markup without any dependency as it divides the page into different contexts separately called frames and updates these frames individually.
Turbo Drive: Every link doesn’t have to make the entire page reload when clicked. Only the HTML contained within the tag will be displayed.
Turbo Streams: To add real-time features to the application, this technique is used. It helps to bring real-time data to the application using CRUD actions.
It represents the JavaScript framework, which is required when JS is a requirement in the application. The interaction with the HTML is possible with the help of a stimulus, as the controllers that help those interactions are written by a stimulus.
Not much information is available about Strada as it has not been officially released yet. However, it works with native applications, and by using HTML bridge attributes, interaction is made possible between web applications and native apps.
Simple diagrammatic representation of Hotwire Stack:
As we are implementing the Ruby on Rails Hotwire tutorial, make sure about the following installations before you can get started.
Looking for an enthusiastic team of ROR developers to shape the vision of your web project?
Contact Bacancy today and hire Ruby developers to start building your dream project!
Find the following commands to create a rails application.
mkdir ~/projects/railshotwire
cd ~/projects/railshotwire
echo "source 'https://rubygems.org'" > Gemfile
echo "gem 'rails', '~> 7.0.0'" >> Gemfile
bundle install
bundle exec rails new . --force -d=postgresql
Now create some files for the project, up till now no usage of Rails Hotwire can be seen.
Fire the following command in your terminal.
echo "class HomeController < ApplicationController" > app/controllers/home_controller.rb
echo "end" >> app/controllers/home_controller.rb
echo "class OtherController < ApplicationController" > app/controllers/other_controller.rb
echo "end" >> app/controllers/home_controller.rb
echo "Rails.application.routes.draw do" > config/routes.rb
echo ' get "home/index"' >> config/routes.rb
echo ' get "other/index"' >> config/routes.rb
echo ' root to: "home#index"' >> config/routes.rb
echo 'end' >> config/routes.rb
mkdir app/views/home
echo '<h1>This is Rails Hotwire homepage</h1>' > app/views/home/index.html.erb
echo '<div><%= link_to "Enter to other page", other_index_path %></div>' >> app/views/home/index.html.erb
mkdir app/views/other
echo '<h1>This is Another page</h1>' > app/views/other/index.html.erb
echo '<div><%= link_to "Enter to home page", root_path %></div>' >> app/views/other/index.html.erb
bin/rails db:create
bin/rails db:migrate
Additionally, you can clone the code and browse through the project. Here’s the source code of the repository: Rails 7 Hotwire application
Now, let’s see how Hotwire Rails can work its magic with various Turbo techniques.
Go to your localhost:3000 on your web browser and right-click on the Inspect and open a Network tab of the DevTools of the browser.
Now click on go to another page link that appears on the home page to redirect from the home page to another page. In our Network tab, we can see that this action of navigation is achieved via XHR. It appears only the part inside HTML is reloaded, here neither the CSS is reloaded nor the JS is reloaded when the navigation action is performed.
By performing this action we can see that Turbo Drive helps to represent the HTML response without loading the full page and only follows redirect and reindeer HTML responses which helps to make the application faster to access.
This technique helps to divide the current page into different sections called frames that can be updated separately independently when new data is added from the server.
Below we discuss the different use cases of Turbo frame like inline edition, sorting, searching, and filtering of data.
Let’s perform some practical actions to see the example of these use cases.
Make changes in the app/controllers/home_controller.rb file
#CODE
class HomeController < ApplicationController
def turbo_frame_form
end
def turbo_frame submit
extracted_anynumber = params[:any][:anynumber]
render :turbo_frame_form, status: :ok, locals: {anynumber: extracted_anynumber, comment: 'turbo_frame_submit ok' }
end
end
Add app/views/home/turbo_frame_form.html.erb file to the application and add this content inside the file.
#CODE
<section>
<%= turbo_frame_tag 'anyframe' do %>
<div>
<h2>Frame view</h2>
<%= form_with scope: :any, url: turbo_frame_submit_path, local: true do |form| %>
<%= form.label :anynumber, 'Type an integer (odd or even)', 'class' => 'my-0 d-inline' %>
<%= form.text_field :anynumber, type: 'number', 'required' => 'true', 'value' => "#{local_assigns[:anynumber] || 0}", 'aria-describedby' => 'anynumber' %>
<%= form.submit 'Submit this number', 'id' => 'submit-number' %>
<% end %>
</div>
<div>
<h2>Data of the view</h2>
<pre style="font-size: .7rem;"><%= JSON.pretty_generate(local_assigns) %></pre>
</div>
<% end %>
</section>
Make some adjustments in routes.rb
#CODE
Rails.application.routes.draw do
get 'home/index'
get 'other/index'
get '/home/turbo_frame_form' => 'home#turbo_frame_form', as: 'turbo_frame_form'
post '/home/turbo_frame_submit' => 'home#turbo_frame_submit', as: 'turbo_frame_submit'
root to: "home#index"
end
#CODE
<h1>This is Rails Hotwire home page</h1>
<div><%= link_to "Enter to other page", other_index_path %></div>
<%= turbo_frame_tag 'anyframe' do %>
<div>
<h2>Home view</h2>
<%= form_with scope: :any, url: turbo_frame_submit_path, local: true do |form| %>
<%= form.label :anynumber, 'Type an integer (odd or even)', 'class' => 'my-0 d-inline' %>
<%= form.text_field :anynumber, type: 'number', 'required' => 'true', 'value' => "#{local_assigns[:anynumber] || 0}", 'aria-describedby' => 'anynumber' %>
<%= form.submit 'Submit this number', 'id' => 'submit-number' %>
<% end %>
<div>
<% end %>
After making all the changes, restart the rails server and refresh the browser, the default view will appear on the browser.
Now in the field enter any digit, after entering the digit click on submit button, and as the submit button is clicked we can see the Turbo Frame in action in the below screen, we can observe that the frame part changed, the first title and first link didn’t move.
Turbo Streams deliver page updates over WebSocket, SSE or in response to form submissions by only using HTML and a series of CRUD-like operations, you are free to say that either
This transmit can be represented by a simple example.
#CODE
class OtherController < ApplicationController
def post_something
respond_to do |format|
format.turbo_stream { }
end
end
end
Add the below line in routes.rb file of the application
#CODE
post '/other/post_something' => 'other#post_something', as: 'post_something'
Superb! Rails will now attempt to locate the app/views/other/post_something.turbo_stream.erb template at any moment the ‘/other/post_something’ endpoint is reached.
For this, we need to add app/views/other/post_something.turbo_stream.erb template in the rails application.
#CODE
<turbo-stream action="append" target="messages">
<template>
<div id="message_1">This changes the existing message!</div>
</template>
</turbo-stream>
This states that the response will try to append the template of the turbo frame with ID “messages”.
Now change the index.html.erb file in app/views/other paths with the below content.
#CODE
<h1>This is Another page</h1>
<div><%= link_to "Enter to home page", root_path %></div>
<div style="margin-top: 3rem;">
<%= form_with scope: :any, url: post_something_path do |form| %>
<%= form.submit 'Post any message %>
<% end %>
<turbo-frame id="messages">
<div>An empty message</div>
</turbo-frame>
</div>
This action shows that after submitting the response, the Turbo Streams help the developer to append the message, without reloading the page.
Another use case we can test is that rather than appending the message, the developer replaces the message. For that, we need to change the content of app/views/other/post_something.turbo_stream.erb template file and change the value of the action attribute from append to replace and check the changes in the browser.
#CODE
<turbo-stream action="replace" target="messages">
<template>
<div id="message_1">This changes the existing message!</div>
</template>
</turbo-stream>
When we click on Post any message button, the message that appear below that button will get replaced with the message that is mentioned in the app/views/other/post_something.turbo_stream.erb template
There are some cases in an application where JS is needed, therefore to cover those scenarios we require Hotwire JS tool. Hotwire has a JS tool because in some scenarios Turbo-* tools are not sufficient. But as we know that Hotwire is used to reduce the usage of JS in an application, Stimulus considers HTML as the single source of truth. Consider the case where we have to give elements on a page some JavaScript attributes, such as data controller, data-action, and data target. For that, a stimulus controller that can access elements and receive events based on those characteristics will be created.
Make a change in app/views/other/index.html.erb template file in rails application
#CODE
<h1>This is Another page</h1>
<div><%= link_to "Enter to home page", root_path %></div>
<div style="margin-top: 2rem;">
<%= form_with scope: :any, url: post_something_path do |form| %>
<%= form.submit 'Post something' %>
<% end %>
<turbo-frame id="messages">
<div>An empty message</div>
</turbo-frame>
</div>
<div style="margin-top: 2rem;">
<h2>Stimulus</h2>
<div data-controller="hello">
<input data-hello-target="name" type="text">
<button data-action="click->hello#greet">
Greet
</button>
<span data-hello-target="output">
</span>
</div>
</div>
Make changes in the hello_controller.js in path app/JavaScript/controllers and add a stimulus controller in the file, which helps to bring the HTML into life.
#CODE
import { Controller } from "@hotwired/stimulus"
export default class extends Controller {
static targets = [ "name", "output" ]
greet() {
this.outputTarget.textContent =
`Hello, ${this.nameTarget.value}!`
}
}
Go to your browser after making the changes in the code and click on Enter to other page link which will navigate to the localhost:3000/other/index page there you can see the changes implemented by the stimulus controller that is designed to augment your HTML with just enough behavior to make it more responsive.
With just a little bit of work, Turbo and Stimulus together offer a complete answer for applications that are quick and compelling.
Using Rails 7 Hotwire helps to load the pages at a faster speed and allows you to render templates on the server, where you have access to your whole domain model. It is a productive development experience in ROR, without compromising any of the speed or responsiveness associated with SPA.
We hope you were satisfied with our Rails Hotwire tutorial. Write to us at service@bacancy.com for any query that you want to resolve, or if you want us to share a tutorial on your query.
For more such solutions on RoR, check out our Ruby on Rails Tutorials. We will always strive to amaze you and cater to your needs.
Original article source at: https://www.bacancytechnology.com/
1592543549
In this PyQt5 article we are going to learn How to Make PyQt5 GUI Calculator, OK before coding let’s just have a simple introduction to PyQt5.
#pyqt5 #gui #programming
1604008800
Static code analysis refers to the technique of approximating the runtime behavior of a program. In other words, it is the process of predicting the output of a program without actually executing it.
Lately, however, the term “Static Code Analysis” is more commonly used to refer to one of the applications of this technique rather than the technique itself — program comprehension — understanding the program and detecting issues in it (anything from syntax errors to type mismatches, performance hogs likely bugs, security loopholes, etc.). This is the usage we’d be referring to throughout this post.
“The refinement of techniques for the prompt discovery of error serves as well as any other as a hallmark of what we mean by science.”
We cover a lot of ground in this post. The aim is to build an understanding of static code analysis and to equip you with the basic theory, and the right tools so that you can write analyzers on your own.
We start our journey with laying down the essential parts of the pipeline which a compiler follows to understand what a piece of code does. We learn where to tap points in this pipeline to plug in our analyzers and extract meaningful information. In the latter half, we get our feet wet, and write four such static analyzers, completely from scratch, in Python.
Note that although the ideas here are discussed in light of Python, static code analyzers across all programming languages are carved out along similar lines. We chose Python because of the availability of an easy to use ast
module, and wide adoption of the language itself.
Before a computer can finally “understand” and execute a piece of code, it goes through a series of complicated transformations:
As you can see in the diagram (go ahead, zoom it!), the static analyzers feed on the output of these stages. To be able to better understand the static analysis techniques, let’s look at each of these steps in some more detail:
The first thing that a compiler does when trying to understand a piece of code is to break it down into smaller chunks, also known as tokens. Tokens are akin to what words are in a language.
A token might consist of either a single character, like (
, or literals (like integers, strings, e.g., 7
, Bob
, etc.), or reserved keywords of that language (e.g, def
in Python). Characters which do not contribute towards the semantics of a program, like trailing whitespace, comments, etc. are often discarded by the scanner.
Python provides the tokenize
module in its standard library to let you play around with tokens:
Python
1
import io
2
import tokenize
3
4
code = b"color = input('Enter your favourite color: ')"
5
6
for token in tokenize.tokenize(io.BytesIO(code).readline):
7
print(token)
Python
1
TokenInfo(type=62 (ENCODING), string='utf-8')
2
TokenInfo(type=1 (NAME), string='color')
3
TokenInfo(type=54 (OP), string='=')
4
TokenInfo(type=1 (NAME), string='input')
5
TokenInfo(type=54 (OP), string='(')
6
TokenInfo(type=3 (STRING), string="'Enter your favourite color: '")
7
TokenInfo(type=54 (OP), string=')')
8
TokenInfo(type=4 (NEWLINE), string='')
9
TokenInfo(type=0 (ENDMARKER), string='')
(Note that for the sake of readability, I’ve omitted a few columns from the result above — metadata like starting index, ending index, a copy of the line on which a token occurs, etc.)
#code quality #code review #static analysis #static code analysis #code analysis #static analysis tools #code review tips #static code analyzer #static code analysis tool #static analyzer
1567822183
In this lab we are going to build demo TARS from Interstellar movie with Python. TARS can help you to automate your tasks such as search videos in YouTube and play them, send emails, open websites, search materials in Wikipedia and read them,inform weather forecast in your country, greetings and more. By building TARS you will increase your Python knowledge and learn many useful libraries/tools. I will push source code to my git repository so feel free to contribute and improve functionality of TARS
Let’s start by creating virtual environment and building the base audio system of TARS.
mkdir TARS
cd TARS
virtualenv venv
To activate the venv run command below
. venv/bin/activate
What is virtual environment?
Once you activated venv, we need to install main libraries by following commands:
pip3 install gTTS
pip3 install SpeechRecognition
pip3 install PyAudio
pip3 install pygame
gTTS (Google Text-to-Speech) is a Python library and CLI tool to interface with Google Translate’s text-to-speech API. This module helps to convert String text to Spoken text and can be saved as .mp3
Speech Recognition is an important feature in several applications used such as home automation, artificial intelligence, etc. Recognizing speech needs audio input, and SpeechRecognition makes it really simple to retrieve this input. Instead of building scripts from scratch to access microphones and process audio files, SpeechRecognition will have you up and running in just a few minutes.
To access your microphone with SpeechRecognizer, you’ll have to install the PyAudio package
Pygame is a cross-platform set of Python modules designed for writing video games. It includes computer graphics and sound libraries designed to be used with the Python programming language.
Now, let’s build voice system of TARS:
from gtts import gTTS
import speech_recognition as sr
from pygame import mixer
def talk(audio):
print(audio)
for line in audio.splitlines():
text_to_speech = gTTS(text=audio, lang='en-uk')
text_to_speech.save('audio.mp3')
mixer.init()
mixer.music.load("audio.mp3")
mixer.music.play()
As you see we are passing audio as an argument to let the TARS speak. For instance, talk(‘Hey I am TARS! How can I help you?’) program will loop these lines with the help of splitlines() method. This method is used to split the lines at line boundaries. Check splitlines() for more. Then, gTTS will handle to convert all these texts to speech. text parameter defines text to be read and lang defines the language (IETF language tag) to read the text in. Once loop finished, save() method writes result to file.
pygame.mixer is a module for loading and playing sounds and must be initialized before using it.
Alright! Now, let’s create a function that will listen for commands.
def myCommand():
#Initialize the recognizer
r = sr.Recognizer()
with sr.Microphone() as source:
print('TARS is Ready...')
r.pause_threshold = 1
#wait for a second to let the recognizer adjust the
#energy threshold based on the surrounding noise level
r.adjust_for_ambient_noise(source, duration=1)
#listens for the user's input
audio = r.listen(source)
try:
command = r.recognize_google(audio).lower()
print('You said: ' + command + '\n')
#loop back to continue to listen for commands if unrecognizable speech is received
except sr.UnknownValueError:
print('Your last command couldn\'t be heard')
command = myCommand();
return command
In this function we are using SpeechRecognition library. It acts as a wrapper for several popular speech APIs and is thus extremely flexible. One of these—the Google Web Speech API—supports a default API key that is hard-coded into the SpeechRecognition library. That means you can get off your feet without having to sign up for a service.
To be able to work with your own voice with speech recognition, you need the PyAudio package. Like Recognizer for audio files, we will need Microphone for real-time speech data.
You can capture input from the microphone using the listen() method of the Recognizer class inside of the with block. This method takes an audio source as its first argument and records input from the source until silence is detected.
Try to say your commands in silence place( with less background noise ) otherwise TARS can confuse.
Take a look The Ultimate Guide To Speech Recognition With Python
import random
def tars(command):
errors=[
"I don\'t know what you mean!",
"Excuse me?",
"Can you repeat it please?",
]
if 'Hello' in command:
talk('Hello! I am TARS. How can I help you?')
else:
error = random.choice(errors)
talk(error)
talk('TARS is ready!')
while True:
assistant(myCommand())
Once you run the program TARS will start talk with you by saying ‘TARS is ready!’ and continue to listen your commands until you stop the program. Start by saying ‘Hello’ :)
When TARS didn’t get the command we will handle the error by random sentences.
Here is the full code of main structure:
from gtts import gTTS
import speech_recognition as sr
from pygame import mixer
import random
def talk(audio):
print(audio)
for line in audio.splitlines():
text_to_speech = gTTS(text=audio, lang='en-uk')
text_to_speech.save('audio.mp3')
mixer.init()
mixer.music.load("audio.mp3")
mixer.music.play()
def myCommand():
#Initialize the recognizer
#The primary purpose of a Recognizer instance is, of course, to recognize speech.
r = sr.Recognizer()
with sr.Microphone() as source:
print('TARS is Ready...')
r.pause_threshold = 2
#wait for a second to let the recognizer adjust the
#energy threshold based on the surrounding noise level
r.adjust_for_ambient_noise(source, duration=1)
#listens for the user's input
audio = r.listen(source)
try:
command = r.recognize_google(audio).lower()
print('You said: ' + command + '\n')
#loop back to continue to listen for commands if unrecognizable speech is received
except sr.UnknownValueError:
print('Your last command couldn\'t be heard')
command = myCommand();
return command
def tars(command):
errors=[
"I don't know what you mean",
"Did you mean astronaut?",
"Can you repeat it please?",
]
if 'hello' in command:
talk('Hello! I am TARS. How can I help you?')
else:
error = random.choice(errors)
talk(error)
talk('TARS is ready!')
#loop to continue executing multiple commands
while True:
tars(myCommand())
Well… Is AI anything more than a bunch of IF statements?
If you are talking about “real” AI , then yes it’s a lot more than just If statements.The development of AI has historically been split into two fields; symbolic AI, and machine learning.
Symbolic AI is the field in which artificially intelligent systems were designed with if-else type logic. Programmers would attempt to define every possible scenario for the system to deal with. Until the late seventies this was the dominant form of AI system development. Experts in the field argued very strongly that machine-learning would never catch on and that AI could only be written in this way.
Now we know that accounting for every possible scenario in an intelligent system is enormously impractical and we use machine-learning instead. Machine learning uses statistics to look for and define patterns in data so that a machine can learn about and improve the tasks that it is designed to perform. This is significantly more flexible.
We are using just bunch of IF statements to understand basics of AI. But we will implement some cool ML algorithms later.
I hope you learned new things so far, now, it is time to teach TARS how to automate stuff.
Open Google and search for something
We are going to import webbrowser module in Python which provides an interface to display Web-based documents.
While we are saying commands, TARS have to detect availability of these commands by matching them. Python has a built-in package called re, which can be used to work with Regular Expressions.
import re
import webbrowser
if 'open google' in command:
#matching command to check it is available
reg_ex = re.search('open google (.*)', command)
url = 'https://www.google.com/'
if reg_ex:
subgoogle = reg_ex.group(1)
url = url + 'r/' + subreddit
webbrowser.open(url)
print('Done!')
The re.search() method takes a regular expression pattern and a string and searches for that pattern within the string. If the search is successful, search() returns a match object or None otherwise. Therefore, the search is usually immediately followed by an if-statement to test if the search succeeded
The code reg_ex = re.search('open google (.)', command)* stores the search result in a variable named “reg_ex”. Then the if-statement tests the match – if true the search succeeded and group() is the matching text. Otherwise if the match is false (None to be more specific), then the search did not succeed, and there is no matching text. The 1 in reg_ex.group(1) represents the first parenthesized subgroup.
Even you can install Selenium to make search in Google by TARS. To install Selenium run the following command:
pip3 install selenium
Selenium WebDriver is a collection of open source APIs which are used to automate the testing of a web application. This tool is used to automate web application testing to verify that it works as expected. It supports many browsers such as Safari, Firefox, IE, and Chrome.
You can search how to use Selenium with Python there is a lot of sources on internet and it is really easy to learn. Let’s add this feature to TARS
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
if 'open google and search' in command:
reg_ex = re.search('open google and search (.*)', command)
search_for = command.split("search",1)[1]
url = 'https://www.google.com/'
if reg_ex:
subgoogle = reg_ex.group(1)
url = url + 'r/' + subgoogle
talk('Okay!')
driver = webdriver.Firefox(executable_path='/path/to/geckodriver') #depends which web browser you are using
driver.get('http://www.google.com')
search = driver.find_element_by_name('q') # finds search
search.send_keys(str(search_for)) #sends search keys
search.send_keys(Keys.RETURN) #hits enter
TARS will consider strings after “open google and search” command and takes all words as a search keys. I am using Firefox so I installed geckodriver but if you are using Chrome check the following StackOverflow question.
We are going to import smtplib to send emails with Python. SMTP stands for Simple Mail Transfer Protocol and it is useful for communicating with mail servers to send mail.
import smtplib
elif 'email' or 'gmail' in command:
talk('What is the subject?')
time.sleep(3)
subject = myCommand()
talk('What should I say?')
time.sleep(3)
message = myCommand()
content = 'Subject: {}\n\n{}'.format(subject, message)
#init gmail SMTP
mail = smtplib.SMTP('smtp.gmail.com', 587)
#identify to server
mail.ehlo()
#encrypt session
mail.starttls()
#login
mail.login('your_gmail', 'your_gmail_password')
#send message
mail.sendmail('FROM', 'TO', content)
#end mail connection
mail.close()
talk('Email sent.')
Note that, in a nutshell, google is not allowing you to log in via smtplib because it has flagged this sort of login as “less secure”, so what you have to do is go to this link while you’re logged in to your google account, and allow the access.
Crawl Data
We are doing great so far! TARS can send mails and search whatever you want on google. Now, let’s implement more complex function to make TARS crawl some Wikipedia data and read it for us.
Beautiful Soup is a Python library for pulling data out of HTML and XML files. It works with your favorite parser to provide idiomatic ways of navigating, searching, and modifying the parse tree. It commonly saves programmers hours or days of work. Run the following command in your terminal to install beautifulsoup:
pip install beautifulsoup4
We also will need requests library for making HTTP requests in Python. It abstracts the complexities of making requests behind a beautiful, simple API so that you can focus on interacting with services and consuming data in your application. Alright! Let’s see the code:
import bs4
import requests
elif 'wikipedia' in command:
reg_ex = re.search('search in wikipedia (.+)', command)
if reg_ex:
query = command.split()
response = requests.get("https://en.wikipedia.org/wiki/" + query[3])
if response is not None:
html = bs4.BeautifulSoup(response.text, 'html.parser')
title = html.select("#firstHeading")[0].text
paragraphs = html.select("p")
for para in paragraphs:
print (para.text)
intro = '\n'.join([ para.text for para in paragraphs[0:5]])
print (intro)
mp3name = 'speech.mp3'
language = 'en'
myobj = gTTS(text=intro, lang=language, slow=False)
myobj.save(mp3name)
mixer.init()
mixer.music.load("speech.mp3")
mixer.music.play()
elif 'stop' in command:
mixer.music.stop()
“search in wikipedia Mars” and TARS will take “Mars” as a keyword to search in Wikipedia. If you search something on Wikipedia you will see URL will look like https://en.wikipedia.org/wiki/Keyword so we are sending get request with keyword(what to search) to access data. Once request succeed, beautifulsoup will parse content inside Wikipedia. The join() method is a string method and returns a string in which the elements of sequence have been joined by str separator and we are using it to separate paragraphs. You already familiar with gTTS and mixer so I am passing that part.
TARS will display the crawled data on console and start to reading it for you.
Search videos on YouTube and play
This function is similar to search with google but this time it is better to use urllib. The main objective is to learn new things with Python, so I don’t want include Selenium in this function. Here is the code:
import urllib.request #used to make requests
import urllib.parse #used to parse values into the url
elif 'youtube' in command:
talk('Ok!')
reg_ex = re.search('youtube (.+)', command)
if reg_ex:
domain = command.split("youtube",1)[1]
query_string = urllib.parse.urlencode({"search_query" : domain})
html_content = urllib.request.urlopen("http://www.youtube.com/results?" + query_string)
search_results = re.findall(r'href=\"\/watch\?v=(.{11})', html_content.read().decode()) # finds all links in search result
webbrowser.open("http://www.youtube.com/watch?v={}".format(search_results[0]))
pass
The urllib module in Python 3 allows you access websites via your program. This opens up as many doors for your programs as the internet opens up for you. urllib in Python 3 is slightly different than urllib2 in Python 2, but they are mostly the same. Through urllib, you can access websites, download data, parse data, modify your headers, and do any GET and POST requests you might need to do.
Check this tutorial for more about urllib
Search key must be encoded before parsing into url. If you search something on YouTube you can see after [http://www.youtube.com/results?"](http://www.youtube.com/results?" “http://www.youtube.com/results?"”) there is a encoded search keys. Once these search keys encoded program can successfully access search results. The expression re.findall() returns all the non-overlapping matches of patterns in a string as a list of strings. Each video on youtube has its own 11 characters ID (https://www.youtube.com/watch?v=gEPmA3USJdI)and re.findall() will find all matches in decoded html_content(in search results page). decode() is used to convert from one encoding scheme, in which argument string is encoded to the desired encoding scheme. This works opposite to the encode. It accepts the encoding of the encoding string to decode it and returns the original string. Finally, it plays first video in search results because usually the first video is nearest one for search keys.
Full Code:
from gtts import gTTS
import speech_recognition as sr
import re
import time
import webbrowser
import random
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import smtplib
import requests
from pygame import mixer
import urllib.request
import urllib.parse
import bs4
def talk(audio):
"speaks audio passed as argument"
print(audio)
for line in audio.splitlines():
text_to_speech = gTTS(text=audio, lang='en-uk')
text_to_speech.save('audio.mp3')
mixer.init()
mixer.music.load("audio.mp3")
mixer.music.play()
def myCommand():
"listens for commands"
#Initialize the recognizer
#The primary purpose of a Recognizer instance is, of course, to recognize speech.
r = sr.Recognizer()
with sr.Microphone() as source:
print('TARS is Ready...')
r.pause_threshold = 1
#wait for a second to let the recognizer adjust the
#energy threshold based on the surrounding noise level
r.adjust_for_ambient_noise(source, duration=1)
#listens for the user's input
audio = r.listen(source)
print('analyzing...')
try:
command = r.recognize_google(audio).lower()
print('You said: ' + command + '\n')
time.sleep(2)
#loop back to continue to listen for commands if unrecognizable speech is received
except sr.UnknownValueError:
print('Your last command couldn\'t be heard')
command = myCommand();
return command
def tars(command):
errors=[
"I don't know what you mean",
"Excuse me?",
"Can you repeat it please?",
]
"if statements for executing commands"
# Search on Google
if 'open google and search' in command:
reg_ex = re.search('open google and search (.*)', command)
search_for = command.split("search",1)[1]
print(search_for)
url = 'https://www.google.com/'
if reg_ex:
subgoogle = reg_ex.group(1)
url = url + 'r/' + subgoogle
talk('Okay!')
driver = webdriver.Firefox(executable_path='/home/coderasha/Desktop/geckodriver')
driver.get('http://www.google.com')
search = driver.find_element_by_name('q')
search.send_keys(str(search_for))
search.send_keys(Keys.RETURN) # hit return after you enter search text
#Send Email
elif 'email' in command:
talk('What is the subject?')
time.sleep(3)
subject = myCommand()
talk('What should I say?')
message = myCommand()
content = 'Subject: {}\n\n{}'.format(subject, message)
#init gmail SMTP
mail = smtplib.SMTP('smtp.gmail.com', 587)
#identify to server
mail.ehlo()
#encrypt session
mail.starttls()
#login
mail.login('your_mail', 'your_mail_password')
#send message
mail.sendmail('FROM', 'TO', content)
#end mail connection
mail.close()
talk('Email sent.')
# search in wikipedia (e.g. Can you search in wikipedia apples)
elif 'wikipedia' in command:
reg_ex = re.search('wikipedia (.+)', command)
if reg_ex:
query = command.split("wikipedia",1)[1]
response = requests.get("https://en.wikipedia.org/wiki/" + query)
if response is not None:
html = bs4.BeautifulSoup(response.text, 'html.parser')
title = html.select("#firstHeading")[0].text
paragraphs = html.select("p")
for para in paragraphs:
print (para.text)
intro = '\n'.join([ para.text for para in paragraphs[0:3]])
print (intro)
mp3name = 'speech.mp3'
language = 'en'
myobj = gTTS(text=intro, lang=language, slow=False)
myobj.save(mp3name)
mixer.init()
mixer.music.load("speech.mp3")
while mixer.music.play()
elif 'stop' in command:
mixer.music.stop()
# Search videos on Youtube and play (e.g. Search in youtube believer)
elif 'youtube' in command:
talk('Ok!')
reg_ex = re.search('youtube (.+)', command)
if reg_ex:
domain = command.split("youtube",1)[1]
query_string = urllib.parse.urlencode({"search_query" : domain})
html_content = urllib.request.urlopen("http://www.youtube.com/results?" + query_string)
search_results = re.findall(r'href=\"\/watch\?v=(.{11})', html_content.read().decode())
#print("http://www.youtube.com/watch?v=" + search_results[0])
webbrowser.open("http://www.youtube.com/watch?v={}".format(search_results[0]))
pass
elif 'hello' in command:
talk('Hello! I am TARS. How can I help you?')
time.sleep(3)
elif 'who are you' in command:
talk('I am one of four former U.S. Marine Corps tactical robots')
time.sleep(3)
else:
error = random.choice(errors)
talk(error)
time.sleep(3)
talk('TARS activated!')
#loop to continue executing multiple commands
while True:
time.sleep(4)
tars(myCommand())
Cool! We just created demo TARS and I hope you learned many things from this lab. Please feel free to contribute this project on GitHub, TARS will wait for improvements.
I hope this tutorial will surely help and you if you liked this tutorial, please consider sharing it with others.
#python #web-development
1641430440
Pandas-Bokeh provides a Bokeh plotting backend for Pandas, GeoPandas and Pyspark DataFrames, similar to the already existing Visualization feature of Pandas. Importing the library adds a complementary plotting method plot_bokeh() on DataFrames and Series.
With Pandas-Bokeh, creating stunning, interactive, HTML-based visualization is as easy as calling:
df.plot_bokeh()
Pandas-Bokeh also provides native support as a Pandas Plotting backend for Pandas >= 0.25. When Pandas-Bokeh is installed, switchting the default Pandas plotting backend to Bokeh can be done via:
pd.set_option('plotting.backend', 'pandas_bokeh')
More details about the new Pandas backend can be found below.
Please visit:
https://patrikhlobil.github.io/Pandas-Bokeh/
for an interactive version of the documentation below, where you can play with the dynamic Bokeh plots.
For more information have a look at the Examples below or at notebooks on the Github Repository of this project.
You can install Pandas-Bokeh from PyPI via pip
pip install pandas-bokeh
or conda:
conda install -c patrikhlobil pandas-bokeh
With the current release 0.5.5, Pandas-Bokeh officially supports Python 3.6 and newer. For more details, see Release Notes.
The Pandas-Bokeh library should be imported after Pandas, GeoPandas and/or Pyspark. After the import, one should define the plotting output, which can be:
pandas_bokeh.output_notebook(): Embeds the Plots in the cell outputs of the notebook. Ideal when working in Jupyter Notebooks.
pandas_bokeh.output_file(filename): Exports the plot to the provided filename as an HTML.
For more details about the plotting outputs, see the reference here or the Bokeh documentation.
import pandas as pd import pandas_bokeh pandas_bokeh.output_notebook()
import pandas as pd import pandas_bokeh pandas_bokeh.output_file("Interactive Plot.html")
For pandas >= 0.25, a plotting backend switch is natively supported. It can be achievied by calling:
import pandas as pd
pd.set_option('plotting.backend', 'pandas_bokeh')
Now, the plotting API is accessible for a Pandas DataFrame via:
df.plot(...)
All additional functionalities of Pandas-Bokeh are then accessible at pd.plotting. So, setting the output to notebook is:
pd.plotting.output_notebook()
or calling the grid layout functionality:
pd.plotting.plot_grid(...)
Note: Backwards compatibility is kept since there will still be the df.plot_bokeh(...) methods for a DataFrame.
Supported plottypes are at the moment:
Also, check out the complementary chapter Outputs, Formatting & Layouts about:
This simple lineplot in Pandas-Bokeh already contains various interactive elements:
Consider the following simple example:
import numpy as np
np.random.seed(42)
df = pd.DataFrame({"Google": np.random.randn(1000)+0.2,
"Apple": np.random.randn(1000)+0.17},
index=pd.date_range('1/1/2000', periods=1000))
df = df.cumsum()
df = df + 50
df.plot_bokeh(kind="line") #equivalent to df.plot_bokeh.line()
Note, that similar to the regular pandas.DataFrame.plot method, there are also additional accessors to directly access the different plotting types like:
df.plot_bokeh(kind="line", ...)
→ df.plot_bokeh.line(...)
df.plot_bokeh(kind="bar", ...)
→ df.plot_bokeh.bar(...)
df.plot_bokeh(kind="hist", ...)
→ df.plot_bokeh.hist(...)
There are various optional parameters to tune the plots, for example:
kind: Which kind of plot should be produced. Currently supported are: "line", "point", "scatter", "bar" and "histogram". In the near future many more will be implemented as horizontal barplot, boxplots, pie-charts, etc.
x: Name of the column to use for the horizontal x-axis. If the x parameter is not specified, the index is used for the x-values of the plot. Alternative, also an array of values can be passed that has the same number of elements as the DataFrame.
y: Name of column or list of names of columns to use for the vertical y-axis.
figsize: Choose width & height of the plot
title: Sets title of the plot
xlim/ylim: Set visibler range of plot for x- and y-axis (also works for datetime x-axis)
xlabel/ylabel: Set x- and y-labels
logx/logy: Set log-scale on x-/y-axis
xticks/yticks: Explicitly set the ticks on the axes
color: Defines a single color for a plot.
colormap: Can be used to specify multiple colors to plot. Can be either a list of colors or the name of a Bokeh color palette
hovertool: If True a Hovertool is active, else if False no Hovertool is drawn.
hovertool_string: If specified, this string will be used for the hovertool (@{column} will be replaced by the value of the column for the element the mouse hovers over, see also Bokeh documentation and here)
toolbar_location: Specify the position of the toolbar location (None, "above", "below", "left" or "right"). Default: "right"
zooming: Enables/Disables zooming. Default: True
panning: Enables/Disables panning. Default: True
fontsize_label/fontsize_ticks/fontsize_title/fontsize_legend: Set fontsize of labels, ticks, title or legend (int or string of form "15pt")
rangetool Enables a range tool scroller. Default False
kwargs**: Optional keyword arguments of bokeh.plotting.figure.line
Try them out to get a feeling for the effects. Let us consider now:
df.plot_bokeh.line(
figsize=(800, 450),
y="Apple",
title="Apple vs Google",
xlabel="Date",
ylabel="Stock price [$]",
yticks=[0, 100, 200, 300, 400],
ylim=(0, 400),
toolbar_location=None,
colormap=["red", "blue"],
hovertool_string=r"""<img
src='https://upload.wikimedia.org/wikipedia/commons/thumb/f/fa/Apple_logo_black.svg/170px-Apple_logo_black.svg.png'
height="42" alt="@imgs" width="42"
style="float: left; margin: 0px 15px 15px 0px;"
border="2"></img> Apple
<h4> Stock Price: </h4> @{Apple}""",
panning=False,
zooming=False)
For lineplots, as for many other plot-kinds, there are some special keyword arguments that only work for this plotting type. For lineplots, these are:
plot_data_points: Plot also the data points on the lines
plot_data_points_size: Determines the size of the data points
marker: Defines the point type (Default: "circle"). Possible values are: 'circle', 'square', 'triangle', 'asterisk', 'circle_x', 'square_x', 'inverted_triangle', 'x', 'circle_cross', 'square_cross', 'diamond', 'cross'
kwargs**: Optional keyword arguments of bokeh.plotting.figure.line```
Let us use this information to have another version of the same plot:
df.plot_bokeh.line(
figsize=(800, 450),
title="Apple vs Google",
xlabel="Date",
ylabel="Stock price [$]",
yticks=[0, 100, 200, 300, 400],
ylim=(100, 200),
xlim=("2001-01-01", "2001-02-01"),
colormap=["red", "blue"],
plot_data_points=True,
plot_data_points_size=10,
marker="asterisk")
ts = pd.Series(np.random.randn(1000), index=pd.date_range('1/1/2000', periods=1000))
df = pd.DataFrame(np.random.randn(1000, 4), index=ts.index, columns=list('ABCD'))
df = df.cumsum()
df.plot_bokeh(rangetool=True)
Pointplot
If you just wish to draw the date points for curves, the pointplot option is the right choice. It also accepts the kwargs of bokeh.plotting.figure.scatter like marker or size:
import numpy as np
x = np.arange(-3, 3, 0.1)
y2 = x**2
y3 = x**3
df = pd.DataFrame({"x": x, "Parabula": y2, "Cube": y3})
df.plot_bokeh.point(
x="x",
xticks=range(-3, 4),
size=5,
colormap=["#009933", "#ff3399"],
title="Pointplot (Parabula vs. Cube)",
marker="x")
With a similar API as the line- & pointplots, one can generate a stepplot. Additional keyword arguments for this plot type are passes to bokeh.plotting.figure.step, e.g. mode (before, after, center), see the following example
import numpy as np
x = np.arange(-3, 3, 1)
y2 = x**2
y3 = x**3
df = pd.DataFrame({"x": x, "Parabula": y2, "Cube": y3})
df.plot_bokeh.step(
x="x",
xticks=range(-1, 1),
colormap=["#009933", "#ff3399"],
title="Pointplot (Parabula vs. Cube)",
figsize=(800,300),
fontsize_title=30,
fontsize_label=25,
fontsize_ticks=15,
fontsize_legend=5,
)
df.plot_bokeh.step(
x="x",
xticks=range(-1, 1),
colormap=["#009933", "#ff3399"],
title="Pointplot (Parabula vs. Cube)",
mode="after",
figsize=(800,300)
)
Note that the step-plot API of Bokeh does so far not support a hovertool functionality.
A basic scatterplot can be created using the kind="scatter" option. For scatterplots, the x and y parameters have to be specified and the following optional keyword argument is allowed:
category: Determines the category column to use for coloring the scatter points
kwargs**: Optional keyword arguments of bokeh.plotting.figure.scatter
Note, that the pandas.DataFrame.plot_bokeh() method return per default a Bokeh figure, which can be embedded in Dashboard layouts with other figures and Bokeh objects (for more details about (sub)plot layouts and embedding the resulting Bokeh plots as HTML click here).
In the example below, we use the building grid layout support of Pandas-Bokeh to display both the DataFrame (using a Bokeh DataTable) and the resulting scatterplot:
# Load Iris Dataset:
df = pd.read_csv(
r"https://raw.githubusercontent.com/PatrikHlobil/Pandas-Bokeh/master/docs/Testdata/iris/iris.csv"
)
df = df.sample(frac=1)
# Create Bokeh-Table with DataFrame:
from bokeh.models.widgets import DataTable, TableColumn
from bokeh.models import ColumnDataSource
data_table = DataTable(
columns=[TableColumn(field=Ci, title=Ci) for Ci in df.columns],
source=ColumnDataSource(df),
height=300,
)
# Create Scatterplot:
p_scatter = df.plot_bokeh.scatter(
x="petal length (cm)",
y="sepal width (cm)",
category="species",
title="Iris DataSet Visualization",
show_figure=False,
)
# Combine Table and Scatterplot via grid layout:
pandas_bokeh.plot_grid([[data_table, p_scatter]], plot_width=400, plot_height=350)
A possible optional keyword parameters that can be passed to bokeh.plotting.figure.scatter is size. Below, we use the sepal length of the Iris data as reference for the size:
#Change one value to clearly see the effect of the size keyword
df.loc[13, "sepal length (cm)"] = 15
#Make scatterplot:
p_scatter = df.plot_bokeh.scatter(
x="petal length (cm)",
y="sepal width (cm)",
category="species",
title="Iris DataSet Visualization with Size Keyword",
size="sepal length (cm)")
In this example you can see, that the additional dimension sepal length cannot be used to clearly differentiate between the virginica and versicolor species.
The barplot API has no special keyword arguments, but accepts optional kwargs of bokeh.plotting.figure.vbar like alpha. It uses per default the index for the bar categories (however, also columns can be used as x-axis category using the x argument).
data = {
'fruits':
['Apples', 'Pears', 'Nectarines', 'Plums', 'Grapes', 'Strawberries'],
'2015': [2, 1, 4, 3, 2, 4],
'2016': [5, 3, 3, 2, 4, 6],
'2017': [3, 2, 4, 4, 5, 3]
}
df = pd.DataFrame(data).set_index("fruits")
p_bar = df.plot_bokeh.bar(
ylabel="Price per Unit [€]",
title="Fruit prices per Year",
alpha=0.6)
Using the stacked keyword argument you also maked stacked barplots:
p_stacked_bar = df.plot_bokeh.bar(
ylabel="Price per Unit [€]",
title="Fruit prices per Year",
stacked=True,
alpha=0.6)
Also horizontal versions of the above barplot are supported with the keyword kind="barh" or the accessor plot_bokeh.barh. You can still specify a column of the DataFrame as the bar category via the x argument if you do not wish to use the index.
#Reset index, such that "fruits" is now a column of the DataFrame:
df.reset_index(inplace=True)
#Create horizontal bar (via kind keyword):
p_hbar = df.plot_bokeh(
kind="barh",
x="fruits",
xlabel="Price per Unit [€]",
title="Fruit prices per Year",
alpha=0.6,
legend = "bottom_right",
show_figure=False)
#Create stacked horizontal bar (via barh accessor):
p_stacked_hbar = df.plot_bokeh.barh(
x="fruits",
stacked=True,
xlabel="Price per Unit [€]",
title="Fruit prices per Year",
alpha=0.6,
legend = "bottom_right",
show_figure=False)
#Plot all barplot examples in a grid:
pandas_bokeh.plot_grid([[p_bar, p_stacked_bar],
[p_hbar, p_stacked_hbar]],
plot_width=450)
For drawing histograms (kind="hist"), Pandas-Bokeh has a lot of customization features. Optional keyword arguments for histogram plots are:
bins: Determines bins to use for the histogram. If bins is an int, it defines the number of equal-width bins in the given range (10, by default). If bins is a sequence, it defines the bin edges, including the rightmost edge, allowing for non-uniform bin widths. If bins is a string, it defines the method used to calculate the optimal bin width, as defined by histogram_bin_edges.
histogram_type: Either "sidebyside", "topontop" or "stacked". Default: "topontop"
stacked: Boolean that overrides the histogram_type as "stacked" if given. Default: False
kwargs**: Optional keyword arguments of bokeh.plotting.figure.quad
Below examples of the different histogram types:
import numpy as np
df_hist = pd.DataFrame({
'a': np.random.randn(1000) + 1,
'b': np.random.randn(1000),
'c': np.random.randn(1000) - 1
},
columns=['a', 'b', 'c'])
#Top-on-Top Histogram (Default):
df_hist.plot_bokeh.hist(
bins=np.linspace(-5, 5, 41),
vertical_xlabel=True,
hovertool=False,
title="Normal distributions (Top-on-Top)",
line_color="black")
#Side-by-Side Histogram (multiple bars share bin side-by-side) also accessible via
#kind="hist":
df_hist.plot_bokeh(
kind="hist",
bins=np.linspace(-5, 5, 41),
histogram_type="sidebyside",
vertical_xlabel=True,
hovertool=False,
title="Normal distributions (Side-by-Side)",
line_color="black")
#Stacked histogram:
df_hist.plot_bokeh.hist(
bins=np.linspace(-5, 5, 41),
histogram_type="stacked",
vertical_xlabel=True,
hovertool=False,
title="Normal distributions (Stacked)",
line_color="black")
Further, advanced keyword arguments for histograms are:
Their usage is shown in these examples:
p_hist = df_hist.plot_bokeh.hist(
y=["a", "b"],
bins=np.arange(-4, 6.5, 0.5),
normed=100,
vertical_xlabel=True,
ylabel="Share[%]",
title="Normal distributions (normed)",
show_average=True,
xlim=(-4, 6),
ylim=(0, 30),
show_figure=False)
p_hist_cum = df_hist.plot_bokeh.hist(
y=["a", "b"],
bins=np.arange(-4, 6.5, 0.5),
normed=100,
cumulative=True,
vertical_xlabel=True,
ylabel="Share[%]",
title="Normal distributions (normed & cumulative)",
show_figure=False)
pandas_bokeh.plot_grid([[p_hist, p_hist_cum]], plot_width=450, plot_height=300)
Areaplot (kind="area") can be either drawn on top of each other or stacked. The important parameters are:
stacked: If True, the areaplots are stacked. If False, plots are drawn on top of each other. Default: False
kwargs**: Optional keyword arguments of bokeh.plotting.figure.patch
Let us consider the energy consumption split by source that can be downloaded as DataFrame via:
df_energy = pd.read_csv(r"https://raw.githubusercontent.com/PatrikHlobil/Pandas-Bokeh/master/docs/Testdata/energy/energy.csv",
parse_dates=["Year"])
df_energy.head()
Year | Oil | Gas | Coal | Nuclear Energy | Hydroelectricity | Other Renewable |
---|---|---|---|---|---|---|
1970-01-01 | 2291.5 | 826.7 | 1467.3 | 17.7 | 265.8 | 5.8 |
1971-01-01 | 2427.7 | 884.8 | 1459.2 | 24.9 | 276.4 | 6.3 |
1972-01-01 | 2613.9 | 933.7 | 1475.7 | 34.1 | 288.9 | 6.8 |
1973-01-01 | 2818.1 | 978.0 | 1519.6 | 45.9 | 292.5 | 7.3 |
1974-01-01 | 2777.3 | 1001.9 | 1520.9 | 59.6 | 321.1 | 7.7 |
Creating the Areaplot can be achieved via:
df_energy.plot_bokeh.area(
x="Year",
stacked=True,
legend="top_left",
colormap=["brown", "orange", "black", "grey", "blue", "green"],
title="Worldwide energy consumption split by energy source",
ylabel="Million tonnes oil equivalent",
ylim=(0, 16000))
Note that the energy consumption of fossile energy is still increasing and renewable energy sources are still small in comparison 😢!!! However, when we norm the plot using the normed keyword, there is a clear trend towards renewable energies in the last decade:
df_energy.plot_bokeh.area(
x="Year",
stacked=True,
normed=100,
legend="bottom_left",
colormap=["brown", "orange", "black", "grey", "blue", "green"],
title="Worldwide energy consumption split by energy source",
ylabel="Million tonnes oil equivalent")
Pieplot
For Pieplots, let us consider a dataset showing the results of all Bundestags elections in Germany since 2002:
df_pie = pd.read_csv(r"https://raw.githubusercontent.com/PatrikHlobil/Pandas-Bokeh/master/docs/Testdata/Bundestagswahl/Bundestagswahl.csv")
df_pie
Partei | 2002 | 2005 | 2009 | 2013 | 2017 |
---|---|---|---|---|---|
CDU/CSU | 38.5 | 35.2 | 33.8 | 41.5 | 32.9 |
SPD | 38.5 | 34.2 | 23.0 | 25.7 | 20.5 |
FDP | 7.4 | 9.8 | 14.6 | 4.8 | 10.7 |
Grünen | 8.6 | 8.1 | 10.7 | 8.4 | 8.9 |
Linke/PDS | 4.0 | 8.7 | 11.9 | 8.6 | 9.2 |
AfD | 0.0 | 0.0 | 0.0 | 0.0 | 12.6 |
Sonstige | 3.0 | 4.0 | 6.0 | 11.0 | 5.0 |
We can create a Pieplot of the last election in 2017 by specifying the "Partei" (german for party) column as the x column and the "2017" column as the y column for values:
df_pie.plot_bokeh.pie(
x="Partei",
y="2017",
colormap=["blue", "red", "yellow", "green", "purple", "orange", "grey"],
title="Results of German Bundestag Election 2017",
)
When you pass several columns to the y parameter (not providing the y-parameter assumes you plot all columns), multiple nested pieplots will be shown in one plot:
df_pie.plot_bokeh.pie(
x="Partei",
colormap=["blue", "red", "yellow", "green", "purple", "orange", "grey"],
title="Results of German Bundestag Elections [2002-2017]",
line_color="grey")
Mapplot
The mapplot method of Pandas-Bokeh allows for plotting geographic points stored in a Pandas DataFrame on an interactive map. For more advanced Geoplots for line and polygon shapes have a look at the Geoplots examples for the GeoPandas API of Pandas-Bokeh.
For mapplots, only (latitude, longitude) pairs in geographic projection (WGS84) can be plotted on a map. The basic API has the following 2 base parameters:
The other optional keyword arguments are discussed in the section about the GeoPandas API, e.g. category for coloring the points.
Below an example of plotting all cities for more than 1 million inhabitants:
df_mapplot = pd.read_csv(r"https://raw.githubusercontent.com/PatrikHlobil/Pandas-Bokeh/master/docs/Testdata/populated%20places/populated_places.csv")
df_mapplot.head()
name | pop_max | latitude | longitude | size |
---|---|---|---|---|
Mesa | 1085394 | 33.423915 | -111.736084 | 1.085394 |
Sharjah | 1103027 | 25.371383 | 55.406478 | 1.103027 |
Changwon | 1081499 | 35.219102 | 128.583562 | 1.081499 |
Sheffield | 1292900 | 53.366677 | -1.499997 | 1.292900 |
Abbottabad | 1183647 | 34.149503 | 73.199501 | 1.183647 |
df_mapplot["size"] = df_mapplot["pop_max"] / 1000000
df_mapplot.plot_bokeh.map(
x="longitude",
y="latitude",
hovertool_string="""<h2> @{name} </h2>
<h3> Population: @{pop_max} </h3>""",
tile_provider="STAMEN_TERRAIN_RETINA",
size="size",
figsize=(900, 600),
title="World cities with more than 1.000.000 inhabitants")
Pandas-Bokeh also allows for interactive plotting of Maps using GeoPandas by providing a geopandas.GeoDataFrame.plot_bokeh() method. It allows to plot the following geodata on a map :
Note: t is not possible to mix up the objects types, i.e. a GeoDataFrame with Points and Lines is for example not allowed.
Les us start with a simple example using the "World Borders Dataset" . Let us first import all neccessary libraries and read the shapefile:
import geopandas as gpd
import pandas as pd
import pandas_bokeh
pandas_bokeh.output_notebook()
#Read in GeoJSON from URL:
df_states = gpd.read_file(r"https://raw.githubusercontent.com/PatrikHlobil/Pandas-Bokeh/master/docs/Testdata/states/states.geojson")
df_states.head()
STATE_NAME | REGION | POPESTIMATE2010 | POPESTIMATE2011 | POPESTIMATE2012 | POPESTIMATE2013 | POPESTIMATE2014 | POPESTIMATE2015 | POPESTIMATE2016 | POPESTIMATE2017 | geometry |
---|---|---|---|---|---|---|---|---|---|---|
Hawaii | 4 | 1363817 | 1378323 | 1392772 | 1408038 | 1417710 | 1426320 | 1428683 | 1427538 | (POLYGON ((-160.0738033454681 22.0041773479577... |
Washington | 4 | 6741386 | 6819155 | 6890899 | 6963410 | 7046931 | 7152818 | 7280934 | 7405743 | (POLYGON ((-122.4020153103835 48.2252163723779... |
Montana | 4 | 990507 | 996866 | 1003522 | 1011921 | 1019931 | 1028317 | 1038656 | 1050493 | POLYGON ((-111.4754253002074 44.70216236909688... |
Maine | 1 | 1327568 | 1327968 | 1328101 | 1327975 | 1328903 | 1327787 | 1330232 | 1335907 | (POLYGON ((-69.77727626137293 44.0741483685119... |
North Dakota | 2 | 674518 | 684830 | 701380 | 722908 | 738658 | 754859 | 755548 | 755393 | POLYGON ((-98.73043728833767 45.93827137024809... |
Plotting the data on a map is as simple as calling:
df_states.plot_bokeh(simplify_shapes=10000)
We also passed the optional parameter simplify_shapes (~meter) to improve plotting performance (for a reference see shapely.object.simplify). The above geolayer thus has an accuracy of about 10km.
Many keyword arguments like xlabel, ylabel, xlim, ylim, title, colormap, hovertool, zooming, panning, ... for costumizing the plot are also available for the geoplotting API and can be uses as in the examples shown above. There are however also many other options especially for plotting geodata:
One of the most common usage of map plots are choropleth maps, where the color of a the objects is determined by the property of the object itself. There are 3 ways of drawing choropleth maps using Pandas-Bokeh, which are described below.
This is the simplest way. Just provide the category keyword for the selection of the property column:
Let us now draw the regions as a choropleth plot using the category keyword (at the moment, only numerical columns are supported for choropleth plots):
df_states.plot_bokeh(
figsize=(900, 600),
simplify_shapes=5000,
category="REGION",
show_colorbar=False,
colormap=["blue", "yellow", "green", "red"],
hovertool_columns=["STATE_NAME", "REGION"],
tile_provider="STAMEN_TERRAIN_RETINA")
When hovering over the states, the state-name and the region are shown as specified in the hovertool_columns argument.
By passing a list of column names of the GeoDataFrame as the dropdown keyword argument, a dropdown menu is shown above the map. This dropdown menu can be used to select the choropleth layer by the user. :
df_states["STATE_NAME_SMALL"] = df_states["STATE_NAME"].str.lower()
df_states.plot_bokeh(
figsize=(900, 600),
simplify_shapes=5000,
dropdown=["POPESTIMATE2010", "POPESTIMATE2017"],
colormap="Viridis",
hovertool_string="""
<img
src="https://www.states101.com/img/flags/gif/small/@STATE_NAME_SMALL.gif"
height="42" alt="@imgs" width="42"
style="float: left; margin: 0px 15px 15px 0px;"
border="2"></img>
<h2> @STATE_NAME </h2>
<h3> 2010: @POPESTIMATE2010 </h3>
<h3> 2017: @POPESTIMATE2017 </h3>""",
tile_provider_url=r"http://c.tile.stamen.com/watercolor/{Z}/{X}/{Y}.jpg",
tile_attribution='Map tiles by <a href="http://stamen.com">Stamen Design</a>, under <a href="http://creativecommons.org/licenses/by/3.0">CC BY 3.0</a>. Data by <a href="http://openstreetmap.org">OpenStreetMap</a>, under <a href="http://www.openstreetmap.org/copyright">ODbL</a>.'
)
Using hovertool_string, one can pass a string that can contain arbitrary HTML elements (including divs, images, ...) that is shown when hovering over the geographies (@{column} will be replaced by the value of the column for the element the mouse hovers over, see also Bokeh documentation).
Here, we also used an OSM tile server with watercolor style via tile_provider_url and added the attribution via tile_attribution.
Another option for interactive choropleth maps is the slider implementation of Pandas-Bokeh. The possible keyword arguments are here:
This can be used to display the change in population relative to the year 2010:
#Calculate change of population relative to 2010:
for i in range(8):
df_states["Delta_Population_201%d"%i] = ((df_states["POPESTIMATE201%d"%i] / df_states["POPESTIMATE2010"]) -1 ) * 100
#Specify slider columns:
slider_columns = ["Delta_Population_201%d"%i for i in range(8)]
#Specify slider-range (Maps "Delta_Population_2010" -> 2010,
# "Delta_Population_2011" -> 2011, ...):
slider_range = range(2010, 2018)
#Make slider plot:
df_states.plot_bokeh(
figsize=(900, 600),
simplify_shapes=5000,
slider=slider_columns,
slider_range=slider_range,
slider_name="Year",
colormap="Inferno",
hovertool_columns=["STATE_NAME"] + slider_columns,
title="Change of Population [%]")
If you wish to display multiple geolayers, you can pass the Bokeh figure of a Pandas-Bokeh plot via the figure keyword to the next plot_bokeh() call:
import geopandas as gpd
import pandas_bokeh
pandas_bokeh.output_notebook()
# Read in GeoJSONs from URL:
df_states = gpd.read_file(r"https://raw.githubusercontent.com/PatrikHlobil/Pandas-Bokeh/master/docs/Testdata/states/states.geojson")
df_cities = gpd.read_file(
r"https://raw.githubusercontent.com/PatrikHlobil/Pandas-Bokeh/master/docs/Testdata/populated%20places/ne_10m_populated_places_simple_bigcities.geojson"
)
df_cities["size"] = df_cities.pop_max / 400000
#Plot shapes of US states (pass figure options to this initial plot):
figure = df_states.plot_bokeh(
figsize=(800, 450),
simplify_shapes=10000,
show_figure=False,
xlim=[-170, -80],
ylim=[10, 70],
category="REGION",
colormap="Dark2",
legend="States",
show_colorbar=False,
)
#Plot cities as points on top of the US states layer by passing the figure:
df_cities.plot_bokeh(
figure=figure, # <== pass figure here!
category="pop_max",
colormap="Viridis",
colormap_uselog=True,
size="size",
hovertool_string="""<h1>@name</h1>
<h3>Population: @pop_max </h3>""",
marker="inverted_triangle",
legend="Cities",
)
Below, you can see an example that use Pandas-Bokeh to plot point data on a map. The plot shows all cities with a population larger than 1.000.000. For point plots, you can select the marker as keyword argument (since it is passed to bokeh.plotting.figure.scatter). Here an overview of all available marker types:
gdf = gpd.read_file(r"https://raw.githubusercontent.com/PatrikHlobil/Pandas-Bokeh/master/docs/Testdata/populated%20places/ne_10m_populated_places_simple_bigcities.geojson")
gdf["size"] = gdf.pop_max / 400000
gdf.plot_bokeh(
category="pop_max",
colormap="Viridis",
colormap_uselog=True,
size="size",
hovertool_string="""<h1>@name</h1>
<h3>Population: @pop_max </h3>""",
xlim=[-15, 35],
ylim=[30,60],
marker="inverted_triangle");
In a similar way, also GeoDataFrames with (multi)line shapes can be drawn using Pandas-Bokeh.
If you want to display the numerical labels on your colorbar with an alternative to the scientific format, you can pass in a one of the bokeh number string formats or an instance of one of the bokeh.models.formatters to the colorbar_tick_format
argument in the geoplot
An example of using the string format argument:
df_states = gpd.read_file(r"https://raw.githubusercontent.com/PatrikHlobil/Pandas-Bokeh/master/docs/Testdata/states/states.geojson")
df_states["STATE_NAME_SMALL"] = df_states["STATE_NAME"].str.lower()
# pass in a string format to colorbar_tick_format to display the ticks as 10m rather than 1e7
df_states.plot_bokeh(
figsize=(900, 600),
category="POPESTIMATE2017",
simplify_shapes=5000,
colormap="Inferno",
colormap_uselog=True,
colorbar_tick_format="0.0a")
An example of using the bokeh PrintfTickFormatter
:
df_states = gpd.read_file(r"https://raw.githubusercontent.com/PatrikHlobil/Pandas-Bokeh/master/docs/Testdata/states/states.geojson")
df_states["STATE_NAME_SMALL"] = df_states["STATE_NAME"].str.lower()
for i in range(8):
df_states["Delta_Population_201%d"%i] = ((df_states["POPESTIMATE201%d"%i] / df_states["POPESTIMATE2010"]) -1 ) * 100
# pass in a PrintfTickFormatter instance colorbar_tick_format to display the ticks with 2 decimal places
df_states.plot_bokeh(
figsize=(900, 600),
category="Delta_Population_2017",
simplify_shapes=5000,
colormap="Inferno",
colorbar_tick_format=PrintfTickFormatter(format="%4.2f"))
The pandas.DataFrame.plot_bokeh API has the following additional keyword arguments:
If you have a Bokeh figure or layout, you can also use the pandas_bokeh.embedded_html function to generate an embeddable HTML representation of the plot. This can be included into any valid HTML (note that this is not possible directly with the HTML generated by the pandas_bokeh.output_file output option, because it includes an HTML header). Let us consider the following simple example:
#Import Pandas and Pandas-Bokeh (if you do not specify an output option, the standard is
#output_file):
import pandas as pd
import pandas_bokeh
#Create DataFrame to Plot:
import numpy as np
x = np.arange(-10, 10, 0.1)
sin = np.sin(x)
cos = np.cos(x)
tan = np.tan(x)
df = pd.DataFrame({"x": x, "sin(x)": sin, "cos(x)": cos, "tan(x)": tan})
#Make Bokeh plot from DataFrame using Pandas-Bokeh. Do not show the plot, but export
#it to an embeddable HTML string:
html_plot = df.plot_bokeh(
kind="line",
x="x",
y=["sin(x)", "cos(x)", "tan(x)"],
xticks=range(-20, 20),
title="Trigonometric functions",
show_figure=False,
return_html=True,
ylim=(-1.5, 1.5))
#Write some HTML and embed the HTML plot below it. For production use, please use
#Templates and the awesome Jinja library.
html = r"""
<script type="text/x-mathjax-config">
MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});
</script>
<script type="text/javascript"
src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<h1> Trigonometric functions </h1>
<p> The basic trigonometric functions are:</p>
<p>$ sin(x) $</p>
<p>$ cos(x) $</p>
<p>$ tan(x) = \frac{sin(x)}{cos(x)}$</p>
<p>Below is a plot that shows them</p>
""" + html_plot
#Export the HTML string to an external HTML file and show it:
with open("test.html" , "w") as f:
f.write(html)
import webbrowser
webbrowser.open("test.html")
This code will open up a webbrowser and show the following page. As you can see, the interactive Bokeh plot is embedded nicely into the HTML layout. The return_html option is ideal for the use in a templating engine like Jinja.
For single plots that have a number of x axis values or for larger monitors, you can auto scale the figure to the width of the entire jupyter cell by setting the sizing_mode
parameter.
df = pd.DataFrame(np.random.rand(10, 4), columns=['a', 'b', 'c', 'd']) df.plot_bokeh(kind="bar", figsize=(500, 200), sizing_mode="scale_width")
The figsize
parameter can be used to change the height and width as well as act as a scaling multiplier against the axis that is not being scaled.
To change the formats of numbers in the hovertool, use the number_format keyword argument. For a documentation about the format to pass, have a look at the Bokeh documentation.Let us consider some examples for the number 3.141592653589793:
Format | Output |
---|---|
0 | 3 |
0.000 | 3.141 |
0.00 $ | 3.14 $ |
This number format will be applied to all numeric columns of the hovertool. If you want to make a very custom or complicated hovertool, you should probably use the hovertool_string keyword argument, see e.g. this example. Below, we use the number_format parameter to specify the "Stock Price" format to 2 decimal digits and an additional $ sign.
import numpy as np
#Lineplot:
np.random.seed(42)
df = pd.DataFrame({
"Google": np.random.randn(1000) + 0.2,
"Apple": np.random.randn(1000) + 0.17
},
index=pd.date_range('1/1/2000', periods=1000))
df = df.cumsum()
df = df + 50
df.plot_bokeh(
kind="line",
title="Apple vs Google",
xlabel="Date",
ylabel="Stock price [$]",
yticks=[0, 100, 200, 300, 400],
ylim=(0, 400),
colormap=["red", "blue"],
number_format="1.00 $")
If you want to suppress the scientific notation for axes, you can use the disable_scientific_axes parameter, which accepts one of "x", "y", "xy":
df = pd.DataFrame({"Animal": ["Mouse", "Rabbit", "Dog", "Tiger", "Elefant", "Wale"],
"Weight [g]": [19, 3000, 40000, 200000, 6000000, 50000000]})
p_scientific = df.plot_bokeh(x="Animal", y="Weight [g]", show_figure=False)
p_non_scientific = df.plot_bokeh(x="Animal", y="Weight [g]", disable_scientific_axes="y", show_figure=False,)
pandas_bokeh.plot_grid([[p_scientific, p_non_scientific]], plot_width = 450)
As shown in the Scatterplot Example, combining plots with plots or other HTML elements is straighforward in Pandas-Bokeh due to the layout capabilities of Bokeh. The easiest way to generate a dashboard layout is using the pandas_bokeh.plot_grid method (which is an extension of bokeh.layouts.gridplot):
import pandas as pd
import numpy as np
import pandas_bokeh
pandas_bokeh.output_notebook()
#Barplot:
data = {
'fruits':
['Apples', 'Pears', 'Nectarines', 'Plums', 'Grapes', 'Strawberries'],
'2015': [2, 1, 4, 3, 2, 4],
'2016': [5, 3, 3, 2, 4, 6],
'2017': [3, 2, 4, 4, 5, 3]
}
df = pd.DataFrame(data).set_index("fruits")
p_bar = df.plot_bokeh(
kind="bar",
ylabel="Price per Unit [€]",
title="Fruit prices per Year",
show_figure=False)
#Lineplot:
np.random.seed(42)
df = pd.DataFrame({
"Google": np.random.randn(1000) + 0.2,
"Apple": np.random.randn(1000) + 0.17
},
index=pd.date_range('1/1/2000', periods=1000))
df = df.cumsum()
df = df + 50
p_line = df.plot_bokeh(
kind="line",
title="Apple vs Google",
xlabel="Date",
ylabel="Stock price [$]",
yticks=[0, 100, 200, 300, 400],
ylim=(0, 400),
colormap=["red", "blue"],
show_figure=False)
#Scatterplot:
from sklearn.datasets import load_iris
iris = load_iris()
df = pd.DataFrame(iris["data"])
df.columns = iris["feature_names"]
df["species"] = iris["target"]
df["species"] = df["species"].map(dict(zip(range(3), iris["target_names"])))
p_scatter = df.plot_bokeh(
kind="scatter",
x="petal length (cm)",
y="sepal width (cm)",
category="species",
title="Iris DataSet Visualization",
show_figure=False)
#Histogram:
df_hist = pd.DataFrame({
'a': np.random.randn(1000) + 1,
'b': np.random.randn(1000),
'c': np.random.randn(1000) - 1
},
columns=['a', 'b', 'c'])
p_hist = df_hist.plot_bokeh(
kind="hist",
bins=np.arange(-6, 6.5, 0.5),
vertical_xlabel=True,
normed=100,
hovertool=False,
title="Normal distributions",
show_figure=False)
#Make Dashboard with Grid Layout:
pandas_bokeh.plot_grid([[p_line, p_bar],
[p_scatter, p_hist]], plot_width=450)
Using a combination of row and column elements (see also Bokeh Layouts) allow for a very easy general arrangement of elements. An alternative layout to the one above is:
p_line.plot_width = 900
p_hist.plot_width = 900
layout = pandas_bokeh.column(p_line,
pandas_bokeh.row(p_scatter, p_bar),
p_hist)
pandas_bokeh.show(layout)
Release Notes
Release Notes can be found here.
Contributing to Pandas-Bokeh
If you wish to contribute to the development of Pandas-Bokeh
you can follow the instructions on the CONTRIBUTING.md.
Author: PatrikHlobil
Source Code: https://github.com/PatrikHlobil/Pandas-Bokeh
License: MIT License