1585187105
How to Extract Data using Spotify’s API, Python and Spotipy
With Spotify, they provide developers access to some of their data regarding playlists, users, and artists through their Web API, which is what I’ll be showing you how to do.
A lot of times, developers will use Web APIs for app integrations, but it can also be used for data extraction and analysis purposes. Here is what I will be walking you through:
- Creating a Spotify developer account.
- Understanding Spotify’s Web API.
- What data we’ll extract.
- What Spotipy is and does.
- How to extract any artist’s data using Python and Spotipy.
Creating a Spotify Developer Account
The first step in using Spotify’s Web API is to create a developer account. Follow these steps to access your developer account to get a client and secret ID.
Step 1: Log in or create an account
You will need to connect a Spotify developer account by logging into your account or creating a new Spotify account.
Step 2: Client ID
When you’ve accessed your developer account, you’ll need to create a client ID, which you can do by pressing the green CREATE A CLIENT ID button.
You will then get a three-page pop up that you’ll need to fill out and each page will look like this:
Although we’re not trying to create an app, we will need this client ID to access the same data.
- Page one:Give it a name. I named mine “Album Analysis”. The description I put in is “analyzing Spotify artist’s data”, and then I checked the box that says “Desktop App” under “What are you building?”.
- Page two:Choose non-commercial use.
- Page three:Agree to all the terms and conditions and submit the form.
Step 3: Retrieve your client ID and client secret
Inside your developer dashboard, click on the new app you just created. On your app’s dashboard page, you’ll see your client ID on the top left-hand side.
Underneath your client ID, you’ll see “Show Client Secret” in green. If you press that, you’ll get your client secret. Keep these locations in mind as we work towards our data extraction, we will need them to gain access to the data.
Understanding Spotify’s Web API
I would recommend exploring Spotify’s user-friendly documentation and familiarizing yourself with different areas of how it works. Here are some quick links you can dive into:
- Spotify’s Web API documentation
- Web API tutorial
- Web API libraries
- Spotify Web API reference (useful for exploring the data endpoints we have access to and accessing the related documentation)
Spotify’s Web API console
Another great way to explore the different data Spotify has to offer and to understand the Web API is to explore their Spotify Web API console.
This console lets you explore different endpoints through an easy-to-use interface and test the various methods to preview the JSON outputs.
The Data We’ll Extract
For this guide, I want to extract the data from every album and single (no features) that Kehlani has put out. There are a few potential ways to create a dataset using this API.
We could request a list of the artist’s albums and then loop through each album track.
Or, instead, we can use a playlist that I created which includes every album Kehlani has to offer on Spotify and loop through that which would require a little less code.
Feel free to create your own playlist if you want to work on a different artist’s data instead.
The meta and features data
When grabbing each track from an album, we can obtain track information such as track name, album, release date, length, and popularity.
More importantly, Spotify’s API allows us to extract a number of “audio features” such as danceability, energy, instrumentalness, liveness, loudness, speechiness, acousticness, and tempo.
Using Spotipy
Spotipy is “a lightweight Python library for the Spotify Web API”. With Spotipy, we can get full access to all of the music data provided by the Spotify platform.
I highly recommend going through Spotipy’s documentation to get an understanding of how it works and what you’re capable of doing while using this library.
Additionally, you will get a better understanding of how my code works. Check out their GitHub examples.
Writing the Code
I highly recommend following along using Repl, it is a simple to use yet powerful online IDE that works great and requires no setup. Perfect for a one-time data extraction or dataset creation project.
Import libraries
Let’s start by calling the necessary libraries that we need. We’ll be using Spotipy, pandas to create a dataframe and save our dataset, and time to pause the execution of the loop.
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
import pandas as pd
import time
Connect to the API
Next, we need to authenticate and connect to Spotify’s API. To do so, we need our “Client ID” and “Client Secret”.
client_id = 'Client ID Here'
client_secret = 'Client Secret Here'
client_credentials_manager = SpotifyClientCredentials(client_id, client_secret)
sp = spotipy.Spotify(client_credentials_manager=client_credentials_manager)
In the code above, replace the Client ID and Client Secret variables with your own and make sure they are inside quotes.
Retrieve IDs for each track
As I mentioned earlier, I’ll be extracting Kehlani’s albums and singles from a playlist I created which is a collection of 54 songs (~3 hours), containing every single and every song from every album she has released that is currently on Spotify, updated with her new singles.
Now we’ll write a function to get the IDs for each track of this playlist.
def getTrackIDs(user, playlist_id):
ids = []
playlist = sp.user_playlist(user, playlist_id)
for item in playlist['tracks']['items']:
track = item['track']
ids.append(track['id'])
return ids
ids = getTrackIDs('angelicadietzel', '4R0BZVh27NUJhHGLNitU08')
At the bottom of this code, where it says ids = getPlaylistTrackIDs, the two variables separated by a comma in quotes will be the username (found in the URL) of the person who created the playlist, and then the playlist URI which you can find by hitting the setting button on the playlist where you’d find the share link.
Now, let’s check what we have so far by running the length of the ids we grabbed to see if it matches the 54 songs we have on the playlist.
print(len(ids))
print(ids)
This is the result:
54
['7AiMnJSODcJoKDejQ3mnoJ', '73C4vh7W8u41Vll5HvBqv7', '5kYZbBLAGrrhFKNbOs6D95', '18z6OV5lknJmKnZi7aA1zH', '1gHtbcRP4tz1O1NsxPpBea', '3kJudfRjZMItdFYVCCaSi6', '6mzaCRuLTRiz1caGOum3zT', '5h4Uqkh9RpRZwm5ADLh5uj', '3rGew9pmFEmGD9nZ12F1tN', '0dYDmow4l5hbPs5E6QLMSC', '1B3jkf6CyuiF8CQcKlUx9y', '6DkmFhzJrkVhDlcgcEy7Pc', '5dKy6Cgv6xwiRY3j3AJ7Uq', '0oz4ZqHuUaz3uEkP2vD0u8', '389hKTL3ZBPPWP3VuXfEyv', '5cw9s2zGrbny2M2p3WRmGm', '3YaMX9Cf68dxiG6RKo0pSY', '1cAL4sFzXXRMbpZnTPa7Zi', '4w5BVeKJFCj2rrrEy31s0n', '4ta2AWru6ldjg1aHzww0aK', '45DJ0PbKPdbslnyrcM80HN', '7yNu82yd6dYmGQ0H1q0jKo', '4v4HwTfMPslhWAnJxIXchn', '6XptjfnUvLfejptpjPRhCT', '1y2SK8EjL3WSnJvJEMWOoq', '4UMp46x46Zmu9OEr8m3Gl2', '7nb50hgKYhnHJLHKZ7qiKO','5y5OzukBTl0yTRMEdNmApJ', '3Hdl3BEFb1IEbL0Jq53enx', '0lsC0OkBgiLYbSsoHOzMnr', '0Pm1BZp4MpoMKkNxIXCfAu', '2droOB3xlZkhgfUM0owDTq', '2Nd2HLWrIq1DcNMiYPTQUC', '4j644tViOFAf4i0BYT12R8', '4k6hX9RKD096K1NCjjJZLc', '0aSW5EMeNnQSMJQ8QN3zIW', '0tkmYNfaEaH9HpR59ApRtE', '0kas95RruYRVqrOb07rgkh', '4BOikd4oZjOYMde9AXfrTo', '6ZRuF2n1CQxyxxAAWsKJOy', '1EGrDTfEuAiRzRdxlblpET', '32s2Dn9EVvO2f85MrpRoBV', '4jM3c9KLTO9iZPm9A7neiL', '6GCRnf1W9OKxok9fvNp3pz', '1Zm9qGPQkTAOBiVpGSnJUq', '3ucRKbRlikYHyoI17gfR0c', '6HUO25AttZZCoKAY0vUVtc', '5Qr7StTFbXhgHt9JlqJx0I', '1QzC4y8h6WFxHE4KlokhVr', '1xz905v9g71heS0BQQM9re', '5QTdOvIF2ehBMZpSIIGzIo', '7IJiDYPZy2AIJn3YVHhvD4', '23wuZgeX1oyJ43QYOTo9s7', '0tBBihoEWiWKqsO5ZlCbwS']
Create a function used to grab all track info from IDs
Now that we have a list of every track ids we need, we will now write a function that will be used to grab the track information such as track name, album, release date, length, and popularity, along with the list of audio features that Spotify’s API has to offer from each track ids.
def getTrackFeatures(id):
meta = sp.track(id)
features = sp.audio_features(id)
# meta
name = meta['name']
album = meta['album']['name']
artist = meta['album']['artists'][0]['name']
release_date = meta['album']['release_date']
length = meta['duration_ms']
popularity = meta['popularity']
# features
acousticness = features[0]['acousticness']
danceability = features[0]['danceability']
energy = features[0]['energy']
instrumentalness = features[0]['instrumentalness']
liveness = features[0]['liveness']
loudness = features[0]['loudness']
speechiness = features[0]['speechiness']
tempo = features[0]['tempo']
time_signature = features[0]['time_signature']
track = [name, album, artist, release_date, length, popularity, danceability, acousticness, danceability, energy, instrumentalness, liveness, loudness, speechiness, tempo, time_signature]
return track
Loop over tracks and apply the function
We’ll now loop over the tracks — applying the function we created— and save the dataset to a .csv file using pandas.
# loop over track ids
tracks = []
for i in range(len(ids)):
time.sleep(.5)
track = getTrackFeatures(ids[i])
tracks.append(track)
# create dataset
df = pd.DataFrame(tracks, columns = ['name', 'album', 'artist', 'release_date', 'length', 'popularity', 'danceability', 'acousticness', 'danceability', 'energy', 'instrumentalness', 'liveness', 'loudness', 'speechiness', 'tempo', 'time_signature'])
df.to_csv("spotify.csv", sep = ',')
My raw dataset looks like this:
There You Have It!
This is how you can extract Spotify data on any artist you like.
The easiest way I’ve found was simply creating my own playlist to loop through, which offered fewer errors and put everything all in one place, but there are many other ways you can go about this.
Next Steps
The next step would be to clean up this data and then analyze it to gain insights into the artist’s music over the years.
Different ways you can analyze and visualize Spotify’s data could be:
- The number of tracks over time.
- Visualize the length of tracks over time.
- Average danceablity over time.
- Top collaborators.
I hope this was a quick and clear intro to Spotify’s Web API and Spotipy!
Thank you for reading!
#python #spotify #data science #ai #machine learning
What is GEEK
Buddha Community
1646946485
Interesting article! Do you know how you can increase the popularity of your tracks in spotify? I would be very interested to know about it!
1646954289
I think this information will be very useful for many! But if you want to become a popular performer in spotify, then I recommend that you learn how to do it on this site. Thanks to this service, you can easily promote your new songs and playlists to the tops of spotify! Use it, it's really very cool and useful!
1669188856
What Is R Programming Language? introduction & Basics
In this R article, we will learn about What Is R Programming Language? introduction & Basics. R is a programming language developed by Ross Ihaka and Robert Gentleman in 1993. R possesses an extensive catalog of statistical and graphical methods. It includes machine learning algorithms, linear regression, time series, statistical inference to name a few. Most of the R libraries are written in R, but for heavy computational tasks, C, C++, and Fortran codes are preferred.
Data analysis with R is done in a series of steps; programming, transforming, discovering, modeling and communicating the results
- Program: R is a clear and accessible programming tool
- Transform: R is made up of a collection of libraries designed specifically for data science
- Discover: Investigate the data, refine your hypothesis and analyze them
- Model: R provides a wide array of tools to capture the right model for your data
- Communicate: Integrate codes, graphs, and outputs to a report with R Markdown or build Shiny apps to share with the world.
What is R used for?
- Statistical inference
- Data analysis
- Machine learning algorithm
As conclusion, R is the world’s most widely used statistics programming language. It’s the 1st choice of data scientists and supported by a vibrant and talented community of contributors. R is taught in universities and deployed in mission-critical business applications.
R-environment setup
Windows Installation – We can download the Windows installer version of R from R-3.2.2 for windows (32/64)
As it is a Windows installer (.exe) with the name “R-version-win.exe”. You can just double click and run the installer accepting the default settings. If your Windows is a 32-bit version, it installs the 32-bit version. But if your windows are 64-bit, then it installs both the 32-bit and 64-bit versions.
After installation, you can locate the icon to run the program in a directory structure “R\R3.2.2\bin\i386\Rgui.exe” under the Windows Program Files. Clicking this icon brings up the R-GUI which is the R console to do R Programming.
R basic Syntax
R Programming is a very popular programming language that is broadly used in data analysis. The way in which we define its code is quite simple. The “Hello World!” is the basic program for all the languages, and now we will understand the syntax of R programming with the “Hello world” program. We can write our code either in the command prompt, or we can use an R script file.
R command prompt
Once you have R environment setup, then it’s easy to start your R command prompt by just typing the following command at your command prompt −
$R
This will launch R interpreter and you will get a prompt > where you can start typing your program as follows −
>myString <- "Hello, World"
>print (myString)
[1] "Hello, World!"Here the first statement defines a string variable myString, where we assign a string “Hello, World!” and then the next statement print() is being used to print the value stored in myString variable.
R data-types
While doing programming in any programming language, you need to use various variables to store various information. Variables are nothing but reserved memory locations to store values. This means that when you create a variable you reserve some space in memory.
In contrast to other programming languages like C and java in R, the variables are not declared as some data type. The variables are assigned with R-Objects and the data type of the R-object becomes the data type of the variable. There are many types of R-objects. The frequently used ones are −
- Vectors
- Lists
- Matrices
- Arrays
- Factors
- Data Frames
Vectors
#create a vector and find the elements which are >5
v<-c(1,2,3,4,5,6,5,8)
v[v>5]
#subset
subset(v,v>5)
#position in the vector created in which square of the numbers of v is >10 holds good
which(v*v>10)
#to know the values
v[v*v>10]Output: [1] 6 8
Output: [1] 6 8
Output: [1] 4 5 6 7 8
Output: [1] 4 5 6 5 8
Matrices
A matrix is a two-dimensional rectangular data set. It can be created using a vector input to the matrix function.
#matrices: a vector with two dimensional attributes
mat<-matrix(c(1,2,3,4))
mat1<-matrix(c(1,2,3,4),nrow=2)
mat1Output: [,1] [,2] [1,] 1 3 [2,] 2 4
mat2<-matrix(c(1,2,3,4),ncol=2,byrow=T)
mat2Output: [,1] [,2] [1,] 1 2 [2,] 3 4
mat3<-matrix(c(1,2,3,4),byrow=T)
mat3
#transpose of matrix
mattrans<-t(mat)
mattrans
#create a character matrix called fruits with elements apple, orange, pear, grapes
fruits<-matrix(c("apple","orange","pear","grapes"),2)
#create 3×4 matrix of marks obtained in each quarterly exams for 4 different subjects
X<-matrix(c(50,70,40,90,60, 80,50, 90,100, 50,30, 70),nrow=3)
X
#give row names and column names
rownames(X)<-paste(prefix="Test.",1:3)
subs<-c("Maths", "English", "Science", "History")
colnames(X)<-subs
XOutput: [,1] [1,] 1 [2,] 2 [3,] 3 [4,] 4 Output: [,1] [,2] [,3] [,4] [1,] 1 2 3 4 Output: [,1] [,2] [,3] [,4] [1,] 50 90 50 50 [2,] 70 60 90 30 [3,] 40 80 100 70 Output: Maths English Science History Test. 1 50 90 50 50 Test. 2 70 60 90 30 Test. 3 40 80 100 70
Arrays
While matrices are confined to two dimensions, arrays can be of any number of dimensions. The array function takes a dim attribute which creates the required number of dimensions. In the below example we create an array with two elements which are 3×3 matrices each.
#Arrays
arr<-array(1:24,dim=c(3,4,2))
arr
#create an array using alphabets with dimensions 3 rows, 2 columns and 3 arrays
arr1<-array(letters[1:18],dim=c(3,2,3))
#select only 1st two matrix of an array
arr1[,,c(1:2)]
#LIST
X<-list(u=2, n='abc')
X
X$u [,1] [,2] [,3] [,4] [,1] [,2] [,3] [,4] [,1] [,2] [,1] [,2]Dataframes
Data frames are tabular data objects. Unlike a matrix in a data frame, each column can contain different modes of data. The first column can be numeric while the second column can be character and the third column can be logical. It is a list of vectors of equal length.
#Dataframes
students<-c("J","L","M","K","I","F","R","S")
Subjects<-rep(c("science","maths"),each=2)
marks<-c(55,70,66,85,88,90,56,78)
data<-data.frame(students,Subjects,marks)
#Accessing dataframes
data[[1]]
data$Subjects
data[,1]Output: [1] J L M K I F R S Levels: F I J K L M R S Output: data$Subjects [1] science science maths maths science science maths maths Levels: maths science
Factors
Factors are the r-objects which are created using a vector. It stores the vector along with the distinct values of the elements in the vector as labels. The labels are always character irrespective of whether it is numeric or character or Boolean etc. in the input vector. They are useful in statistical modeling.
Factors are created using the factor() function. The nlevels function gives the count of levels.
#Factors
x<-c(1,2,3)
factor(x)
#apply function
data1<-data.frame(age=c(55,34,42,66,77),bmi=c(26,25,21,30,22))
d<-apply(data1,2,mean)
d
#create two vectors age and gender and find mean age with respect to gender
age<-c(33,34,55,54)
gender<-factor(c("m","f","m","f"))
tapply(age,gender,mean)
Output: [1] 1 2 3 Levels: 1 2 3 Output: age bmi 54.8 24.8 Output: f m 44 44
R Variables
A variable provides us with named storage that our programs can manipulate. A variable in R can store an atomic vector, a group of atomic vectors, or a combination of many R objects. A valid variable name consists of letters, numbers, and the dot or underlines characters.
Rules for writing Identifiers in R
- Identifiers can be a combination of letters, digits, period (.), and underscore (_).
- It must start with a letter or a period. If it starts with a period, it cannot be followed by a digit.
- Reserved words in R cannot be used as identifiers.
Valid identifiers in R
total, sum, .fine.with.dot, this_is_acceptable, Number5
Invalid identifiers in R
tot@l, 5um, _fine, TRUE, .0ne
Best Practices
Earlier versions of R used underscore (_) as an assignment operator. So, the period (.) was used extensively in variable names having multiple words. Current versions of R support underscore as a valid identifier but it is good practice to use a period as word separators.
For example, a.variable.name is preferred over a_variable_name or alternatively we could use camel case as aVariableName.
Constants in R
Constants, as the name suggests, are entities whose value cannot be altered. Basic types of constant are numeric constants and character constants.
Numeric Constants
All numbers fall under this category. They can be of type integer, double or complex. It can be checked with the typeof() function.
Numeric Constants followed by L are regarded as integers and those followed by i are regarded as complex.
> typeof(5)
> typeof(5L)
> typeof(5L)[1] “double” [1] “double” [[1] “double”
Character Constants
Character constants can be represented using either single quotes (‘) or double quotes (“) as delimiters.
> 'example'
> typeof("5")[1] "example" [1] "character"
R Operators
Operators – Arithmetic, Relational, Logical, Assignment, and some of the Miscellaneous Operators that R programming language provides.
There are four main categories of Operators in the R programming language.
- Arithmetic Operators
- Relational Operators
- Logical Operators
- Assignment Operators
- Mixed Operators
x <- 35
y<-10x+y > x-y > x*y > x/y > x%/%y > x%%y > x^y [1] 45 [1] 25 [1] 350 [1] 3.5 [1] 3 [1] 5 [1]2.75e+15
Logical Operators
The below table shows the logical operators in R. Operators & and | perform element-wise operation producing result having a length of the longer operand. But && and || examines only the first element of the operands resulting in a single length logical vector.
a <- c(TRUE,TRUE,FALSE,0,6,7)
b <- c(FALSE,TRUE,FALSE,TRUE,TRUE,TRUE)
a&b
[1] FALSE TRUE FALSE FALSE TRUE TRUE
a&&b
[1] FALSE
> a|b
[1] TRUE TRUE FALSE TRUE TRUE TRUE
> a||b
[1] TRUE
> !a
[1] FALSE FALSE TRUE TRUE FALSE FALSE
> !b
[1] TRUE FALSE TRUE FALSE FALSE FALSE
R functions
Functions are defined using the function() directive and are stored as R objects just like anything else. In particular, they are R objects of class “function”. Here’s a simple function that takes no arguments simply prints ‘Hi statistics’.
#define the function
f <- function() {
print("Hi statistics!!!")
}
#Call the function
f()Output: [1] "Hi statistics!!!"
Now let’s define a function called standardize, and the function has a single argument x which is used in the body of a function.
#Define the function that will calculate standardized score.
standardize = function(x) {
m = mean(x)
sd = sd(x)
result = (x – m) / sd
result
}
input<- c(40:50) #Take input for what we want to calculate a standardized score.
standardize(input) #Call the functionOutput: standardize(input) #Call the function [1] -1.5075567 -1.2060454 -0.9045340 -0.6030227 -0.3015113 0.0000000 0.3015113 0.6030227 0.9045340 1.2060454 1.5075567
Loop Functions
R has some very useful functions which implement looping in a compact form to make life easier. The very rich and powerful family of applied functions is made of intrinsically vectorized functions. These functions in R allow you to apply some function to a series of objects (eg. vectors, matrices, data frames, or files). They include:
- lapply(): Loop over a list and evaluate a function on each element
- sapply(): Same as lapply but try to simplify the result
- apply(): Apply a function over the margins of an array
- tapply(): Apply a function over subsets of a vector
- mapply(): Multivariate version of lapply
There is another function called split() which is also useful, particularly in conjunction with lapply.
R Vectors
A vector is a sequence of data elements of the same basic type. Members in a vector are officially called components. Vectors are the most basic R data objects and there are six types of atomic vectors. They are logical, integer, double, complex, character, and raw.
The c() function can be used to create vectors of objects by concatenating things together.
x <- c(1,2,3,4,5) #double
x #If you use only x auto-printing occurs
l <- c(TRUE, FALSE) #logical
l <- c(T, F) ## logical
c <- c("a", "b", "c", "d") ## character
i <- 1:20 ## integer
cm <- c(2+2i, 3+3i) ## complex
print(l)
print(c)
print(i)
print(cm)
You can see the type of each vector using typeof() function in R.
typeof(x)
typeof(l)
typeof(c)
typeof(i)
typeof(cm)Output: print(l) [1] TRUE FALSE print(c) [1] "a" "b" "c" "d" print(i) [1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 print(cm) [1] 2+2i 3+3i Output: typeof(x) [1] "double" typeof(l) [1] "logical" typeof(c) [1] "character" typeof(i) [1] "integer" typeof(cm) [1] "complex"
Creating a vector using seq() function:
We can use the seq() function to create a vector within an interval by specifying step size or specifying the length of the vector.
seq(1:10) #By default it will be incremented by 1
seq(1, 20, length.out=5) # specify length of the vector
seq(1, 20, by=2) # specify step sizeOutput: > seq(1:10) #By default it will be incremented by 1 [1] 1 2 3 4 5 6 7 8 9 10 > seq(1, 20, length.out=5) # specify length of the vector [1] 1.00 5.75 10.50 15.25 20.00 > seq(1, 20, by=2) # specify step size [1] 1 3 5 7 9 11 13 15 17 19
Extract Elements from a Vector:
Elements of a vector can be accessed using indexing. The vector indexing can be logical, integer, or character. The [ ] brackets are used for indexing. Indexing starts with position 1, unlike most programming languages where indexing starts from 0.
Extract Using Integer as Index:
We can use integers as an index to access specific elements. We can also use negative integers to return all elements except that specific element.
x<- 101:110
x[1] #access the first element
x[c(2,3,4,5)] #Extract 2nd, 3rd, 4th, and 5th elements
x[5:10] #Extract all elements from 5th to 10th
x[c(-5,-10)] #Extract all elements except 5th and 10th
x[-c(5:10)] #Extract all elements except from 5th to 10th
Output: x[1] #Extract the first element [1] 101 x[c(2,3,4,5)] #Extract 2nd, 3rd, 4th, and 5th elements [1] 102 103 104 105 x[5:10] #Extract all elements from 5th to 10th [1] 105 106 107 108 109 110 x[c(-5,-10)] #Extract all elements except 5th and 10th [1] 101 102 103 104 106 107 108 109 x[-c(5:10)] #Extract all elements except from 5th to 10th [1] 101 102 103 104
Extract Using Logical Vector as Index:
If you use a logical vector for indexing, the position where the logical vector is TRUE will be returned.
x[x < 105]
x[x>=104]Output: x[x < 105] [1] 101 102 103 104 x[x>=104] [1] 104 105 106 107 108 109 110
Modify a Vector in R:
We can modify a vector and assign a new value to it. You can truncate a vector by using reassignments. Check the below example.
x<- 10:12
x[1]<- 101 #Modify the first element
x
x[2]<-102 #Modify the 2nd element
x
x<- x[1:2] #Truncate the last element
x
Output: x [1] 101 11 12 x[2]<-102 #Modify the 2nd element x [1] 101 102 12 x<- x[1:2] #Truncate the last element x [1] 101 102
Arithmetic Operations on Vectors:
We can use arithmetic operations on two vectors of the same length. They can be added, subtracted, multiplied, or divided. Check the output of the below code.
# Create two vectors.
v1 <- c(1:10)
v2 <- c(101:110)
# Vector addition.
add.result <- v1+v2
print(add.result)
# Vector subtraction.
sub.result <- v2-v1
print(sub.result)
# Vector multiplication.
multi.result <- v1*v2
print(multi.result)
# Vector division.
divi.result <- v2/v1
print(divi.result)
Output: print(add.result) [1] 102 104 106 108 110 112 114 116 118 120 print(sub.result) [1] 100 100 100 100 100 100 100 100 100 100 print(multi.result) [1] 101 204 309 416 525 636 749 864 981 1100 print(divi.result) [1] 101.00000 51.00000 34.33333 26.00000 21.00000 17.66667 15.28571 13.50000 12.11111 11.00000
Find Minimum and Maximum in a Vector:
The minimum and the maximum of a vector can be found using the min() or the max() function. range() is also available which returns the minimum and maximum in a vector.
x<- 1001:1010
max(x) # Find the maximum
min(x) # Find the minimum
range(x) #Find the rangeOutput: max(x) # Find the maximum [1] 1010 min(x) # Find the minimum [1] 1001 range(x) #Find the range [1] 1001 1010
R Lists
The list is a data structure having elements of mixed data types. A vector having all elements of the same type is called an atomic vector but a vector having elements of a different type is called list.
We can check the type with typeof() or class() function and find the length using length()function.
x <- list("stat",5.1, TRUE, 1 + 4i)
x
class(x)
typeof(x)
length(x)
Output: x [[1]] [1] "stat" [[2]] [1] 5.1 [[3]] [1] TRUE [[4]] [1] 1+4i class(x) [1] “list” typeof(x) [1] “list” length(x) [1] 4
You can create an empty list of a prespecified length with the vector() function.
x <- vector("list", length = 10)
xOutput: x [[1]] NULL [[2]] NULL [[3]] NULL [[4]] NULL [[5]] NULL [[6]] NULL [[7]] NULL [[8]] NULL [[9]] NULL [[10]] NULL
How to extract elements from a list?
Lists can be subset using two syntaxes, the $ operator, and square brackets []. The $ operator returns a named element of a list. The [] syntax returns a list, while the [[]] returns an element of a list.
# subsetting
l$e
l["e"]
l[1:2]
l[c(1:2)] #index using integer vector
l[-c(3:length(l))] #negative index to exclude elements from 3rd up to last.
l[c(T,F,F,F,F)] # logical index to access elementsOutput: > l$e [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [1,] 1 0 0 0 0 0 0 0 0 0 [2,] 0 1 0 0 0 0 0 0 0 0 [3,] 0 0 1 0 0 0 0 0 0 0 [4,] 0 0 0 1 0 0 0 0 0 0 [5,] 0 0 0 0 1 0 0 0 0 0 [6,] 0 0 0 0 0 1 0 0 0 0 [7,] 0 0 0 0 0 0 1 0 0 0 [8,] 0 0 0 0 0 0 0 1 0 0 [9,] 0 0 0 0 0 0 0 0 1 0 [10,] 0 0 0 0 0 0 0 0 0 1 > l["e"] $e [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [1,] 1 0 0 0 0 0 0 0 0 0 [2,] 0 1 0 0 0 0 0 0 0 0 [3,] 0 0 1 0 0 0 0 0 0 0 [4,] 0 0 0 1 0 0 0 0 0 0 [5,] 0 0 0 0 1 0 0 0 0 0 [6,] 0 0 0 0 0 1 0 0 0 0 [7,] 0 0 0 0 0 0 1 0 0 0 [8,] 0 0 0 0 0 0 0 1 0 0 [9,] 0 0 0 0 0 0 0 0 1 0 [10,] 0 0 0 0 0 0 0 0 0 1 > l[1:2] [[1]] [1] 1 2 3 4 [[2]] [1] FALSE > l[c(1:2)] #index using integer vector [[1]] [1] 1 2 3 4 [[2]] [1] FALSE > l[-c(3:length(l))] #negative index to exclude elements from 3rd up to last. [[1]] [1] 1 2 3 4 [[2]] [1] FALSE l[c(T,F,F,F,F)] [[1]] [1] 1 2 3 4
Modifying a List in R:
We can change components of a list through reassignment.
l[["name"]] <- "Kalyan Nandi"
lOutput: [[1]] [1] 1 2 3 4 [[2]] [1] FALSE [[3]] [1] “Hello Statistics!” $d function (arg = 42) { print(“Hello World!”) } $name [1] “Kalyan Nandi”
R Matrices
In R Programming Matrix is a two-dimensional data structure. They contain elements of the same atomic types. A Matrix can be created using the matrix() function. R can also be used for matrix calculations. Matrices have rows and columns containing a single data type. In a matrix, the order of rows and columns is important. Dimension can be checked directly with the dim() function and all attributes of an object can be checked with the attributes() function. Check the below example.
Creating a matrix in R
m <- matrix(nrow = 2, ncol = 3)
dim(m)
attributes(m)
m <- matrix(1:20, nrow = 4, ncol = 5)
mOutput: dim(m) [1] 2 3 attributes(m) $dim [1] 2 3 m <- matrix(1:20, nrow = 4, ncol = 5) m [,1] [,2] [,3] [,4] [,5] [1,] 1 5 9 13 17 [2,] 2 6 10 14 18 [3,] 3 7 11 15 19 [4,] 4 8 12 16 20
Matrices can be created by column-binding or row-binding with the cbind() and rbind() functions.
x<-1:3
y<-10:12
z<-30:32
cbind(x,y,z)
rbind(x,y,z)Output: cbind(x,y,z) x y z [1,] 1 10 30 [2,] 2 11 31 [3,] 3 12 32 rbind(x,y,z) [,1] [,2] [,3] x 1 2 3 y 10 11 12 z 30 31 32
By default, the matrix function reorders a vector into columns, but we can also tell R to use rows instead.
x <-1:9
matrix(x, nrow = 3, ncol = 3)
matrix(x, nrow = 3, ncol = 3, byrow = TRUE)
Output cbind(x,y,z) x y z [1,] 1 10 30 [2,] 2 11 31 [3,] 3 12 32 rbind(x,y,z) [,1] [,2] [,3] x 1 2 3 y 10 11 12 z 30 31 32
R Arrays
In R, Arrays are the data types that can store data in more than two dimensions. An array can be created using the array() function. It takes vectors as input and uses the values in the dim parameter to create an array. If you create an array of dimensions (2, 3, 4) then it creates 4 rectangular matrices each with 2 rows and 3 columns. Arrays can store only data type.
Give a Name to Columns and Rows:
We can give names to the rows, columns, and matrices in the array by setting the dimnames parameter.
v1 <- c(1,2,3)
v2 <- 100:110
col.names <- c("Col1","Col2","Col3","Col4","Col5","Col6","Col7")
row.names <- c("Row1","Row2")
matrix.names <- c("Matrix1","Matrix2")
arr4 <- array(c(v1,v2), dim=c(2,7,2), dimnames = list(row.names,col.names, matrix.names))
arr4
Output: , , Matrix1 Col1 Col2 Col3 Col4 Col5 Col6 Col7 Row1 1 3 101 103 105 107 109 Row2 2 100 102 104 106 108 110 , , Matrix2 Col1 Col2 Col3 Col4 Col5 Col6 Col7 Row1 1 3 101 103 105 107 109 Row2 2 100 102 104 106 108 110
Accessing/Extracting Array Elements:
# Print the 2nd row of the 1st matrix of the array.
print(arr4[2,,1])
# Print the element in the 2nd row and 4th column of the 2nd matrix.
print(arr4[2,4,2])
# Print the 2nd Matrix.
print(arr4[,,2])Output: > print(arr4[2,,1]) Col1 Col2 Col3 Col4 Col5 Col6 Col7 2 100 102 104 106 108 110 > > # Print the element in the 2nd row and 4th column of the 2nd matrix. > print(arr4[2,4,2]) [1] 104 > > # Print the 2nd Matrix. > print(arr4[,,2]) Col1 Col2 Col3 Col4 Col5 Col6 Col7 Row1 1 3 101 103 105 107 109 Row2 2 100 102 104 106 108 110
R Factors
Factors are used to represent categorical data and can be unordered or ordered. An example might be “Male” and “Female” if we consider gender. Factor objects can be created with the factor() function.
x <- factor(c("male", "female", "male", "male", "female"))
x
table(x)Output: x [1] male female male male female Levels: female male table(x) x female male 2 3
By default, Levels are put in alphabetical order. If you print the above code you will get levels as female and male. But if you want to get your levels in a particular order then set levels parameter like this.
x <- factor(c("male", "female", "male", "male", "female"), levels=c("male", "female"))
x
table(x)Output: x [1] male female male male female Levels: male female table(x) x male female 3 2
R Dataframes
Data frames are used to store tabular data in R. They are an important type of object in R and are used in a variety of statistical modeling applications. Data frames are represented as a special type of list where every element of the list has to have the same length. Each element of the list can be thought of as a column and the length of each element of the list is the number of rows. Unlike matrices, data frames can store different classes of objects in each column. Matrices must have every element be the same class (e.g. all integers or all numeric).
Creating a Data Frame:
Data frames can be created explicitly with the data.frame() function.
employee <- c('Ram','Sham','Jadu')
salary <- c(21000, 23400, 26800)
startdate <- as.Date(c('2016-11-1','2015-3-25','2017-3-14'))
employ_data <- data.frame(employee, salary, startdate)
employ_data
View(employ_data)Output: employ_data employee salary startdate 1 Ram 21000 2016-11-01 2 Sham 23400 2015-03-25 3 Jadu 26800 2017-03-14 View(employ_data)
Get the Structure of the Data Frame:
If you look at the structure of the data frame now, you see that the variable employee is a character vector, as shown in the following output:
str(employ_data)Output: > str(employ_data) 'data.frame': 3 obs. of 3 variables: $ employee : Factor w/ 3 levels "Jadu","Ram","Sham": 2 3 1 $ salary : num 21000 23400 26800 $ startdate: Date, format: "2016-11-01" "2015-03-25" "2017-03-14"
Note that the first column, employee, is of type factor, instead of a character vector. By default, data.frame() function converts character vector into factor. To suppress this behavior, we can pass the argument stringsAsFactors=FALSE.
employ_data <- data.frame(employee, salary, startdate, stringsAsFactors = FALSE)
str(employ_data)Output: 'data.frame': 3 obs. of 3 variables: $ employee : chr "Ram" "Sham" "Jadu" $ salary : num 21000 23400 26800 $ startdate: Date, format: "2016-11-01" "2015-03-25" "2017-03-14"
R Packages
The primary location for obtaining R packages is CRAN.
You can obtain information about the available packages on CRAN with the available.packages() function.
a <- available.packages()
head(rownames(a), 30) # Show the names of the first 30 packages
Packages can be installed with the install.packages() function in R. To install a single package, pass the name of the lecture to the install.packages() function as the first argument.
The following code installs the ggplot2 package from CRAN.
install.packages(“ggplot2”)
You can install multiple R packages at once with a single call to install.packages(). Place the names of the R packages in a character vector.
install.packages(c(“caret”, “ggplot2”, “dplyr”))
Loading packages
Installing a package does not make it immediately available to you in R; you must load the package. The library() function is used to load packages into R. The following code is used to load the ggplot2 package into R. Do not put the package name in quotes.
library(ggplot2)
If you have Installed your packages without root access using the command install.packages(“ggplot2″, lib=”/data/Rpackages/”). Then to load use the below command.
library(ggplot2, lib.loc=”/data/Rpackages/”)
After loading a package, the functions exported by that package will be attached to the top of the search() list (after the workspace).
library(ggplot2)
search()
R – CSV() files
In R, we can read data from files stored outside the R environment. We can also write data into files that will be stored and accessed by the operating system. R can read and write into various file formats like CSV, Excel, XML, etc.
Getting and Setting the Working Directory
We can check which directory the R workspace is pointing to using the getwd() function. You can also set a new working directory using setwd()function.
# Get and print current working directory.
print(getwd())
# Set current working directory.
setwd("/web/com")
# Get and print current working directory.
print(getwd())Output: [1] "/web/com/1441086124_2016" [1] "/web/com"
Input as CSV File
The CSV file is a text file in which the values in the columns are separated by a comma. Let’s consider the following data present in the file named input.csv.
You can create this file using windows notepad by copying and pasting this data. Save the file as input.csv using the save As All files(*.*) option in notepad.
Reading a CSV File
Following is a simple example of read.csv() function to read a CSV file available in your current working directory −
data <- read.csv("input.csv")
print(data) id, name, salary, start_date, deptR- Charts and Graphs
R- Pie Charts
Pie charts are created with the function pie(x, labels=) where x is a non-negative numeric vector indicating the area of each slice and labels= notes a character vector of names for the slices.
Syntax
The basic syntax for creating a pie-chart using the R is −
pie(x, labels, radius, main, col, clockwise)
Following is the description of the parameters used −
- x is a vector containing the numeric values used in the pie chart.
- labels are used to give a description of the slices.
- radius indicates the radius of the circle of the pie chart. (value between −1 and +1).
- main indicates the title of the chart.
- col indicates the color palette.
- clockwise is a logical value indicating if the slices are drawn clockwise or anti-clockwise.
Simple Pie chart
# Simple Pie Chart
slices <- c(10, 12,4, 16, 8)
lbls <- c("US", "UK", "Australia", "Germany", "France")
pie(slices, labels = lbls, main="Pie Chart of Countries")
3-D pie chart
The pie3D( ) function in the plotrix package provides 3D exploded pie charts.
# 3D Exploded Pie Chart
library(plotrix)
slices <- c(10, 12, 4, 16, 8)
lbls <- c("US", "UK", "Australia", "Germany", "France")
pie3D(slices,labels=lbls,explode=0.1,
main="Pie Chart of Countries ")R -Bar Charts
A bar chart represents data in rectangular bars with a length of the bar proportional to the value of the variable. R uses the function barplot() to create bar charts. R can draw both vertical and Horizontal bars in the bar chart. In the bar chart, each of the bars can be given different colors.
Let us suppose, we have a vector of maximum temperatures (in degree Celsius) for seven days as follows.
max.temp <- c(22, 27, 26, 24, 23, 26, 28)
barplot(max.temp)Some of the frequently used ones are, “main” to give the title, “xlab” and “ylab” to provide labels for the axes, names.arg for naming each bar, “col” to define color, etc.
We can also plot bars horizontally by providing the argument horiz=TRUE.
# barchart with added parameters
barplot(max.temp,
main = "Maximum Temperatures in a Week",
xlab = "Degree Celsius",
ylab = "Day",
names.arg = c("Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"),
col = "darkred",
horiz = TRUE)Simply doing barplot(age) will not give us the required plot. It will plot 10 bars with height equal to the student’s age. But we want to know the number of students in each age category.
This count can be quickly found using the table() function, as shown below.
> table(age)
age
16 17 18 19
1 2 6 1Now plotting this data will give our required bar plot. Note below, that we define the argument “density” to shade the bars.
barplot(table(age),
main="Age Count of 10 Students",
xlab="Age",
ylab="Count",
border="red",
col="blue",
density=10
)
A histogram represents the frequencies of values of a variable bucketed into ranges. Histogram is similar to bar chat but the difference is it groups the values into continuous ranges. Each bar in histogram represents the height of the number of values present in that range.
R creates histogram using hist() function. This function takes a vector as an input and uses some more parameters to plot histograms.
Syntax
The basic syntax for creating a histogram using R is −
hist(v,main,xlab,xlim,ylim,breaks,col,border)
Following is the description of the parameters used −
- v is a vector containing numeric values used in the histogram.
- main indicates the title of the chart.
- col is used to set the color of the bars.
- border is used to set the border color of each bar.
- xlab is used to give a description of the x-axis.
- xlim is used to specify the range of values on the x-axis.
- ylim is used to specify the range of values on the y-axis.
- breaks are used to mention the width of each bar.
Example
A simple histogram is created using input vector, label, col, and border parameters.
The script given below will create and save the histogram in the current R working directory.
# Create data for the graph.
v <- c(9,13,21,8,36,22,12,41,31,33,19)
# Give the chart file a name.
png(file = "histogram.png")
# Create the histogram.
hist(v,xlab = "Weight",col = "yellow",border = "blue")
# Save the file.
dev.off()
Range of X and Y values
To specify the range of values allowed in X axis and Y axis, we can use the xlim and ylim parameters.
The width of each bar can be decided by using breaks.
# Create data for the graph.
v <- c(9,13,21,8,36,22,12,41,31,33,19)
# Give the chart file a name.
png(file = "histogram_lim_breaks.png")
# Create the histogram.
hist(v,xlab = "Weight",col = "green",border = "red", xlim = c(0,40), ylim = c(0,5),
breaks = 5)
# Save the file.
dev.off()R vs SAS – Which Tool is Better?
The debate around data analytics tools has been going on forever. Each time a new one comes out, comparisons transpire. Although many aspects of the tool remain subjective, beginners want to know which tool is better to start with.
The most popular and widely used tools for data analytics are R and SAS. Both of them have been around for a long time and are often pitted against each other. So, let’s compare them based on the most relevant factors.
- Availability and Cost: SAS is widely used in most private organizations as it is a commercial software. It is more expensive than any other data analytics tool available. It might thus be a bit difficult buying the software if you are an individual professional or a student starting out. On the other hand, R is an open source software and is completely free to use. Anyone can begin using it right away without having to spend a penny. So, regarding availability and cost, R is hands down the better tool.
- Ease of learning: Since SAS is a commercial software, it has a whole lot of online resources available. Also, those who already know SQL might find it easier to adapt to SAS as it comes with PROC SQL option. The tool has a user-friendly GUI. It comes with an extensive documentation and tutorial base which can help early learners get started seamlessly. Whereas, the learning curve for R is quite steep. You need to learn to code at the root level and carrying out simple tasks demand a lot of time and effort with R. However, several forums and online communities post religiously about its usage.
- Data Handling Capabilities: When it comes to data handling, both SAS and R perform well, but there are some caveats for the latter. While SAS can even churn through terabytes of data with ease, R might be constrained as it makes use of the available RAM in the machine. This can be a hassle for 32-bit systems with low RAM capacity. Due to this, R can at times become unresponsive or give an ‘out of memory’ error. Both of them can run parallel computations, support integrations for Hadoop, Spark, Cloudera and Apache Pig among others. Also, the availability of devices with better RAM capacity might negate the disadvantages of R.
- Graphical Capabilities: Graphical capabilities or data visualization is the strongest forte of R. This is where SAS lacks behind in a major way. R has access to packages like GGPlot, RGIS, Lattice, and GGVIS among others which provide superior graphical competency. In comparison, Base SAS is struggling hard to catch up with the advancements in graphics and visualization in data analytics. Even the graphics packages available in SAS are poorly documented which makes them difficult to use.
- Advancements in Tool: Advancements in the industry give way to advancements in tools, and both SAS and R hold up pretty well in this regard. SAS, being a corporate software, rolls out new features and technologies frequently with new versions of its software. However, the updates are not as fast as R since it is open source software and has many contributors throughout the world. Alternatively, the latest updates in SAS are pushed out after thorough testing, making them much more stable, and reliable than R. Both the tools come with a fair share of pros & cons.
- Job Scenario: Currently, large corporations insist on using SAS, but SMEs and start-ups are increasingly opting for R, given that it’s free. The current job trend seems to show that while SAS is losing its momentum, R is gaining potential. The job scenario is on the cusp of change, and both the tools seem strong, but since R is on an uphill path, it can probably witness more jobs in the future, albeit not in huge corporates.
- Deep Learning Support: While SAS has just begun work on adding deep learning support, R has added support for a few packages which enable deep learning capabilities in the tool. You can use KerasR and keras package in R which are mere interfaces for the original Keras package built on Python. Although none of the tools are excellent facilitators of deep learning, R has seen some recent active developments on this front.
- Customer Service Support and Community: As one would expect from full-fledged commercial software, SAS offers excellent customer service support as well as the backing of a helpful community. Since R is free open-source software, expecting customer support will be hard to justify. However, it has a vast online community that can help you with almost everything. On the other hand, no matter what problem you face with SAS, you can immediately reach out to their customer support and get it solved without any hassles.
Final Verdict
As per estimations by the Economic Times, the analytics industry will grow to $16 billion till 2025 in India. If you wish to venture into this domain, there can’t be a better time. Just start learning the tool you think is better based on the comparison points above.
Original article source at: https://www.mygreatlearning.com
1667425440
Pdf2gerb: Perl Script Converts PDF Files to Gerber format
pdf2gerb
Perl script converts PDF files to Gerber format
Pdf2Gerb generates Gerber 274X photoplotting and Excellon drill files from PDFs of a PCB. Up to three PDFs are used: the top copper layer, the bottom copper layer (for 2-sided PCBs), and an optional silk screen layer. The PDFs can be created directly from any PDF drawing software, or a PDF print driver can be used to capture the Print output if the drawing software does not directly support output to PDF.
The general workflow is as follows:
- Design the PCB using your favorite CAD or drawing software.
- Print the top and bottom copper and top silk screen layers to a PDF file.
- Run Pdf2Gerb on the PDFs to create Gerber and Excellon files.
- Use a Gerber viewer to double-check the output against the original PCB design.
- Make adjustments as needed.
- Submit the files to a PCB manufacturer.
Please note that Pdf2Gerb does NOT perform DRC (Design Rule Checks), as these will vary according to individual PCB manufacturer conventions and capabilities. Also note that Pdf2Gerb is not perfect, so the output files must always be checked before submitting them. As of version 1.6, Pdf2Gerb supports most PCB elements, such as round and square pads, round holes, traces, SMD pads, ground planes, no-fill areas, and panelization. However, because it interprets the graphical output of a Print function, there are limitations in what it can recognize (or there may be bugs).
See docs/Pdf2Gerb.pdf for install/setup, config, usage, and other info.
pdf2gerb_cfg.pm
#Pdf2Gerb config settings:
#Put this file in same folder/directory as pdf2gerb.pl itself (global settings),
#or copy to another folder/directory with PDFs if you want PCB-specific settings.
#There is only one user of this file, so we don't need a custom package or namespace.
#NOTE: all constants defined in here will be added to main namespace.
#package pdf2gerb_cfg;
use strict; #trap undef vars (easier debug)
use warnings; #other useful info (easier debug)
##############################################################################################
#configurable settings:
#change values here instead of in main pfg2gerb.pl file
use constant WANT_COLORS => ($^O !~ m/Win/); #ANSI colors no worky on Windows? this must be set < first DebugPrint() call
#just a little warning; set realistic expectations:
#DebugPrint("${\(CYAN)}Pdf2Gerb.pl ${\(VERSION)}, $^O O/S\n${\(YELLOW)}${\(BOLD)}${\(ITALIC)}This is EXPERIMENTAL software. \nGerber files MAY CONTAIN ERRORS. Please CHECK them before fabrication!${\(RESET)}", 0); #if WANT_DEBUG
use constant METRIC => FALSE; #set to TRUE for metric units (only affect final numbers in output files, not internal arithmetic)
use constant APERTURE_LIMIT => 0; #34; #max #apertures to use; generate warnings if too many apertures are used (0 to not check)
use constant DRILL_FMT => '2.4'; #'2.3'; #'2.4' is the default for PCB fab; change to '2.3' for CNC
use constant WANT_DEBUG => 0; #10; #level of debug wanted; higher == more, lower == less, 0 == none
use constant GERBER_DEBUG => 0; #level of debug to include in Gerber file; DON'T USE FOR FABRICATION
use constant WANT_STREAMS => FALSE; #TRUE; #save decompressed streams to files (for debug)
use constant WANT_ALLINPUT => FALSE; #TRUE; #save entire input stream (for debug ONLY)
#DebugPrint(sprintf("${\(CYAN)}DEBUG: stdout %d, gerber %d, want streams? %d, all input? %d, O/S: $^O, Perl: $]${\(RESET)}\n", WANT_DEBUG, GERBER_DEBUG, WANT_STREAMS, WANT_ALLINPUT), 1);
#DebugPrint(sprintf("max int = %d, min int = %d\n", MAXINT, MININT), 1);
#define standard trace and pad sizes to reduce scaling or PDF rendering errors:
#This avoids weird aperture settings and replaces them with more standardized values.
#(I'm not sure how photoplotters handle strange sizes).
#Fewer choices here gives more accurate mapping in the final Gerber files.
#units are in inches
use constant TOOL_SIZES => #add more as desired
(
#round or square pads (> 0) and drills (< 0):
.010, -.001, #tiny pads for SMD; dummy drill size (too small for practical use, but needed so StandardTool will use this entry)
.031, -.014, #used for vias
.041, -.020, #smallest non-filled plated hole
.051, -.025,
.056, -.029, #useful for IC pins
.070, -.033,
.075, -.040, #heavier leads
# .090, -.043, #NOTE: 600 dpi is not high enough resolution to reliably distinguish between .043" and .046", so choose 1 of the 2 here
.100, -.046,
.115, -.052,
.130, -.061,
.140, -.067,
.150, -.079,
.175, -.088,
.190, -.093,
.200, -.100,
.220, -.110,
.160, -.125, #useful for mounting holes
#some additional pad sizes without holes (repeat a previous hole size if you just want the pad size):
.090, -.040, #want a .090 pad option, but use dummy hole size
.065, -.040, #.065 x .065 rect pad
.035, -.040, #.035 x .065 rect pad
#traces:
.001, #too thin for real traces; use only for board outlines
.006, #minimum real trace width; mainly used for text
.008, #mainly used for mid-sized text, not traces
.010, #minimum recommended trace width for low-current signals
.012,
.015, #moderate low-voltage current
.020, #heavier trace for power, ground (even if a lighter one is adequate)
.025,
.030, #heavy-current traces; be careful with these ones!
.040,
.050,
.060,
.080,
.100,
.120,
);
#Areas larger than the values below will be filled with parallel lines:
#This cuts down on the number of aperture sizes used.
#Set to 0 to always use an aperture or drill, regardless of size.
use constant { MAX_APERTURE => max((TOOL_SIZES)) + .004, MAX_DRILL => -min((TOOL_SIZES)) + .004 }; #max aperture and drill sizes (plus a little tolerance)
#DebugPrint(sprintf("using %d standard tool sizes: %s, max aper %.3f, max drill %.3f\n", scalar((TOOL_SIZES)), join(", ", (TOOL_SIZES)), MAX_APERTURE, MAX_DRILL), 1);
#NOTE: Compare the PDF to the original CAD file to check the accuracy of the PDF rendering and parsing!
#for example, the CAD software I used generated the following circles for holes:
#CAD hole size: parsed PDF diameter: error:
# .014 .016 +.002
# .020 .02267 +.00267
# .025 .026 +.001
# .029 .03167 +.00267
# .033 .036 +.003
# .040 .04267 +.00267
#This was usually ~ .002" - .003" too big compared to the hole as displayed in the CAD software.
#To compensate for PDF rendering errors (either during CAD Print function or PDF parsing logic), adjust the values below as needed.
#units are pixels; for example, a value of 2.4 at 600 dpi = .0004 inch, 2 at 600 dpi = .0033"
use constant
{
HOLE_ADJUST => -0.004 * 600, #-2.6, #holes seemed to be slightly oversized (by .002" - .004"), so shrink them a little
RNDPAD_ADJUST => -0.003 * 600, #-2, #-2.4, #round pads seemed to be slightly oversized, so shrink them a little
SQRPAD_ADJUST => +0.001 * 600, #+.5, #square pads are sometimes too small by .00067, so bump them up a little
RECTPAD_ADJUST => 0, #(pixels) rectangular pads seem to be okay? (not tested much)
TRACE_ADJUST => 0, #(pixels) traces seemed to be okay?
REDUCE_TOLERANCE => .001, #(inches) allow this much variation when reducing circles and rects
};
#Also, my CAD's Print function or the PDF print driver I used was a little off for circles, so define some additional adjustment values here:
#Values are added to X/Y coordinates; units are pixels; for example, a value of 1 at 600 dpi would be ~= .002 inch
use constant
{
CIRCLE_ADJUST_MINX => 0,
CIRCLE_ADJUST_MINY => -0.001 * 600, #-1, #circles were a little too high, so nudge them a little lower
CIRCLE_ADJUST_MAXX => +0.001 * 600, #+1, #circles were a little too far to the left, so nudge them a little to the right
CIRCLE_ADJUST_MAXY => 0,
SUBST_CIRCLE_CLIPRECT => FALSE, #generate circle and substitute for clip rects (to compensate for the way some CAD software draws circles)
WANT_CLIPRECT => TRUE, #FALSE, #AI doesn't need clip rect at all? should be on normally?
RECT_COMPLETION => FALSE, #TRUE, #fill in 4th side of rect when 3 sides found
};
#allow .012 clearance around pads for solder mask:
#This value effectively adjusts pad sizes in the TOOL_SIZES list above (only for solder mask layers).
use constant SOLDER_MARGIN => +.012; #units are inches
#line join/cap styles:
use constant
{
CAP_NONE => 0, #butt (none); line is exact length
CAP_ROUND => 1, #round cap/join; line overhangs by a semi-circle at either end
CAP_SQUARE => 2, #square cap/join; line overhangs by a half square on either end
CAP_OVERRIDE => FALSE, #cap style overrides drawing logic
};
#number of elements in each shape type:
use constant
{
RECT_SHAPELEN => 6, #x0, y0, x1, y1, count, "rect" (start, end corners)
LINE_SHAPELEN => 6, #x0, y0, x1, y1, count, "line" (line seg)
CURVE_SHAPELEN => 10, #xstart, ystart, x0, y0, x1, y1, xend, yend, count, "curve" (bezier 2 points)
CIRCLE_SHAPELEN => 5, #x, y, 5, count, "circle" (center + radius)
};
#const my %SHAPELEN =
#Readonly my %SHAPELEN =>
our %SHAPELEN =
(
rect => RECT_SHAPELEN,
line => LINE_SHAPELEN,
curve => CURVE_SHAPELEN,
circle => CIRCLE_SHAPELEN,
);
#panelization:
#This will repeat the entire body the number of times indicated along the X or Y axes (files grow accordingly).
#Display elements that overhang PCB boundary can be squashed or left as-is (typically text or other silk screen markings).
#Set "overhangs" TRUE to allow overhangs, FALSE to truncate them.
#xpad and ypad allow margins to be added around outer edge of panelized PCB.
use constant PANELIZE => {'x' => 1, 'y' => 1, 'xpad' => 0, 'ypad' => 0, 'overhangs' => TRUE}; #number of times to repeat in X and Y directions
# Set this to 1 if you need TurboCAD support.
#$turboCAD = FALSE; #is this still needed as an option?
#CIRCAD pad generation uses an appropriate aperture, then moves it (stroke) "a little" - we use this to find pads and distinguish them from PCB holes.
use constant PAD_STROKE => 0.3; #0.0005 * 600; #units are pixels
#convert very short traces to pads or holes:
use constant TRACE_MINLEN => .001; #units are inches
#use constant ALWAYS_XY => TRUE; #FALSE; #force XY even if X or Y doesn't change; NOTE: needs to be TRUE for all pads to show in FlatCAM and ViewPlot
use constant REMOVE_POLARITY => FALSE; #TRUE; #set to remove subtractive (negative) polarity; NOTE: must be FALSE for ground planes
#PDF uses "points", each point = 1/72 inch
#combined with a PDF scale factor of .12, this gives 600 dpi resolution (1/72 * .12 = 600 dpi)
use constant INCHES_PER_POINT => 1/72; #0.0138888889; #multiply point-size by this to get inches
# The precision used when computing a bezier curve. Higher numbers are more precise but slower (and generate larger files).
#$bezierPrecision = 100;
use constant BEZIER_PRECISION => 36; #100; #use const; reduced for faster rendering (mainly used for silk screen and thermal pads)
# Ground planes and silk screen or larger copper rectangles or circles are filled line-by-line using this resolution.
use constant FILL_WIDTH => .01; #fill at most 0.01 inch at a time
# The max number of characters to read into memory
use constant MAX_BYTES => 10 * M; #bumped up to 10 MB, use const
use constant DUP_DRILL1 => TRUE; #FALSE; #kludge: ViewPlot doesn't load drill files that are too small so duplicate first tool
my $runtime = time(); #Time::HiRes::gettimeofday(); #measure my execution time
print STDERR "Loaded config settings from '${\(__FILE__)}'.\n";
1; #last value must be truthful to indicate successful load
#############################################################################################
#junk/experiment:
#use Package::Constants;
#use Exporter qw(import); #https://perldoc.perl.org/Exporter.html
#my $caller = "pdf2gerb::";
#sub cfg
#{
# my $proto = shift;
# my $class = ref($proto) || $proto;
# my $settings =
# {
# $WANT_DEBUG => 990, #10; #level of debug wanted; higher == more, lower == less, 0 == none
# };
# bless($settings, $class);
# return $settings;
#}
#use constant HELLO => "hi there2"; #"main::HELLO" => "hi there";
#use constant GOODBYE => 14; #"main::GOODBYE" => 12;
#print STDERR "read cfg file\n";
#our @EXPORT_OK = Package::Constants->list(__PACKAGE__); #https://www.perlmonks.org/?node_id=1072691; NOTE: "_OK" skips short/common names
#print STDERR scalar(@EXPORT_OK) . " consts exported:\n";
#foreach(@EXPORT_OK) { print STDERR "$_\n"; }
#my $val = main::thing("xyz");
#print STDERR "caller gave me $val\n";
#foreach my $arg (@ARGV) { print STDERR "arg $arg\n"; }Download Details:
Author: swannman
Source Code: https://github.com/swannman/pdf2gerb
License: GPL-3.0 license
1685205672
Create Pagination Using jQuery
In this article, we will see how to create pagination using jquery. We will create jquery pagination using multiple ways. You can create pagination using different ways like creating pagination using simple HTML, you can create pagination in laravel using paginate() method. Also, create pagination laravel livewire, pagination using bootstrap.
We will create simple jquery pagination. Also, create pagination using jquery without a plugin and create jquery pagination with next and previous buttons
So, let's see dynamic pagination in jquery and bootstrap pagination in jquery
Example:
In this example, we will create pagination using jquery without using a plugin. Also, you can customize the pagination.
<!DOCTYPE html>
<html lang="en">
<head>
<title>How To Create Pagination Using jQuery - Websolutionstuff</title>
<style>
.current {
color: green;
}
#pagin li {
display: inline-block;
font-weight: 500;
}
.prev {
cursor: pointer;
}
.next {
cursor: pointer;
}
.last {
cursor:pointer;
margin-left:10px;
}
.first {
cursor:pointer;
margin-right:10px;
}
.line-content, #pagin, h3 {
text-align:center;
}
.line-content {
margin-top:20px;
}
#pagin {
margin-top:10px;
padding-left:0;
}
h3 {
margin:50px 0;
}
</style>
</head>
<body>
<h3>How To Create Pagination Using jQuery - Websolutionstuff</h3>
<div class="line-content">This is Page 1 content example with next and prev.</div>
<div class="line-content">This is Page 2 content example with next and prev.</div>
<div class="line-content">This is Page 3 content example with next and prev.</div>
<div class="line-content">This is Page 4 content example with next and prev.</div>
<div class="line-content">This is Page 5 content example with next and prev.</div>
<div class="line-content">This is Page 6 content example with next and prev.</div>
<div class="line-content">This is Page 7 content example with next and prev.</div>
<div class="line-content">This is Page 8 content example with next and prev.</div>
<div class="line-content">This is Page 9 content example with next and prev.</div>
<div class="line-content">This is Page 10 content example with next and prev.</div>
<div class="line-content">This is Page 11 content example with next and prev.</div>
<div class="line-content">This is Page 12 content example with next and prev.</div>
<div class="line-content">This is Page 13 content example with next and prev.</div>
<div class="line-content">This is Page 14 content example with next and prev.</div>
<div class="line-content">This is Page 15 content example with next and prev.</div>
<div class="line-content">This is Page 16 content example with next and prev.</div>
<div class="line-content">This is Page 17 content example with next and prev.</div>
<div class="line-content">This is Page 18 content example with next and prev.</div>
<div class="line-content">This is Page 19 content example with next and prev.</div>
<div class="line-content">This is Page 20 content example with next and prev.</div>
<div class="line-content">This is Page 21 content example with next and prev.</div>
<div class="line-content">This is Page 22 content example with next and prev.</div>
<div class="line-content">This is Page 23 content example with next and prev.</div>
<div class="line-content">This is Page 24 content example with next and prev.</div>
<div class="line-content">This is Page 25 content example with next and prev.</div>
<div class="line-content">This is Page 26 content example with next and prev.</div>
<div class="line-content">This is Page 27 content example with next and prev.</div>
<div class="line-content">This is Page 28 content example with next and prev.</div>
<div class="line-content">This is Page 29 content example with next and prev.</div>
<div class="line-content">This is Page 30 content example with next and prev.</div>
<div class="line-content">This is Page 31 content example with next and prev.</div>
<div class="line-content">This is Page 32 content example with next and prev.</div>
<div class="line-content">This is Page 33 content example with next and prev.</div>
<div class="line-content">This is Page 34 content example with next and prev.</div>
<div class="line-content">This is Page 35 content example with next and prev.</div>
<div class="line-content">This is Page 36 content example with next and prev.</div>
<div class="line-content">This is Page 37 content example with next and prev.</div>
<div class="line-content">This is Page 38 content example with next and prev.</div>
<div class="line-content">This is Page 39 content example with next and prev.</div>
<div class="line-content">This is Page 40 content example with next and prev.</div>
<div class="line-content">This is Page 41 content example with next and prev.</div>
<div class="line-content">This is Page 42 content example with next and prev.</div>
<div class="line-content">This is Page 43 content example with next and prev.</div>
<div class="line-content">This is Page 44 content example with next and prev.</div>
<div class="line-content">This is Page 45 content example with next and prev.</div>
<ul id="pagin"></ul>
</body>
</html>
<script src="https://code.jquery.com/jquery-3.6.1.min.js" integrity="sha256-o88AwQnZB+VDvE9tvIXrMQaPlFFSUTR+nldQm1LuPXQ=" crossorigin="anonymous"></script>
<script>
pageSize = 5;
incremSlide = 5;
startPage = 0;
numberPage = 0;
var pageCount = $(".line-content").length / pageSize;
var totalSlidepPage = Math.floor(pageCount / incremSlide);
for(var i = 0 ; i<pageCount;i++){
$("#pagin").append('<li><a href="#">'+(i+1)+'</a></li> ');
if(i>pageSize){
$("#pagin li").eq(i).hide();
}
}
var prev = $("<li/>").addClass("prev").html("Prev").click(function(){
startPage-=5;
incremSlide-=5;
numberPage--;
slide();
});
prev.hide();
var next = $("<li/>").addClass("next").html("Next").click(function(){
startPage+=5;
incremSlide+=5;
numberPage++;
slide();
});
$("#pagin").prepend(prev).append(next);
$("#pagin li").first().find("a").addClass("current");
slide = function(sens){
$("#pagin li").hide();
for(t=startPage;t<incremSlide;t++){
$("#pagin li").eq(t+1).show();
}
if(startPage == 0){
next.show();
prev.hide();
}else if(numberPage == totalSlidepPage ){
next.hide();
prev.show();
}else{
next.show();
prev.show();
}
}
showPage = function(page) {
$(".line-content").hide();
$(".line-content").each(function(n) {
if (n >= pageSize * (page - 1) && n < pageSize * page){
$(this).show();
}
});
}
showPage(1);
$("#pagin li a").eq(0).addClass("current");
$("#pagin li a").click(function() {
$("#pagin li a").removeClass("current");
$(this).addClass("current");
showPage(parseInt($(this).text()));
});
</script>Output:
Example:
In this example, we will create bootstrap pagination with help of jquery.
<!DOCTYPE html>
<html lang="en">
<head>
<title>How To Create Bootstrap Pagination Using jQuery - Websolutionstuff</title>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css">
<style>
#data tr {
display: none;
}
.page {
margin: 30px;
}
table, th, td {
border: 1px solid black;
}
#data {
font-family: Arial, Helvetica, sans-serif;
border-collapse: collapse;
width: 100%;
}
#data td, #data th {
border: 1px solid #ddd;
padding: 8px;
}
#data tr:nth-child(even) {
background-color: #f2f2f2;
}
#data tr:hover {
background-color: #ddd;
}
#data th {
padding-top: 12px;
padding-bottom: 12px;
text-align: left;
background-color: #03aa96;
color: white;
}
#nav a {
color: #03aa96;
font-size: 20px;
margin-top: 22px;
font-weight: 600;
}
a:hover, a:visited, a:link, a:active {
text-decoration: none;
}
#nav {
margin-top: 20px;
}
</style>
</head>
<body>
<h2 align="center" class="mt-4">How To Create Bootstrap Pagination Using jQuery - Websolutionstuff</h2>
<div class="page" align="center">
<table id="data">
<tr>
<th>Id</th>
<th>Name</th>
<th>Country</th>
</tr>
<tr>
<td>1</td>
<td>Maria</td>
<td>Germany</td>
</tr>
<tr>
<td>2</td>
<td>Christina</td>
<td>Sweden</td>
</tr>
<tr>
<td>3</td>
<td>Chang</td>
<td>Mexico</td>
</tr>
<tr>
<td>4</td>
<td>Mendel</td>
<td>Austria</td>
</tr>
<tr>
<td>5</td>
<td>Helen</td>
<td>United Kingdom</td>
</tr>
<tr>
<td>6</td>
<td>Philip</td>
<td>Germany</td>
</tr>
<tr>
<td>7</td>
<td>Tannamuri</td>
<td>Canada</td>
</tr>
<tr>
<td>8</td>
<td>Rovelli</td>
<td>Italy</td>
</tr>
<tr>
<td>9</td>
<td>Dell</td>
<td>United Kingdom</td>
</tr>
<tr>
<td>10</td>
<td>Trump</td>
<td>France</td>
</tr>
</table>
</div>
</body>
</html>
<script src="https://code.jquery.com/jquery-3.6.1.min.js" integrity="sha256-o88AwQnZB+VDvE9tvIXrMQaPlFFSUTR+nldQm1LuPXQ=" crossorigin="anonymous"></script>
<script>
$(document).ready (function () {
$('#data').after ('<div id="nav"></div>');
var rowsShown = 5;
var rowsTotal = $('#data tbody tr').length;
var numPages = rowsTotal/rowsShown;
for (i = 0;i < numPages;i++) {
var pageNum = i + 1;
$('#nav').append ('<a href="#" rel="'+i+'">'+pageNum+'</a> ');
}
$('#data tbody tr').hide();
$('#data tbody tr').slice (0, rowsShown).show();
$('#nav a:first').addClass('active');
$('#nav a').bind('click', function() {
$('#nav a').removeClass('active');
$(this).addClass('active');
var currPage = $(this).attr('rel');
var startItem = currPage * rowsShown;
var endItem = startItem + rowsShown;
$('#data tbody tr').css('opacity','0.0').hide().slice(startItem, endItem).
css('display','table-row').animate({opacity:1}, 300);
});
});
</script>Output:
Example:
In this example, we will create pagination using the twbsPagination plugin. This jQuery plugin simplifies the usage of Bootstrap Pagination.
<!DOCTYPE html>
<html lang="en">
<head>
<title>jQuery Pagination Using Plugin - Websolutionstuff</title>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/4.0.0-alpha.6/css/bootstrap.min.css">
<style>
.wrapper{
margin: 60px auto;
text-align: center;
}
h2{
margin-bottom: 1.25em;
}
#pagination-demo{
display: inline-block;
margin-bottom: 1.75em;
}
#pagination-demo li{
display: inline-block;
}
.page-content{
background: #eee;
display: inline-block;
padding: 10px;
width: 100%;
max-width: 660px;
}
</style>
</head>
<body>
<div class="wrapper">
<div class="container">
<div class="row">
<div class="col-sm-12">
<h2>jQuery Pagination Using Plugin - Websolutionstuff</h2>
<p>Simple pagination using the TWBS pagination JS library.</p>
<ul id="pagination-demo" class="pagination-sm"></ul>
</div>
</div>
<div id="page-content" class="page-content">Page 1</div>
</div>
</div>
</body>
</html>
<script src="https://code.jquery.com/jquery-3.6.1.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/4.0.0-alpha.6/js/bootstrap.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/twbs-pagination/1.4.1/jquery.twbsPagination.min.js"></script>
<script>
$(document).ready (function () {
$('#pagination-demo').twbsPagination({
totalPages: 16,
visiblePages: 6,
next: 'Next',
prev: 'Prev',
onPageClick: function (event, page) {
$('#page-content').text('Page ' + page) + ' content here';
}
});
});
</script>Output:
Original article source at: https://websolutionstuff.com/
1620466520
Your Data Architecture: Simple Best Practices for Your Data Strategy
If you accumulate data on which you base your decision-making as an organization, you should probably think about your data architecture and possible best practices.
If you accumulate data on which you base your decision-making as an organization, you most probably need to think about your data architecture and consider possible best practices. Gaining a competitive edge, remaining customer-centric to the greatest extent possible, and streamlining processes to get on-the-button outcomes can all be traced back to an organization’s capacity to build a future-ready data architecture.
In what follows, we offer a short overview of the overarching capabilities of data architecture. These include user-centricity, elasticity, robustness, and the capacity to ensure the seamless flow of data at all times. Added to these are automation enablement, plus security and data governance considerations. These points from our checklist for what we perceive to be an anticipatory analytics ecosystem.
#big data #data science #big data analytics #data analysis #data architecture #data transformation #data platform #data strategy #cloud data platform #data acquisition
1593156510
Basic Data Types in Python | Python Web Development For Beginners
At the end of 2019, Python is one of the fastest-growing programming languages. More than 10% of developers have opted for Python development.
In the programming world, Data types play an important role. Each Variable is stored in different data types and responsible for various functions. Python had two different objects, and They are mutable and immutable objects.
Table of Contents hide
III Built-in data types in Python
Mutable objects
The Size and declared value and its sequence of the object can able to be modified called mutable objects.
Mutable Data Types are list, dict, set, byte array
Immutable objects
The Size and declared value and its sequence of the object can able to be modified.
Immutable data types are int, float, complex, String, tuples, bytes, and frozen sets.
id() and type() is used to know the Identity and data type of the object
a**=25+**85j
type**(a)**
output**:<class’complex’>**
b**={1:10,2:“Pinky”****}**
id**(b)**
output**:**238989244168
Built-in data types in Python
a**=str(“Hello python world”)****#str**
b**=int(18)****#int**
c**=float(20482.5)****#float**
d**=complex(5+85j)****#complex**
e**=list((“python”,“fast”,“growing”,“in”,2018))****#list**
f**=tuple((“python”,“easy”,“learning”))****#tuple**
g**=range(10)****#range**
h**=dict(name=“Vidu”,age=36)****#dict**
i**=set((“python”,“fast”,“growing”,“in”,2018))****#set**
j**=frozenset((“python”,“fast”,“growing”,“in”,2018))****#frozenset**
k**=bool(18)****#bool**
l**=bytes(8)****#bytes**
m**=bytearray(8)****#bytearray**
n**=memoryview(bytes(18))****#memoryview**
Numbers (int,Float,Complex)
Numbers are stored in numeric Types. when a number is assigned to a variable, Python creates Number objects.
#signed interger
age**=**18
print**(age)**
Output**:**18
Python supports 3 types of numeric data.
int (signed integers like 20, 2, 225, etc.)
float (float is used to store floating-point numbers like 9.8, 3.1444, 89.52, etc.)
complex (complex numbers like 8.94j, 4.0 + 7.3j, etc.)
A complex number contains an ordered pair, i.e., a + ib where a and b denote the real and imaginary parts respectively).
String
The string can be represented as the sequence of characters in the quotation marks. In python, to define strings we can use single, double, or triple quotes.
# String Handling
‘Hello Python’
#single (') Quoted String
“Hello Python”
# Double (") Quoted String
“”“Hello Python”“”
‘’‘Hello Python’‘’
# triple (‘’') (“”") Quoted String
In python, string handling is a straightforward task, and python provides various built-in functions and operators for representing strings.
The operator “+” is used to concatenate strings and “*” is used to repeat the string.
“Hello”+“python”
output**:****‘Hello python’**
"python "*****2
'Output : Python python ’
#python web development #data types in python #list of all python data types #python data types #python datatypes #python types #python variable type