# NumPy Tutorial for Beginners

I will walk you through the basics of NumPy. If you want to do machine learning then knowledge of NumPy is necessary. It one of the most widely used Python libraries. It is the most useful library if you are dealing with numbers in Python. NumPy guarantees great execution speed compared to standard Python libraries. It comes with a great number of built-in functions.

Advantages of using NumPy with Python:

• Array-oriented computing.
• Efficiently implemented multi-dimensional arrays.
• Designed for scientific computation.

First, let’s talk about its installation. NumPy is not part of the basic Python installation. We need to install it after the installation of Python in our system. We can do it by the pip using command, `pip install NumPy`, or by installing Conda.

We are done with the installation and now we can jump right into NumPy. First, let’s start with the most important object in NumPy, the ndarray or multi-dimensional array. A multi-dimensional array is an array of arrays. In multi-dimensional arrays, this array, `[1,2,3]`, is a one-dimensional array because it contains only one row. The below is array is a two-dimensional array, as it contains multiple rows as well as multiple columns.

<pre class="ql-syntax" spellcheck="false">[[1 2 3]

[4 5 6]

[7 8 9]]
</pre>

Let’s do some coding now. Here I am using Jupyter Notebook to run my code; you can use any IDE available and best suited to you.

We start with `import NumPy`.

In the following code, I am renaming the package to `np` for convenience sake.

<pre class="ql-syntax" spellcheck="false">import numpy as np
</pre>

Now, in order to create an array in NumPy, we use its array function as shown below:

<pre class="ql-syntax" spellcheck="false">array = np.array([1,2,3])

print(array)

Output: [1 2 3]
</pre>

This an example of a one-dimensional array.

Another way to create an array in NumPy is by using the `zeros` function.

<pre class="ql-syntax" spellcheck="false">zeros = np.zeros(3)

print(zeros)

Output: [0. 0. 0.]
</pre>

If you look closely at the output, the generated array contains three zeros, but the type of the value is a float and, by default, NumPy creates the array of float values.

<pre class="ql-syntax" spellcheck="false">type(zeros[0])

Output: numpy.float64
</pre>

Going back to the first example inside NumPy’s `array` function, we pass a list so we can also pass the `list` variable inside the `array` function and the output will be the same.

<pre class="ql-syntax" spellcheck="false">my_list = [1,2,3]

array = np.array(my_list)

print(array)

Output: [1 2 3]
</pre>

Now, let’s look into how to create a two-dimensional array using NumPy. Instead of passing the list now we have to pass a list of tuples or list of lists as mentioned below.

<pre class="ql-syntax" spellcheck="false">two_dim_array = np.array([(1,2,3), (4,5,6), (7,8,9)])

print(two_dim_array)

Output:

[[1 2 3]

[4 5 6]

[7 8 9]]
</pre>

Note that the number of columns should be equal, otherwise NumPy will create an array of a list.

<pre class="ql-syntax" spellcheck="false">arr = np.array([[1,2,3], [4,6], [7,8,9]])

print(arr)

Output: [list([1, 2, 3]) list([4, 6]) list([7, 8, 9])]
</pre>

Now, to create an array of a range, which is very good for making plots, we use the `linspace` function.

<pre class="ql-syntax" spellcheck="false">range_array = np.linspace(0, 10, 4)

print(range_array)

Output: [ 0. 3.33333333 6.66666667 10. ]
</pre>

Here, the first argument is the starting point and next is the endpoint and the last argument defines how many elements you want in your array.

Now, to create random arrays we can use the `random` function. Here, I’ve created an array of random integers, and, therefore, used `randint` where first I specified the maximum value and then the size of my array.

<pre class="ql-syntax" spellcheck="false">random_array = np.random.randint(15, size=10)

print(random_array)

Output: [ 7 11 8 2 6 4 9 6 10 9]
</pre>

Now we know the basics of how to create arrays in NumPy. Now let’s look into some of its basic operations. First, we will start by finding the size and shape of an array. Size will give the number of elements in an array whereas shape will give us the shape of an array.

For a one dimensional array, the shape would be `(n, )`, where `n` is the number of elements in your array.

For a two dimensional array, the shape would be `(n,m)`, where `n` is the number of rows and `m` is the number of columns in your array

<pre class="ql-syntax" spellcheck="false">print(array.size)

Output: 3

print(array.shape)

Output: (3,)

print(multi_dim_array.size)

Output: 9

print(multi_dim_array.shape)

Output: (3, 3)
</pre>

If we want to change the shape of an array we can easily do it with the `reshape` function. It will look like something like this:

<pre class="ql-syntax" spellcheck="false">two_dim_array = np.array([(1,2,3,4), (5,6,7,8)])

two_dim_array = two_dim_array.reshape(4,2)

print(two_dim_array)

Output:

[[1 2]

[3 4]

[5 6]

[7 8]]
</pre>

We need to make sure that the rows and columns can be interchangeable. For example, here, we can change rows and columns from (2,4) to (4,2) but can not change them to (4,3) because, for that, we’d need 12 elements and we have only 8. Doing so will give an error as shown below.

<pre class="ql-syntax" spellcheck="false">ValueError: cannot reshape array of size 8 into shape (4,3)
</pre>

To check the dimensions of our array. we can use the `ndim` function.

<pre class="ql-syntax" spellcheck="false">print(two_dim_array.ndim)

Output: 2
</pre>

Now, to get values from an array, a process known as slicing can be done in various ways. For example, `array[1]` will fetch the second element of my array, but if we want a range we can use `array[0:1]`, which will give us the first two elements. For the last value of the array, we can use `array[-1]`, which is similar to the standard method of getting elements from a list in Python.

Now to find the sum all we have to use is the `sum()`, function but if we want to find the sum of the axis we can pass an argument for the axis.

<pre class="ql-syntax" spellcheck="false">print(two_dim_array.sum(axis=0))

Output: [ 6 8 10 12]

print(two_dim_array.sum(axis=1))

Output: [10 26]
</pre>

Now to add two arrays all we have to use if + operator. For example:

<pre class="ql-syntax" spellcheck="false">print(two_dim_array + two_dim_array)

Output:

[[ 2 4 6 8]

[10 12 14 16]]
</pre>

Similarly, we can use other operands as well, like multiple, subtract, and divide.

We have many other operations present in NumPy like `sqrt`, which will give us the square root of every element, and `std`, which is used to find the standard deviation. To explore more about these operations visit the NumPy’s documentation.

And that’s it for the introduction of NumPy.

# How to Using Python Pandas for Log Analysis

In this quick article, I'll walk you through how to use the Pandas library for Python in order to analyze a CSV log file for offload analysis.

## Background

Python Pandas is a library that provides data science capabilities to Python. Using this library, you can use data structures like DataFrames. This data structure allows you to model the data like an in-memory database. By doing so, you will get query-like capabilities over the data set.

## Use Case

Suppose we have a URL report from taken from either the Akamai Edge server logs or the Akamai Portal report. In this case, I am using the Akamai Portal report. In this workflow, I am trying to find the top URLs that have a volume offload less than 50%. I've attached the code at the end. I am going to walk through the code line-by-line. Here are the column names within the CSV file for reference.

``````Offloaded Hits,Origin Hits,Origin OK Volume (MB),Origin Error Volume (MB)

``````

## Initialize the Library

The first step is to initialize the Pandas library. In almost all the references, this library is imported as pd. We'll follow the same convention.

``````import pandas as pd

``````

## Read the CSV as a DataFrame

The next step is to read the whole CSV file into a DataFrame. Note that this function to read CSV data also has options to ignore leading rows, trailing rows, handling missing values, and a lot more. I am not using these options for now.

``````urls_df = pd.read_csv('urls_report.csv')

``````

Pandas automatically detects the right data formats for the columns. So the URL is treated as a string and all the other values are considered floating point values.

The default URL report does not have a column for Offload by Volume. So we need to compute this new column.

``````urls_df['Volume Offload'] = (urls_df['OK Volume']*100) / (urls_df[

``````

We are using the columns named OK Volume and Origin OK Volumn (MB) to arrive at the percent offloads.

## Filter the Data

At this point, we need to have the entire data set with the offload percentage computed. Since we are interested in URLs that have a low offload, we add two filters:

• Consider the rows having a volume offload of less than 50% and it should have at least some traffic (we don't want rows that have zero traffic).

• We will also remove some known patterns. This is based on the customer context but essentially indicates URLs that can never be cached.

## Sort Data

At this point, we have the right set of URLs but they are unsorted. We need the rows to be sorted by URLs that have the most volume and least offload. We can achieve this sorting by columns using the sort command.

``````low_offload_urls.sort_values(by=['OK Volume','Volume Offload'],inplace

``````

## Print the Data

For simplicity, I am just listing the URLs. We can export the result to CSV or Excel as well.

First, we project the URL (i.e., extract just one column) from the dataframe. We then list the URLs with a simple for loop as the projection results in an array.

``````for each_url in low_offload_urls['URL']:
print (each_url)
``````

I hope you found this useful and get inspired to pick up Pandas for your analytics as well!

## References

I was able to pick up Pandas after going through an excellent course on Coursera titled Introduction to Data Science in Python. During this course, I realized that Pandas has excellent documentation.

• Pandas Documentation: http://pandas.pydata.org/pandas-docs/stable/

Full Code

``````import pandas as pd

#now convert to right types
urls_df['Volume Offload'] = (urls_df['OK Volume']*100) / (urls_df['OK Volume'] + urls_df['Origin OK Volume (MB)'])

print (each_url)
``````

# 30 Helpful Python Snippets You Should Learn Today

Python is a no-BS programming language. Readability and simplicity of design are two of the biggest reasons for its immense popularity.

Part of the reason for this popularity is its simplicity and easiness to learn it.

If you are reading this, then it is highly likely that you already use Python or at least have an interest in it.

In this article, we will briefly see 30 short code snippets that you can understand and learn in 30 seconds or less.

The following tricks will prove handy in your day-to-day coding exercises.

## 1. All unique

The following method checks whether the given list has duplicate elements. It uses the property of set() which removes duplicate elements from the list.

``````def all_unique(lst):
return len(lst) == len(set(lst))

x = [1,1,2,2,3,2,3,4,5,6]
y = [1,2,3,4,5]
all_unique(x) # False
all_unique(y) # True
``````

## 2. Anagrams

This method can be used to check if two strings are anagrams. An anagram is a word or phrase formed by rearranging the letters of a different word or phrase, typically using all the original letters exactly once.

``````from collections import Counter

def anagram(first, second):
return Counter(first) == Counter(second)

anagram("abcd3", "3acdb") # True
``````

## 3. Memory

This snippet can be used to check the memory usage of an object.

``````import sys

variable = 30
print(sys.getsizeof(variable)) # 24
``````

## 4. Byte size

This method returns the length of a string in bytes.

``````def byte_size(string):
return(len(string.encode('utf-8')))

byte_size('😀') # 4
byte_size('Hello World') # 11
``````

## 5. Print a string N times

This snippet can be used to print a string n times without having to use loops to do it.

``````n = 2;
s ="Programming";

print(s * n); # ProgrammingProgramming
``````

## 6. Capitalize first letters

This snippet simply uses the method title() to capitalize first letters of every word in a string.

``````s = "programming is awesome"

print(s.title()) # Programming Is Awesome
``````

## 7. Chunk

This method chunks a list into smaller lists of a specified size.

``````def chunk(list, size):
return [list[i:i+size] for i in range(0,len(list), size)]
``````

## 8. Compact

This method removes falsy values (False, None, 0 and “”) from a list by using filter().

``````def compact(lst):
return list(filter(None, lst))

compact([0, 1, False, 2, '', 3, 'a', 's', 34]) # [ 1, 2, 3, 'a', 's', 34 ]
``````

## 9. Count by

This snippet can be used to transpose a 2D array.

``````array = [['a', 'b'], ['c', 'd'], ['e', 'f']]
transposed = zip(*array)
print(transposed) # [('a', 'c', 'e'), ('b', 'd', 'f')]
``````

## 10. Chained comparison

You can do multiple comparisons with all kinds of operators in a single line.

``````a = 3
print( 2 < a < 8) # True
print(1 == a < 2) # False
``````

## 11. Comma-separated

This snippet can be used to turn a list of strings into a single string with each element from the list separated by commas.

``````hobbies = ["basketball", "football", "swimming"]

print("My hobbies are:") # My hobbies are:
print(", ".join(hobbies)) # basketball, football, swimming
``````

## 12. Get vowels

This method gets vowels (‘a’, ‘e’, ‘i’, ‘o’, ‘u’) found in a string.

``````def get_vowels(string):
return [each for each in string if each in 'aeiou']

get_vowels('foobar') # ['o', 'o', 'a']
get_vowels('gym') # []
``````

## 13. Decapitalize

This method can be used to turn the first letter of the given string into lowercase.

``````def decapitalize(str):
return str[:1].lower() + str[1:]

decapitalize('FooBar') # 'fooBar'
decapitalize('FooBar') # 'fooBar'
``````

## 14. Flatten

The following methods flatten a potentially deep list using recursion.

``````def spread(arg):
ret = []
for i in arg:
if isinstance(i, list):
ret.extend(i)
else:
ret.append(i)
return ret

def deep_flatten(xs):
flat_list = []
[flat_list.extend(deep_flatten(x)) for x in xs] if isinstance(xs, list) else flat_list.append(xs)
return flat_list

deep_flatten([1, [2], [[3], 4], 5]) # [1,2,3,4,5]
``````

## 15. Difference

This method finds the difference between two iterables by keeping only the values that are in the first one.

``````def difference(a, b):
set_a = set(a)
set_b = set(b)
comparison = set_a.difference(set_b)
return list(comparison)

difference([1,2,3], [1,2,4]) # [3]
``````

## 16. Difference by

The following method returns the difference between two lists after applying a given function to each element of both lists.

``````def difference_by(a, b, fn):
b = set(map(fn, b))
return [item for item in a if fn(item) not in b]

from math import floor
difference_by([2.1, 1.2], [2.3, 3.4], floor) # [1.2]
difference_by([{ 'x': 2 }, { 'x': 1 }], [{ 'x': 1 }], lambda v : v['x']) # [ { x: 2 } ]
``````

## 17. Chained function call

You can call multiple functions inside a single line.

``````def add(a, b):
return a + b

def subtract(a, b):
return a - b

a, b = 4, 5
print((subtract if a > b else add)(a, b)) # 9
``````

## 18. Has duplicates

The following method checks whether a list has duplicate values by using the fact that set() contains only unique elements.

``````def has_duplicates(lst):
return len(lst) != len(set(lst))

x = [1,2,3,4,5,5]
y = [1,2,3,4,5]
has_duplicates(x) # True
has_duplicates(y) # False
``````

## 19. Merge two dictionaries

The following method can be used to merge two dictionaries.

``````def merge_two_dicts(a, b):
c = a.copy()   # make a copy of a
c.update(b)    # modify keys and values of a with the ones from b
return c

a = { 'x': 1, 'y': 2}
b = { 'y': 3, 'z': 4}
print(merge_two_dicts(a, b)) # {'y': 3, 'x': 1, 'z': 4}
``````

In Python 3.5 and above, you can also do it like the following:

``````def merge_dictionaries(a, b)
return {**a, **b}

a = { 'x': 1, 'y': 2}
b = { 'y': 3, 'z': 4}
print(merge_dictionaries(a, b)) # {'y': 3, 'x': 1, 'z': 4}
``````

## 20. Convert two lists into a dictionary

The following method can be used to convert two lists into a dictionary.

``````def to_dictionary(keys, values):
return dict(zip(keys, values))

keys = ["a", "b", "c"]
values = [2, 3, 4]
print(to_dictionary(keys, values)) # {'a': 2, 'c': 4, 'b': 3}
``````

## 21. Use enumerate

This snippet shows that you can use enumerate to get both the values and the indexes of lists.

``````list = ["a", "b", "c", "d"]
for index, element in enumerate(list):
print("Value", element, "Index ", index, )
# ('Value', 'a', 'Index ', 0)
# ('Value', 'b', 'Index ', 1)
#('Value', 'c', 'Index ', 2)
# ('Value', 'd', 'Index ', 3)
``````

## 22. Time spent

This snippet can be used to calculate the time it takes to execute a particular code.

``````import time

start_time = time.time()

a = 1
b = 2
c = a + b
print(c) #3

end_time = time.time()
total_time = end_time - start_time
print("Time: ", total_time)

# ('Time: ', 1.1205673217773438e-05)
``````

## 23. Try else

You can have an else clause as part of a try/except block, which is executed if no exception is thrown.

``````try:
2*3
except TypeError:
print("An exception was raised")
else:
print("Thank God, no exceptions were raised.")

#Thank God, no exceptions were raised.
``````

## 24. Most frequent

This method returns the most frequent element that appears in a list.

``````def most_frequent(list):
return max(set(list), key = list.count)

numbers = [1,2,1,2,3,2,1,4,2]
most_frequent(numbers)
``````

## 25. Palindrome

This method checks whether a given string is a palindrome.

``````def palindrome(a):
return a == a[::-1]

palindrome('mom') # True
``````

## 26. Calculator without if-else

The following snippet shows how you can write a simple calculator without the need to use if-else conditions.

``````import operator
action = {
"-": operator.sub,
"/": operator.truediv,
"*": operator.mul,
"**": pow
}
print(action['-'](50, 25)) # 25
``````

## 27. Shuffle

This snippet can be used to randomize the order of the elements in a list. Note that shuffle works in place, and returns None.

``````from random import shuffle

foo = [1, 2, 3, 4]
shuffle(foo)
print(foo) # [1, 4, 3, 2] , foo = [1, 2, 3, 4]
``````

This method flattens a list similarly like [].concat(…arr) in JavaScript.

``````def spread(arg):
ret = []
for i in arg:
if isinstance(i, list):
ret.extend(i)
else:
ret.append(i)
return ret

``````

## 29. Swap values

A really quick way for swapping two variables without having to use an additional one.

``````a, b = -1, 14
a, b = b, a

print(a) # 14
print(b) # -1
``````

## 30. Get default value for missing keys

This snippet shows how you can get a default value in case a key you are looking for is not included in the dictionary.

``````d = {'a': 1, 'b': 2}

print(d.get('c', 3)) # 3
``````

# How to building Python Data Science Container using Docker

Artificial Intelligence(AI) and Machine Learning(ML) are literally on fire these days. Powering a wide spectrum of use-cases ranging from self-driving cars to drug discovery and to God knows what. AI and ML have a bright and thriving future ahead of them.

On the other hand, Docker revolutionized the computing world through the introduction of ephemeral lightweight containers. Containers basically package all the software required to run inside an image(a bunch of read-only layers) with a COW(Copy On Write) layer to persist the data.

## Python Data Science Packages

Our Python data science container makes use of the following super cool python packages:

1. NumPy: NumPy or Numeric Python supports large, multi-dimensional arrays and matrices. It provides fast precompiled functions for mathematical and numerical routines. In addition, NumPy optimizes Python programming with powerful data structures for efficient computation of multi-dimensional arrays and matrices.

2. SciPy: SciPy provides useful functions for regression, minimization, Fourier-transformation, and many more. Based on NumPy, SciPy extends its capabilities. SciPy’s main data structure is again a multidimensional array, implemented by Numpy. The package contains tools that help with solving linear algebra, probability theory, integral calculus, and many more tasks.

3. Pandas: Pandas offer versatile and powerful tools for manipulating data structures and performing extensive data analysis. It works well with incomplete, unstructured, and unordered real-world data — and comes with tools for shaping, aggregating, analyzing, and visualizing datasets.

4. SciKit-Learn: Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. It is one of the best-known machine-learning libraries for python. The Scikit-learn package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. The primary emphasis is upon ease of use, performance, documentation, and API consistency. With minimal dependencies and easy distribution under the simplified BSD license, SciKit-Learn is widely used in academic and commercial settings. Scikit-learn exposes a concise and consistent interface to the common machine learning algorithms, making it simple to bring ML into production systems.

5. Matplotlib: Matplotlib is a Python 2D plotting library, capable of producing publication quality figures in a wide variety of hardcopy formats and interactive environments across platforms. Matplotlib can be used in Python scripts, the Python and IPython shell, the Jupyter notebook, web application servers, and four graphical user interface toolkits.

6. NLTK: NLTK is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning.

## Building the Data Science Container

Python is fast becoming the go-to language for data scientists and for this reason we are going to use Python as the language of choice for building our data science container.

## The Base Alpine Linux Image

Alpine Linux is a tiny Linux distribution designed for power users who appreciate security, simplicity and resource efficiency.

As claimed by Alpine:

Small. Simple. Secure. Alpine Linux is a security-oriented, lightweight Linux distribution based on musl libc and busybox.

The Alpine image is surprisingly tiny with a size of no more than 8MB for containers. With minimal packages installed to reduce the attack surface on the underlying container. This makes Alpine an image of choice for our data science container.

``````\$ docker container run --rm alpine:latest cat /etc/os-release

``````

In our, Dockerfile we can simply use the Alpine base image as:

``````FROM alpine:latest
``````

### Talk is cheap let’s build the Dockerfile

Now let’s work our way through the Dockerfile.

The FROM directive is used to set alpine:latest as the base image. Using the WORKDIR directive we set the /var/www as the working directory for our container. The ENV PACKAGES lists the software packages required for our container like git, blas and libgfortran. The python packages for our data science container are defined in the ENV PACKAGES.

We have combined all the commands under a single Dockerfile RUN directive to reduce the number of layers which in turn helps in reducing the resultant image size.

## Building and tagging the image

Now that we have our Dockerfile defined, navigate to the folder with the Dockerfile using the terminal and build the image using the following command:

``````\$ docker build -t faizanbashir/python-datascience:2.7 -f Dockerfile .

``````

The -t flag is used to name a tag in the 'name:tag' format. The -f tag is used to define the name of the Dockerfile (Default is 'PATH/Dockerfile').

## Running the container

We have successfully built and tagged the docker image, now we can run the container using the following command:

``````\$ docker container run --rm -it faizanbashir/python-datascience:2.7 python

``````

Voila, we are greeted by the sight of a python shell ready to perform all kinds of cool data science stuff.

``````Python 2.7.15 (default, Aug 16 2018, 14:17:09) [GCC 6.4.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>

``````

Our container comes with Python 2.7, but don’t be sad if you wanna work with Python 3.6. Lo, behold the Dockerfile for Python 3.6:

Build and tag the image like so:

``````\$ docker build -t faizanbashir/python-datascience:3.6 -f Dockerfile .

``````

Run the container like so:

``````\$ docker container run --rm -it faizanbashir/python-datascience:3.6 python

``````

With this, you have a ready to use container for doing all kinds of cool data science stuff.

## Serving Puddin’

Figures, you have the time and resources to set up all this stuff. In case you don’t, you can pull the existing images that I have already built and pushed to Docker’s registry Docker Hub using:

``````# For Python 2.7 pull
\$ docker pull faizanbashir/python-datascience:2.7# For Python 3.6 pull
\$ docker pull faizanbashir/python-datascience:3.6
``````

After pulling the images you can use the image or extend the same in your Dockerfile file or use it as an image in your docker-compose or stack file.

## Aftermath

The world of AI, ML is getting pretty exciting these days and will continue to become even more exciting. Big players are investing heavily in these domains. About time you start to harness the power of data, who knows it might lead to something wonderful.

You can check out the code here.

# Go Top Programming Languages in 2020 from Authentic Surveys

Comparing Programming Languages is a very complex thing and so there are many graphical illustration/jokes trying to symbolize Programming language. I found few and I am starting this article with those.

In simple words, Programming Language empowers human to instruct and control machine. So, it is natural that there will be so many languages which try to make this process more powerful and simple. For this very reason there are hundreds of programming languages, many of those programming languages are now out of active use, few are going to be obsolete in coming years and then there are languages which is going to continue and prove its usage in coming years and then there are new programming language fighting for it acceptance.

This article going to present the trends of top Programming Languages which will continue in the coming year 2020. To predict the trend of the programming language in 2020 this article uses data from authentic surveys, various collected statistics, search results and salary trends according to programming languages. This article will help the new learner to pick a programming language to learn and for expert, it will help to decide either to switch to another language or to continue with his expertise language.

In the next section, I have prepared two tables which summarize the popularity trend of Programming Languages in the last five years (2015-19). The data is taken from Stackoverflow popularity survey 2015-2019. For a clear and accurate understanding, the programming languages are divided into two groups, first, languages which have origin before 2000 and the second group has languages which came after 2000. The selection of 2000 as the boundary is just random but very helpful to understand the programming trend under these two groups. The table also lists origin year and main or documented purpose of these programming/scripting languages.

## Observations:

There is a decrease in the popularity of all languages from 2018 to 2019 except Python.

### Python

Python is the only language continuously on rising since last five years. It is a general-purpose language, so someone wants to learn just one programming in 2020 and want to cover more area of software development then Python could be chosen**.**

### Java

Java was on rising but fall in 2019, the reason could Kotlin gaining popularity on the Android platform. Java is a good choice for a programming language but now it is under Oracle and Google is promoting Kotlin so it is in the conflicted zone. As a matter of fact still, the large number of the company is using Java and going to continue with Java due to its developers base, framework, and legacy application.

### C/C++

C and C++ are still holding with approx 20% and it will be there due to its inherent features and legacy system.

### JavaScript

JavaScript popularity can be attributed to the growth of popular JavaScript library and framework like node.js, etc. JS is the language for the dynamic website and this going to be top for coming years because of its active development, support from Mozilla and penalty of libraries and frameworks. So, if someone wants to be web development, javascript is a must.

### R

R is gaining popularity in recent years and reason would be growth and popularity of data analysis. It is used by data scientist but again much behind in comparison to Python which has established as general-purpose languages and enjoy active developers with lots of data science libraries and modules. So, one can prefer Python over R if they have to choose only one otherwise if wanted carrier in Data Sciences then learning both will a good option.

### Ruby

Like PHP, Ruby is also facing tough competition from JavaScript and even Python to establish as back-end web development programming language. So, again for web development javascript and Python (server-side (Flask, Django, etc.) would be a good choice and will offer more domain flexibility than Ruby.

### PHP

There is a sharp decline in PHP popularity in 2019 and it can be traced back to server-side acceptance of javascript and Python. So, if someone wants to go to server-side web development then still PHP is a good choice with a large number of popular framework like CakePHP, Codeigniter, etc. otherwise choosing a general-purpose programming language would be better.

### Objective-C

Objective-C was the main language for Apple’s software such as macOS, iOS, etc. before Apple moved to Swift language. So this transition is reflected in the popularity of both languages, i.e. there is a fall in popularity for Objective-C and the popularity of Swift is rising. So, again if someone wants to be a developer for Apple’s products then Swift should be the language of choice.

### Swift

Swift has replaced the Objective-C as the main language for Apple-related software and application. Since it is supported and promoted by Apple so there is an increase in popularity since its inception and as Apple is going to continue with it so if someone is looking for Apple-specific development platform then Swift is going to be a must-know programming language. This is mostly vendor and product-specific language with very less usage outside Apple’s eco-system.

### Go

Go (Golang) is getting popularity as maintain, use and promoted by Google. The motivation of Go development was to address criticism of some of the popular languages and keeping the best of them in one place. Since 2017, Go is moving upward in popularity and with Google support, it is going to enjoy this in coming years. Google is also making Go as a primary language for new projects and replacing other languages with Go, this trend going to make useful and important to learn in coming years so one can pick Go as a new programming language.

### Kotlin

Kotlin is being offered as an alternative of Java for Android development and again it is supported and promoted by Google so it is also picking up by developers and gaining popularity in recent years. So, with the growth of Android, support of Google and with clean and short syntax it is going to be a choice of Android application developers and is a good choice to learn for Android App developer. Kotlin going to be shine as a prominent programming environment for Android development.

### Scala

Scala tries to establish as an alternative to Java but didn’t get very well among developers. It doesn’t have big support from any multi-national company, perceive as functional languages and dependency on JVM doesn’t provide much scope to rise in popularity. There could be steady growth but very slow and surely not a language to learn as a beginner.

### Julia

Julia aims to bring the speed of ‘C’ and simplicity of Python but it is strange that didn’t found any popularity in Stackoverflow survey but gaining popularity among data science domain and being seen as a challenger for R and Python in long run. Surely, there will be growth in Julia but still, Python or R is better for job and growth.

### C#

C# is the language for the .NET framework and developed by Microsoft. Its popularity is approx constant over past years and going to continue with a similar trend. This is vendor-specific language so one can pick this language if want to work in the Microsoft development environment. Recently, Microsoft has open-sourced the .NET so there would be some upward trend but again it is vendor-specific so there won’t be much affected.

### Rust

Rust, Clojure, etc. are languages which have a user-base but not so popular so surely not going to have an upward swing in popularity.

## A Picture Says a Thousand Words

To understand a clear trend and picture of top programming language growth let keep a picture of it by the various chart. The figure 1 and figure2 gives a very clear picture that in old language stack JavaScript is far ahead than others and credit goes to boom in web development, then C and C++ together competing very closer to Java. Python is moving upward in popularity and only language which popularity is constantly increasing in the last 5 years. New languages are getting popularity and most of them are supported by the multi-national company and bit IT giant like Microsoft, Google and Apple.

## Loved and Wanted Languages

From above Table and Figure, few observations are very obvious, Love of Rust is growing in last five years whereas Swift is loosing love from developers and Python is in between these two and last two years have gain for Python. One more unique observation is that out of 5 loved languages 4 are from post 2000 group while only Python is the older language and Kotlin love started with addition of Kotlin for Android development post 2017.

From above table, wish of developing in javascript and Python is growing in last years and this reflect in popularity and love for the language. There is a sharp decline in Java and this is due to the addition of Kotlin as alternative for Android app development and also change of policy by Oracle who own Java now.

## Technologies and Programming Languages

In this figure, one can see that the largest cluster is for Web development and JavaScript and its various framework is dominating the cluster this is USP of JavaScript growth. The second-largest cluster is of Microsoft technologies and Python technologies which again clear the popularity and love for the language. Python cluster is linked with data science technologies which highlight the growth story of Python.

## TIOBE Index

TIOBE index rank programming language based on search engine search result. The selection of search engines and programming language is defined in its page. The ratings are calculated by counting hits of the most popular search engines. The search query that is used is +”<language> programming”. In TIOBE index Java is dominating the ranking in the last two decades where C is holding the 1st and 2nd rank for the last 30 years. Python has come a long way in the last two decades i.e. 24th in 1999 to 3rd in 2019. If someone merges the C and C++ then it would hold the 1st positions forever.

In the new languages (post-2000), Rust moved up in ranking i.e. from 33rd to 28th, and Julia from 50th to 39th. It is also interesting to note that Kotlin doesn’t seem to come closer to the top 20.

## Popularity of Programming Language (PYPL) Index

The PYPL index is created by analyzing how often language tutorials are searched on Google. The more a language tutorial is searched, the more popular the language is assumed to be. It is a leading indicator. The raw data comes from Google Trends.

Below Figure verifies that the top 3 languages are Python, Java, and JavaScript. C#, PHP, C/C++ also secure top position, this trend is again similar to stack-overflow, and TIOBE index.

Above Figure indicates that among new programming Language i.e. post 2000 Kotlin, Go, Rust, and Julia is moving up in the ranking.

## Job Market and Salary

Salary depends upon the geographical area and demand of the products, a programming language based salary comparison is just a tool to predict or estimate the salary trend. We have summarized salary based on programming language from popular survey i.e. Dice salary survey 2018 and Stack-overflow survey 2018 and 2019.

From the above table, it is very clear from both survey that Go/Golang is a very high paid job in the market and even stands 1st rank in a high paid job in stack-overflow 2019 survey and Dice Salary Survey 2018.

## Language Predictability

So, as closing remarks, It is easy to predict a language trend but choosing only one language to learn is a really difficult choice and totally depend upon the individual choice and their future plans, for example, if you want to work in Web Development you can’t afford neglecting Javascript, if you want to work with Apple’s products you can’t neglect Swift now, if your taste is in system-level programming then C and C++ is your friend, Python makes you run faster in many domains and currently darling in Data science. You see each language takes you on a different journey. Choose your destination and then drive with the language of that path.

You may also like: Programming Languages - Trend Predictions in 2020 and Next Years.

We’ll love to know your opinion. What is the Best Programming Language for you?

Python is a no-BS programming language. Readability and simplicity of design are two of the biggest reasons for its immense popularity.

This is why it is worthwhile to remember some common Python tricks to help improve your code design. These will save you the trouble of surfing Stack Overflow every time you need to do something.
The following tricks will prove handy in your day-to-day coding exercises.

## 1. Finding Unique Elements in a String

The following snippet can be used to find all the unique elements in a string. We use the property that all elements in a set are unique.

``````my_string = "aavvccccddddeee"

# converting the string to a set
temp_set = set(my_string)

# stitching set into a string using join
new_string = ''.join(temp_set)

print(new_string)
``````

## 2. Using rhe Title Case (First Letter Caps)

The following snippet can be used to convert a string to title case. This is done using the `title()` method of the string class.

``````my_string = "my name is chaitanya baweja"

# using the title() function of string class
new_string = my_string.title()

print(new_string)

# Output
# My Name Is Chaitanya Baweja
``````

## 3. Reversing a String

The following snippet reverses a string using the Python slicing operation.

``````# Reversing a string using slicing

my_string = "ABCDE"
reversed_string = my_string[::-1]

print(reversed_string)

# Output
# EDCBA
``````

## 4. Printing a String or a List n Times

You can use multiplication (*) with strings or lists. This allows us to multiply them as many times as we like.

``````n = 3 # number of repetitions

my_string = "abcd"
my_list = [1,2,3]

print(my_string*n)
# abcdabcdabcd

print(my_list*n)
# [1,2,3,1,2,3,1,2,3]
``````

An interesting use case of this could be to define a list with a constant value — let’s say zero.

``````n = 4
my_list = [0]*n # n denotes the length of the required list
# [0, 0, 0, 0]
``````

## 5. Combining a List of Strings Into a Single String

The join() method combines a list of strings passed as an argument into a single string. In our case, we separate them using the comma separator.

``````list_of_strings = ['My', 'name', 'is', 'Chaitanya', 'Baweja']

# Using join with the comma separator
print(','.join(list_of_strings))

# Output
# My,name,is,Chaitanya,Baweja
``````

## 6. Swap Values Between Two Variables

Python makes it quite simple to swap values between two variables without using another variable.

``````a = 1
b = 2

a, b = b, a

print(a) # 2
print(b) # 1
``````

## 7. Split a String Into a List of Substrings

We can split a string into a list of substrings using the .split() method in the string class. You can also pass as an argument the separator on which you wish to split.

``````string_1 = "My name is Chaitanya Baweja"
string_2 = "sample/ string 2"

# default separator ' '
print(string_1.split())
# ['My', 'name', 'is', 'Chaitanya', 'Baweja']

# defining separator as '/'
print(string_2.split('/'))
# ['sample', ' string 2']
``````

## 8. List Comprehension

List comprehension provides us with an elegant way of creating lists based on other lists.
The following snippet creates a new list by multiplying each element of the old list by two.

``````# Multiplying each element in a list by 2

original_list = [1,2,3,4]

new_list = [2*x for x in original_list]

print(new_list)
# [2,4,6,8]
``````

## 9. Check If a Given String Is a Palindrome or Not

We have already discussed how to reverse a string. So palindromes become a straightforward program in Python.

``````my_string = "abcba"

if my_string == my_string[::-1]:
print("palindrome")
else:
print("not palindrome")

# Output
# palindrome
``````

## 10. Using Enumerate to Get Index/Value Pairs

The following script uses enumerate to iterate through values in a list along with their indices.

``````my_list = ['a', 'b', 'c', 'd', 'e']

for index, value in enumerate(my_list):
print('{0}: {1}'.format(index, value))

# 0: a
# 1: b
# 2: c
# 3: d
# 4: e
``````

## 11. Find Whether Two Strings are Anagrams

An interesting application of the `Counter `class is to find anagrams.

An anagram is a word or phrase formed by rearranging the letters of a different word or phrase.

If the `Counter` objects of two strings are equal, then they are anagrams.

``````from collections import Counter

str_1, str_2, str_3 = "acbde", "abced", "abcda"
cnt_1, cnt_2, cnt_3  = Counter(str_1), Counter(str_2), Counter(str_3)

if cnt_1 == cnt_2:
print('1 and 2 anagram')
if cnt_1 == cnt_3:
print('1 and 3 anagram')
``````

## 12. Using the try-except-else Block

Error handling in Python can be done easily using the try/except block. Adding an else statement to this block might be useful. It’s run when there is no exception raised in the try block.
If you need to run something irrespective of exception, use `finally`.

``````a, b = 1,0

try:
print(a/b)
# exception raised when b is 0
except ZeroDivisionError:
print("division by zero")
else:
print("no exceptions raised")
finally:
print("Run this always")
``````

## 13. Frequency of Elements in a List

There are multiple ways of doing this, but my favorite is using the Python `Counter` class.

Python counter keeps track of the frequency of each element in the container. `Counter()` returns a dictionary with elements as keys and frequency as values.

We also use the `most_common()` function to get the`most_frequent`element in the list.

``````# finding frequency of each element in a list
from collections import Counter

my_list = ['a','a','b','b','b','c','d','d','d','d','d']
count = Counter(my_list) # defining a counter object

print(count) # Of all elements
# Counter({'d': 5, 'b': 3, 'a': 2, 'c': 1})

print(count['b']) # of individual element
# 3

print(count.most_common(1)) # most frequent element
# [('d', 5)]
``````

## 14. Check the Memory Usage of an Object

The following script can be used to check the memory usage of an object. Read more about it here.

``````import sys

num = 21

print(sys.getsizeof(num))

# In Python 2, 24
# In Python 3, 28
``````

## 15. Sampling From a List

The following snippet generates` n` number of random samples from a given list using the `random` library.

``````import random

my_list = ['a', 'b', 'c', 'd', 'e']
num_samples = 2

samples = random.sample(my_list,num_samples)
print(samples)
# [ 'a', 'e'] this will have any 2 random values
``````

I have been recommended the secrets library for generating random samples for cryptography purposes. The following snippet will work
only on Python 3.

``````import secrets                              # imports secure module.
secure_random = secrets.SystemRandom()      # creates a secure random object.

my_list = ['a','b','c','d','e']
num_samples = 2

samples = secure_random.sample(my_list, num_samples)

print(samples)
# [ 'e', 'd'] this will have any 2 random values
``````

## 16. Time Taken to Execute a Piece of Code

The following snippet uses the `time` library to calculate the time taken to execute a piece of code.

``````import time

start_time = time.time()
# Code to check follows
a, b = 1,2
c = a+ b
# Code to check ends
end_time = time.time()
time_taken_in_micro = (end_time- start_time)*(10**6)

print(" Time taken in micro_seconds: {0} ms").format(time_taken_in_micro)
``````

## 17. Flattening a List of Lists

Sometimes you’re not sure about the nesting depth of your list, and you simply want all the elements in a single flat list.
Here’s how you can get that:

``````from iteration_utilities import deepflatten

# if you only have one depth nested_list, use this
def flatten(l):
return [item for sublist in l for item in sublist]

l = [[1,2,3],[3]]
print(flatten(l))
# [1, 2, 3, 3]

# if you don't know how deep the list is nested
l = [[1,2,3],[4,[5],[6,7]],[8,[9,[10]]]]

print(list(deepflatten(l, depth=3)))
# [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
``````

Numpy flatten is a better way to do this if you have a properly formatted array.

## 18. Merging Two Dictionaries

While in Python 2, we used the `update()` method to merge two dictionaries; Python 3.5 made the process even simpler.
In the script given below, two dictionaries are merged. Values from the second dictionary are used in case of intersections.

``````dict_1 = {'apple': 9, 'banana': 6}
dict_2 = {'banana': 4, 'orange': 8}

combined_dict = {**dict_1, **dict_2}

print(combined_dict)
# Output
# {'apple': 9, 'banana': 4, 'orange': 8}
``````