Pandas is a Python library for data analysis and manipulation. Almost all operations in pandas
revolve around DataFrame
s, an abstract data structure tailor-made for handling a metric ton of data.
In the aforementioned metric ton of data, some of it is bound to be missing for various reasons. Resulting in a missing (null
/None
/Nan
) value in our DataFrame
.
Which is why, in this article, we’ll be discussing how to handle missing data in a Pandas DataFrame
.
Real-world datasets are rarely perfect. They may contain missing values, wrong data types, unreadable characters, erroneous lines, etc.
The first step to to any proper data analysis is cleaning and organizing the data we’ll later be using. We will discuss a few common problems related to data that might occur in a dataset.
We will be working with small employees dataset for this. The .csv
file looks like this:
First Name,Gender,Salary,Bonus %,Senior Management,Team
Douglas,Male,97308,6.945,TRUE,Marketing
Thomas,Male,61933,NaN,TRUE,
Jerry,Male,NA,9.34,TRUE,Finance
Dennis,n.a.,115163,10.125,FALSE,Legal
,Female,0,11.598,,Finance
Angela,,,18.523,TRUE,Engineering
Shawn,Male,111737,6.414,FALSE,na
Rachel,Female,142032,12.599,FALSE,Business Development
Linda,Female,57427,9.557,TRUE,Client Services
Stephanie,Female,36844,5.574,TRUE,Business Development
,,,,,
Let’s import it into a DataFrame
:
df = pd.read_csv('out.csv')
df
#python #pandas #tool