As a Machine learning engineer, it is a common practice to save the data and models in a CSV format. Though CSV format helps in storing data in a rectangular tabular format, it might not always be suitable for persisting all Pandas Dataframes. CSV files tend to be slow to read and write, take up more memory and space and most importantly CSVs don’t store information about data types.

In this article, we will understand how CSV handles different file formats, explore different ways to store the data and the performance of each of these persisting methods.

#developers corner #csv #pandas #persisting methods #pickle