Achieve ×66 speedup read time, ×25 write time, and ×0.39 filesize on your daily I/O operations.

Reading and writing files using Pandas and NumPy is an everyday task for Data Scientists and Engineers.

Let’s compare the most common functions that these libraries provide to write/read tabular data.

We can make our code much faster in these I/O operations, save time, and make our boss and ourselves happy.

We can also save serious amounts of disk space by choosing the appropriate save function.

First, let’s create a DataFrame of 10,000,000 rows and 2 columns.

#data-science #machine-learning #programming #pandas #csv(), np.save(), to_hdf(), to_pickle() functions #filesize