File management with python

File management with python

File management with python

Most of us have had that one experience where we had a ton of dis-organized files in our machines. It happens. One minute, you're opening a large zip file, the next thing you know, the files are everywhere in the directory, mixing with all your important files and randomly placed leaving you with the task of manually sorting what needs to go where. It's a real pain. To ease this process, we're going to delve into file management with python the smart way.

Work smart, not hard.

Let's begin. We'll be using python 3.4 or greater. 

Assuming you've got python up and running already, we're going to take a walk with the OS module and a few others we will introduce along the way. Most of these come with python, so there's no need to install anything else to follow along.

Creating random files

Create a directory to work with. Call it ManageFiles. Inside this folder create another folder RandomFiles. Your directory structure should now look like this:

ManageFiles/
 |
 |_RandomFiles/

We're going to create random files to play with in the RandomFiles directory

Create a file createrandomfiles.py inside ManageFiles directory. You now have this:

ManageFiles/
 |
 |_ create_random_files.py
 |_RandomFiles/

Done? Now get in the following code, we'll get into its details in a moment.

import os
from pathlib import Path
import random


list_of_extensions = ['.rst','.txt','.md','.docx','.odt','.html','.ppt','.doc']

get into the RandomFiles directory

os.chdir('./RandomFiles')

for item in list_of_extensions: # create 20 random files for each file extension for num in range(20): # let the file begin with a random number between 1 to 50 file_name = random.randint(1,50) file_to_create = str(file_name) + item Path(file_to_create).touch()

As of python 3.4, we 've got pathlib, our little magic box. We also import python's random function for creating random numbers; Hold on to that thought, we're going to cover it when as we get to the line that uses it.

First off, we create a list of file extensions from where we will get our random files. Feel free to add to it. 

Next up, we change to the RandomFiles directory, then comes our loop, so here goes.

We are simply saying, take each item in this list_of_extensions and do the following to it. Let's take the .txt for instance. We get into another loop, where to this .txt, we do something to it 20 times.

Remember our import of random? We use it to select a random number between 1 and 50 for our file. In short, what this little loop does is save us, the less creative lot(don't worry, I'm part of this crew), the time of naming random files. We will simply create a file say 23.txt or 14.txt, provided it falls within our range of 50, twenty times. This is just so as to create a mess large enough to give pain when moving manually. The same process will be done with the other extensions. Next? Run this in your terminal.

python create_random_files.py

Congratulations! We now have a mess of a directory. Now to clean it up.

In the same location where our create_random_files.py is, create a file clean_up.py and get the below in.

Method 1:

import os
import shutil
import glob

get into the RandomFiles directory

os.chdir('./RandomFiles')

get the list of files in the directory RandomFiles

files_to_group = [] for random_file in os.listdir('.'): files_to_group.append(random_file)

get all the file extensions present

file_extensions = [] for our_file in files_to_group: file_extensions.append(os.path.splitext(our_file)[1])

print(set(file_extensions))

file_types = set(file_extensions)

for type in file_types: new_directory = type.replace(".", " ") os.mkdir(new_directory) # create directory with given name

for fname in glob.glob(f'*.{type[1:]}'):
    shutil.move(fname, new_directory)

For this, we import two new libraries; shutil and glob. The shutil will help us move our files while the glob will help find the files to classify. Just like before, this will all become clear as we get to the line.

First off, we get a list of all the files in the directory.

Here, we assume that we do not have a clue of what files are in the directory. This means unlike where you can get all the extensions present manually and use if statements or switch, we want the program to look through the directory and do this for us. What if the file had dozens of extensions or log files? Would you do this manually?

Once we get a list of all the files in the folder, we get into another loop, to get the file extensions of these files.

Notice how we use:

os.path.splitext(our_file)[1]

Currently, the our_file variable looks something like this 5.docx (for instance). When we split it, we get this:

('5', '.docx')

we then get the index [1] from it which in turn takes .docx since 5 is index [0].

So we now have the list of all file extensions present in the folder, whether repeated or not.

To make it non-repetitive, we make a set. This takes all the items from the list and gets only the unique items. In our case, if we had a list where we had an extension say .docx repeating itself over and over in the set would ensure we had only one of it.

# create a set and assign it to a variable 
file_types = set(file_extensions)

Remember our list of file types still has the . for every file extension. This would mean if we were to create a folder named exactly the same way, we would end up creating hidden folders and that is something we do not want.

So, as we loop over this set, we create a directory with the same extension name, only this time, we replace the . in the name with an empty string.

new_directory = type.replace(".", " ")

our directory would now be called 'docx'

We still need the .docx extension to move the files.

for fname in glob.glob(f'.{type[1:]}')

This simply implies take any file that ends with the .docx file extension(Notice the spacing used in f'*.{type[1:]}').There is no space.

The wild card  means a file can be named anything, provided it ends in .docx. Since we have already placed the period . we take the string we have and have everything else afterwards and that's why we use [1:] which just means take from after the first character, hence take docx.

What next? Move any file with this extension into the directory named as so.

shutil.move(fname, new_directory)

In this way, once a directory for the first file found in the loop has been created, no other duplicates can be made. In short, we will not have a folder to store 5.docx and many others to store 34.docx and so on. Once we have a directory made, all other folders looking like so will move there. That's it!

Method 2

You can alternatively, use generators. This is a fancy way of creating a list with a one liner.

import os
import shutil
import glob

get into the RandomFiles directory

os.chdir('./RandomFiles')

#take every file from the directory and add to a list for all files all_files = [x for x in os.listdir('.') ]

make a set for the extensions present in the directory

file_types = set((os.path.splitext(f)[1] for f in all_files))

for ftype in file_types: new_directory = ftype.replace(".", '') os.mkdir(new_directory)

for fname in glob.glob(f'*.{ftype[1:]}'):
    shutil.move(fname, new_directory)

Both of these will work. You've now got all your files sorted according to extension.

ManageFiles/
 |
 |_create_random_files.py
 |_RandomFiles/
    |_doc
    |_docx
    |_html
    |_md
    |_odt
    |_ppt

Woosh! That was a lot.We did save some time though. Any questions? Feel free to reach out. That's it for now.Stick around as we take it up a notch next week.

Originally published by Marvin  at dev.to

=================================================================

Thanks for reading :heart: If you liked this post, share it with all of your programming buddies! Follow me on Facebook | Twitter

Learn More

☞ Complete Python Bootcamp: Go from zero to hero in Python 3

☞ Python for Time Series Data Analysis

☞ Python Programming For Beginners From Scratch

☞ Python Network Programming | Network Apps & Hacking Tools

☞ Intro To SQLite Databases for Python Programming

☞ Ethical Hacking With Python, JavaScript and Kali Linux

☞ Beginner’s guide on Python: Learn python from scratch! (New)

☞ Python for Beginners: Complete Python Programming


python

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Python Tricks Every Developer Should Know

In this tutorial, you’re going to learn a variety of Python tricks that you can use to write your Python code in a more readable and efficient way like a pro.

How to Remove all Duplicate Files on your Drive via Python

Today you're going to learn how to use Python programming in a way that can ultimately save a lot of space on your drive by removing all the duplicates. We gonna use Python OS remove( ) method to remove the duplicates on our drive. Well, that's simple you just call remove ( ) with a parameter of the name of the file you wanna remove done.

Basic Data Types in Python | Python Web Development For Beginners

In the programming world, Data types play an important role. Each Variable is stored in different data types and responsible for various functions. Python had two different objects, and They are mutable and immutable objects.

How To Compare Tesla and Ford Company By Using Magic Methods in Python

Magic Methods are the special methods which gives us the ability to access built in syntactical features such as ‘<’, ‘>’, ‘==’, ‘+’ etc.. You must have worked with such methods without knowing them to be as magic methods. Magic methods can be identified with their names which start with __ and ends with __ like __init__, __call__, __str__ etc. These methods are also called Dunder Methods, because of their name starting and ending with Double Underscore (Dunder).

The Basics of Python OS Module

The OS module is a python module that provides the interface for interacting with the underlying operating system that Python is running.