How to Search a Codebase with Python

In this post, I will show you how you can search an entire directory to find keywords in a file. Once we find files with offending code we will create a list so we can find the most offensive files and focus on those.

Iterating the File Tree

First, we will collect the paths for files we are interested in. Here we have the directory to search defined at the top. Also, we created an exclusion list. The exclusion list makes sure we don’t get files that aren’t relevant to our platform. The node_modules folder is a prime example of one we don’t want to include.

We initialize file paths outside the loop, as we will be iterating over it later.

OS.walk will help us with the hard work of iterating each of the directories. It returns the root directory name (as we iterate we are getting an updated root). Also, it will give us a list of subdirectory names and file names.

We are ignoring the subdirectory names because we just want the paths to files. Collecting them means iterating over the list of file names in the directory, check to make sure it has an appropriate extension, and make sure it doesn’t exist in our exclusion list.

import os

walk_dir = 'C:\\Directory\\ForWalking\\'

# If the file path contains these we dont want them
#    eg. C:\\Directory\\ForWalking\\node_modules will be ignored
exclusions = ["node_modules", "SolutionFiles", ".bin", "Test"]

# Array to store all our file paths
file_paths = []

# Iterate file tree
for root, sub_dirs, file_names in os.walk(walk_dir):
    
    # Iterate the file names in the directory
    for file_name in file_names:
        
        # We only are interested in Typscript and JS Files
        if file_name.endswith(".ts") 
           or file_name.endswith(".tsx") 
           or file_name.endswith(".js"):
           
            # If the file path doesnt have 
            #   anything from the exclusion list
            if not any(exclusion in root for exclusion in exclusions):
                file_paths.append(os.path.join(root, file_name))

Sort Through the File

Next, we will want to go through all the files we found and see if they have any of the offensive code. We created an array that contains all the methods we want to search for. Any occurrences of these will need to be updated later.

We will visit each of the files we found in our last step. When we visit them we will open and count the occurrences of any offensive code. If we find an occurrence, we will add to the occurrences array with the path, unsupported code, and the number of times it appeared.

# Occurances will track each time an offensive bit of code is found
# Its format will be:
#   File Path, Function, Num Occurances
occurances = []

# Methods that need to be update
nogos = [
".SetFocus(",
".IsValid(",
".Clear",
".IsDirty",
".RemoveItem",
".SetTime",
".RemoveItem",
".SetTime"
]

# Iterate previously collected file paths
for file_path in file_paths:
    
    # Open the file as read only ignoring unknown chars
    with open(file_path, 'r', encoding='utf8', errors='ignore' ) as f:
        contents = f.read()
        
        # Check each of the offensive code bits
        for string in nogos:
            countNogo = contents.count(string)
            
            # If there is offensive code in the file append it to 
            #   the occurances array
            if countNogo > 0:
                occurances.append([file_path, string, str(countNogo)])

# Create an output csv string
outCSV = "\n".join([",".join(line) for line in occurances])

# Write to file
with open("Outfile.csv", 'w+') as f:
    f.write(outCSV)

Results

Using this sort bit of scripting saved me from having to open 150+ files of code. Instead, it found the 26 files with offensive code so I can focus on those. Also, I was able to give my manager a better idea of the scope of the project in just a few minutes rather than several days.

For those who may look over the code and point out this could have been done in fewer steps using map, filter, and reduce — you are right! Not every situation needs polished code though. This is a great example of how with relatively little Python experience, anyone can save time at their job.

Thank for reading

#python #python3 #programming #cloud #developer

What is GEEK

Buddha Community

Why use Python for Software Development

No programming language is pretty much as diverse as Python. It enables building cutting edge applications effortlessly. Developers are as yet investigating the full capability of end-to-end Python development services in various areas. 

By areas, we mean FinTech, HealthTech, InsureTech, Cybersecurity, and that's just the beginning. These are New Economy areas, and Python has the ability to serve every one of them. The vast majority of them require massive computational abilities. Python's code is dynamic and powerful - equipped for taking care of the heavy traffic and substantial algorithmic capacities. 

Programming advancement is multidimensional today. Endeavor programming requires an intelligent application with AI and ML capacities. Shopper based applications require information examination to convey a superior client experience. Netflix, Trello, and Amazon are genuine instances of such applications. Python assists with building them effortlessly. 

5 Reasons to Utilize Python for Programming Web Apps 

Python can do such numerous things that developers can't discover enough reasons to admire it. Python application development isn't restricted to web and enterprise applications. It is exceptionally adaptable and superb for a wide range of uses.

Robust frameworks 

Python is known for its tools and frameworks. There's a structure for everything. Django is helpful for building web applications, venture applications, logical applications, and mathematical processing. Flask is another web improvement framework with no conditions. 

Web2Py, CherryPy, and Falcon offer incredible capabilities to customize Python development services. A large portion of them are open-source frameworks that allow quick turn of events. 

Simple to read and compose 

Python has an improved sentence structure - one that is like the English language. New engineers for Python can undoubtedly understand where they stand in the development process. The simplicity of composing allows quick application building. 

The motivation behind building Python, as said by its maker Guido Van Rossum, was to empower even beginner engineers to comprehend the programming language. The simple coding likewise permits developers to roll out speedy improvements without getting confused by pointless subtleties. 

Utilized by the best 

Alright - Python isn't simply one more programming language. It should have something, which is the reason the business giants use it. Furthermore, that too for different purposes. Developers at Google use Python to assemble framework organization systems, parallel information pusher, code audit, testing and QA, and substantially more. Netflix utilizes Python web development services for its recommendation algorithm and media player. 

Massive community support 

Python has a steadily developing community that offers enormous help. From amateurs to specialists, there's everybody. There are a lot of instructional exercises, documentation, and guides accessible for Python web development solutions. 

Today, numerous universities start with Python, adding to the quantity of individuals in the community. Frequently, Python designers team up on various tasks and help each other with algorithmic, utilitarian, and application critical thinking. 

Progressive applications 

Python is the greatest supporter of data science, Machine Learning, and Artificial Intelligence at any enterprise software development company. Its utilization cases in cutting edge applications are the most compelling motivation for its prosperity. Python is the second most well known tool after R for data analytics.

The simplicity of getting sorted out, overseeing, and visualizing information through unique libraries makes it ideal for data based applications. TensorFlow for neural networks and OpenCV for computer vision are two of Python's most well known use cases for Machine learning applications.

Summary

Thinking about the advances in programming and innovation, Python is a YES for an assorted scope of utilizations. Game development, web application development services, GUI advancement, ML and AI improvement, Enterprise and customer applications - every one of them uses Python to its full potential. 

The disadvantages of Python web improvement arrangements are regularly disregarded by developers and organizations because of the advantages it gives. They focus on quality over speed and performance over blunders. That is the reason it's a good idea to utilize Python for building the applications of the future.

#python development services #python development company #python app development #python development #python in web development #python software development

Binary Search in Python (Recursive and Iterative)

Searching for an element’s presence in a list is usually done using linear search and binary search. Linear search is time-consuming and memory expensive but is the simplest way to search for an element. On the other hand, Binary search is effective mainly due to the reduction of list dimension with each recursive function call or iteration. A practical implementation of binary search is autocompletion.

Python Binary Search Algorithm:

The objective of this project is to create a simple python program to implement binary search. It can be implemented in two ways: recursive (function calls) and iterative.

Project Prerequisites:

The project uses loops and functions to implement the search function. Hence good knowledge of python loops and function calls is sufficient to understand the code flow.

#python tutorials #binary search python #binary search python program #iterative binary search python #recursive binary search python

Python Tricks Every Developer Should Know

Python is awesome, it’s one of the easiest languages with simple and intuitive syntax but wait, have you ever thought that there might ways to write your python code simpler?

In this tutorial, you’re going to learn a variety of Python tricks that you can use to write your Python code in a more readable and efficient way like a pro.

Let’s get started

Swapping value in Python

Instead of creating a temporary variable to hold the value of the one while swapping, you can do this instead

>>> FirstName = "kalebu"
>>> LastName = "Jordan"
>>> FirstName, LastName = LastName, FirstName 
>>> print(FirstName, LastName)
('Jordan', 'kalebu')

#python #python-programming #python3 #python-tutorials #learn-python #python-tips #python-skills #python-development

How to Remove all Duplicate Files on your Drive via Python

Today you’re going to learn how to use Python programming in a way that can ultimately save a lot of space on your drive by removing all the duplicates.

Intro

In many situations you may find yourself having duplicates files on your disk and but when it comes to tracking and checking them manually it can tedious.

Heres a solution

Instead of tracking throughout your disk to see if there is a duplicate, you can automate the process using coding, by writing a program to recursively track through the disk and remove all the found duplicates and that’s what this article is about.

But How do we do it?

If we were to read the whole file and then compare it to the rest of the files recursively through the given directory it will take a very long time, then how do we do it?

The answer is hashing, with hashing can generate a given string of letters and numbers which act as the identity of a given file and if we find any other file with the same identity we gonna delete it.

There’s a variety of hashing algorithms out there such as

• md5
• sha1
• sha224, sha256, sha384 and sha512

#python-programming #python-tutorials #learn-python #python-project #python3 #python #python-skills #python-tips

Search Emails From A Domain Through Search Engines for Python

EmailFinder

Search emails from a domain through search engines for python

Installation:

> pip3 install emailfinder

Upgrades are also available using:

> pip3 install emailfinder --upgrade

Search Engines

• google: Ok (note cookies policy and Captcha!).
• bing: OK.
• baidu: OK (few requests).
• bing: Hunting Robots very fast.
  …

#email #python #search emails #search emails through search engines #search emails from a domain through search engines for python #domain