Proper way of reading in files from a directory using Python 2.6 in bash shell

Proper way of reading in files from a directory using Python 2.6 in bash shell

I am trying to read in files for text processing, and the idea is to run them through Hadoop pseudo distributed file system on my virtual machine, using map-reduce code I am writing. The interface is Ubuntu Linux, I am running Python 2.6 with the installation. I need to use sys.stdin for reading in the files, and sys.stdout so I pass from mapper to reducer. So here is my test code for the mapper:

I am trying to read in files for text processing, and the idea is to run them through Hadoop pseudo distributed file system on my virtual machine, using map-reduce code I am writing. The interface is Ubuntu Linux, I am running Python 2.6 with the installation. I need to use sys.stdin for reading in the files, and sys.stdout so I pass from mapper to reducer. So here is my test code for the mapper:

#!/usr/bin/env python

import sys import string import glob import os

files = glob.glob(sys.stdin) for file in files: with open(file) as infile: txt = infile.read() txt = txt.split() print(txt)

I'm not sure how glob works with sys.stdin, but this is not working. I get the following errors:

After testing with piping:

[[email protected] data]$ cat test | ./mapper.py

I get this:

cat: test: Is a directory
Traceback (most recent call last):
  File "./mapper.py", line 8, in <module>
    files = glob.glob(sys.stdin)
  File "/usr/lib64/python2.6/glob.py", line 16, in glob
    return list(iglob(pathname))
  File "/usr/lib64/python2.6/glob.py", line 24, in iglob
    if not has_magic(pathname):
  File "/usr/lib64/python2.6/glob.py", line 78, in has_magic
    return magic_check.search(s) is not None
TypeError: expected string or buffer

For the moment I am just trying to read in three small .txt files in one directory.

Thanks!

python bash hadoop

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Basic Data Types in Python | Python Web Development For Beginners

In the programming world, Data types play an important role. Each Variable is stored in different data types and responsible for various functions. Python had two different objects, and They are mutable and immutable objects.

How To Compare Tesla and Ford Company By Using Magic Methods in Python

Magic Methods are the special methods which gives us the ability to access built in syntactical features such as ‘<’, ‘>’, ‘==’, ‘+’ etc.. You must have worked with such methods without knowing them to be as magic methods. Magic methods can be identified with their names which start with __ and ends with __ like __init__, __call__, __str__ etc. These methods are also called Dunder Methods, because of their name starting and ending with Double Underscore (Dunder).

Python Programming: A Beginner’s Guide

Python is an interpreted, high-level, powerful general-purpose programming language. You may ask, Python’s a snake right? and Why is this programming language named after it?

Hire Python Developers

Are you looking for experienced, reliable, and qualified Python developers? If yes, you have reached the right place. At **[HourlyDeveloper.io](https://hourlydeveloper.io/ "HourlyDeveloper.io")**, our full-stack Python development services...

Python any: How to Check If Element is Iterable or Not

Python any() function returns True if any element of an iterable is True otherwise any() function returns False. The syntax is any().