Working with files

The built-in open function lets you create, open, and modify files. This module adds those extra functions you need to rename and remove files:

Working with directories

The os module also contains a number of functions that work on entire directories.
The listdir function returns a list of all filenames in a given directory. The current and parent directory markers used on Unix and Windows (. and ..) are not included in this list.

# File: os-example-5.py
import os
for file in os.listdir("samples"):
    print file

sample.au
sample.jpg
sample.wav
...

The getcwd and chdir functions are used to get and set the current directory:

# File: os-example-4.py
import os
# where are we?
cwd = os.getcwd()
print "1", cwd
# go down
os.chdir("samples")
print "2", os.getcwd()
# go back up
os.chdir(os.pardir)
print "3", os.getcwd()

1 /ematter/librarybook
2 /ematter/librarybook/samples
3 /ematter/librarybook

The makedirs and removedirs functions are used to create and remove directory hierarchies.

# File: os-example-6.py
import os
os.makedirs("test/multiple/levels")
fp = open("test/multiple/levels/file", "w")
fp.write("inspector praline")
fp.close()
# remove the file
os.remove("test/multiple/levels/file")
# and all empty directories above it
os.removedirs("test/multiple/levels")

Note that removedirs removes all empty directories along the given path, starting with the last directory in the given path name. In contrast, the mkdir and rmdir functions can only handle a single directory level.

# File: os-example-7.py
import os
os.mkdir("test")
os.rmdir("test")
os.rmdir("samples") # this will fail

Traceback (innermost last):
  File "os-example-7", line 6, in ?
OSError: [Errno 41] Directory not empty: 'samples'

To remove non-empty directories, you can use the rmtree function in the shutil module.

Working with file attributes

The stat function fetches information about an existing file. It returns a 9-tuple which contains the size, inode change timestamp, modification timestamp, and access privileges.

# File: os-example-2.py
import os
import stat, time
infile = "samples/sample.jpg"
outfile = "out.jpg"
# copy contents
fi = open(infile, "rb")
fo = open(outfile, "wb")
while 1:
    s = fi.read(10000)
    if not s:
        break
    fo.write(s)
fi.close()
fo.close()
# copy mode and timestamp
st = os.stat(infile)
os.chmod(outfile, stat.S_IMODE(st[stat.ST_MODE]))
os.utime(outfile, (st[stat.ST_ATIME], st[stat.ST_MTIME]))
print "original", "=>"
print "mode", oct(stat.S_IMODE(st[stat.ST_MODE]))
print "atime", time.ctime(st[stat.ST_ATIME])
print "mtime", time.ctime(st[stat.ST_MTIME])
print "copy", "=>"
st = os.stat(outfile)
print "mode", oct(stat.S_IMODE(st[stat.ST_MODE]))
print "atime", time.ctime(st[stat.ST_ATIME])
print "mtime", time.ctime(st[stat.ST_MTIME])

original =>
mode 0666
atime Thu Oct 14 15:15:50 1999
mtime Mon Nov 13 15:42:36 1995
copy =>
mode 0666
atime Thu Oct 14 15:15:50 1999
mtime Mon Nov 13 15:42:36 1995

Working with processes

The system function runs a new command under the current process, and waits for it to finish.

Warning: Since os.system passes the command on to the shell as is, it can be dangerous to use if you don’t check the arguments carefully (consider running os.system(“viewer %s” % file) with the file variable set to “sample.jpg; rm -rf $HOME”). When unsure, it’s usually better to use exec or spawn instead (see below).

The exec function starts a new process, replacing the current one (“go to process”, in other words). In the following example, note that the “goodbye” message is never printed:

# File: os-exec-example-1.py
import os
import sys
program = "python"
arguments = ["hello.py"]
print os.execvp(program, (program,) +  tuple(arguments))
print "goodbye"

hello again, and welcome to the show

Python provides a whole bunch of exec functions, with slightly varying behavior. The above example uses execvp, which searches for the program along the standard path, passes the contents of the second argument tuple as individual arguments to that program, and runs it with the current set of environment variables. See the Python Library Reference for more information on the other seven ways to call this function.
Under Unix, you can call other programs from the current one by combining exec with two other functions, fork and wait. The former makes a copy of the current process, the latter waits for a child process to finish.

# File: os-exec-example-2.py
import os
import sys
def run(program, *args):
    pid = os.fork()
    if not pid:
        os.execvp(program, (program,) +  args)
    return os.wait()[0]
run("python", "hello.py")
print "goodbye"

hello again, and welcome to the show
goodbye

The fork returns zero in the new process (the return from fork is the first thing that happens in that process!), and a non-zero process identifier in the original process. Or in other words, “not pid” is true only if we’re in the new process.
fork and wait are not available on Windows, but you can use the spawn function instead. Unfortunately, there’s no standard version of spawn that searches for an executable along the path, so you have to do that yourself:

Working with daemon processes

On Unix, fork can also be used to turn the current process into a background process (a “daemon”). Basically, all you need to do is to fork off a copy of the current process, and terminate the original process:

# File: os-example-14.py
import os
import time
pid = os.fork()
if pid:
    os._exit(0) # kill original
print "daemon started"
time.sleep(10)
print "daemon terminated"

However, it takes a bit more work to create a real daemon. First, call setpgrp to make the new process a “process group leader”. Otherwise, signals sent to a (by that time) unrelated process group might cause problems in your daemon:

os.setpgrp()

It’s also a good idea to remove the user mode mask, to make sure files created by the daemon actually gets the mode flags specified by the program:

os.umask(0)

Then, you should redirect the stdout/stderr files, instead of just closing them. If you don’t do this, you may get unexpected exceptions the day some of your code tries to write something to the console via stdout or stderr.

class NullDevice:
    def write(self, s):
        pass
sys.stdin.close()
sys.stdout = NullDevice()
sys.stderr = NullDevice()

In other words, while Python’s print and C’s printf/fprintf won’t crash your program if the devices have been disconnected, sys.stdout.write() happily throws an IOError exception when the application runs as a daemon. But your program works just fine when running in the foreground…
By the way, the _exit function used in the examples above terminates the current process. In contrast to sys.exit, this works also if the caller happens to catch the SystemExit exception:

# File: os-example-9.py
import os
import sys
try:
    sys.exit(1)
except SystemExit, value:
    print "caught exit(%s)" % value
try:
    os._exit(2)
except SystemExit, value:
    print "caught exit(%s)" % value
print "bye!"

caught exit(1)