Reading from Standard Output and Standard Error

When you run programs with Popen, output is just sent to stdout as usual which is why you've been able to see the output from all the examples to far.

If you want to be able to read the standard output from a program you have to set the stdout argument in the initial call to Popen to specify that a pipe should be opened to the process. The special value you set is subprocess.PIPE:

subprocess.PIPE

    Special value that can be used as the stdin, stdout or stderr argument to Popen and indicates that a pipe to the standard stream should be opened.

Here's an example:

>>> process = subprocess.Popen(['echo', 'Hello World!'], shell=False, stdout=subprocess.PIPE)

To read the output from the pipe you use the communicate() method:

>>> print process.communicate()
('Hello World!\n', None)

The value returned from a call to communicate() is a tuple with two values, the first is the data from stdout and the second is the data from stderr.

Here's a program we can use to test the stdout and stderr behaviour. Save it as test1.py:

import sys
sys.stdout.write('Message to stdout\n')
sys.stderr.write('Message to stderr\n')

Let's execute this:

>>> process = subprocess.Popen(['python', 'test1.py'], shell=False, stdout=subprocess.PIPE)
Message to stderr
>>> print process.communicate()
('Message to stdout\n', None)

Notice that the message to stderr gets displayed as it is generated but the message to stdout is read via the pipe. This is because we only set up a pipe to stdout. Let's set up one to stderr too.

>>> process = subprocess.Popen(['python', 'test1.py'], shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> print process.communicate()
('Message to stdout\n', 'Message to stderr\n')

This time both stdout and stderr can be accessed from Python.

Now that all the messages have been printed, if we call communicate() again we get an error:

>>> print process.communicate()
Traceback (most recent call last):
  File "", line 1, in
  File "/usr/lib/python2.5/subprocess.py", line 668, in communicate
    return self._communicate(input)
  File "/usr/lib/python2.5/subprocess.py", line 1207, in _communicate
    rlist, wlist, xlist = select.select(read_set, write_set, [])
ValueError: I/O operation on closed file

The communicate() method only reads data from stdout and stderr, until end-of-file is reached.

Redirecting stderr to stdout

If you want messages to stderr to be piped to stdout, you can set a special value for the stderr argument: 

stderr=subprocess.STDOUT.