How to Capture Output in Python Subprocess

1. Python Subprocess Capture Output
2. Approach 1: Use check_output to Capture Output of subprocess in Python
3. Approach 2: Use Popen.communicate to Capture Output of subprocess in Python

The main aim of this article is to demonstrate how can the output of a Subprocess be captured, stored and shown in Python.

Python Subprocess Capture Output

Subprocess is a built-in Python module that comes pre-installed with an installation file of Python. It is used to run new codes and applications by creating new processes where the given task (according to what was passed) is executed separately from its process.

One of the main advantages of using Subprocess is that it allows us to launch new programs from the program we have currently written.

An example of running such programs is if you want to clone some project from the Git repository or perhaps execute a C++ compiler binary that performs its specific tasks.

Consider we want to execute a program written in C++, whose code is below.

#include <iostream>

using namespace std;

int main() {
  int rows = 5, count = 0, count1 = 0, k = 0;
  for (int i = 1; i <= rows; ++i) {
    for (int space = 1; space <= rows - i; ++space) {
      cout << "  ";
      ++count;
    }

    while (k != 2 * i - 1) {
      if (count <= rows - 1) {
        cout << i + k << " ";
        ++count;

      }

      else {
        ++count1;
        cout << i + k - 2 * count1 << " ";
      }
      ++k;
    }
    count1 = count = k = 0;
    cout << endl;
  }

  return 0;
}

Which open executing independently gives the following output:

Now we execute the compiled binary file of the program written using Python.

import subprocess

p2 = subprocess.Popen("program\\pyramid.exe")
print(p2)

Output:

<Popen: returncode: None args: '....\\program\\pyramid.exe>
        1
      2 3 2
    3 4 5 4 3
  4 5 6 7 6 5 4
5 6 7 8 9 8 7 6 5

As seen in the code above, it is pretty evident that, instead of getting the string, we get the instance of the Popen, which is not the ideal case as we need the output of the string to perform processing on the output or for other purposes.

It must be noted that the pyramid output we see is not the result of printing the variable named p2 but rather the output of the executed program using the Popen.

This problem can be approached in multiple ways, some of which are described below.

Approach 1: Use check_output to Capture Output of subprocess in Python

Consider the following code:

from subprocess import check_output

out = check_output(["program\\pyramid.exe"])

print(out)

Output:

b'        1 \r\n      2 3 2 \r\n    3 4 5 4 3 \r\n  4 5 6 7 6 5 4 \r\n5 6 7 8 9 8 7 6 5 \r\n'

The output is in bytes form, which can be easily converted to string using the decode() function.

print(out.decode("utf-8"))

Output:

This is our desired output in the string variable named out.

The check_output() function runs our desired command and returns the output of the command of the executed program in the form of a byte string.

We can then use the decode() method to convert the byte string to a normal string. The encoding parameter to decode() parameters can vary depending on the output type.

In our case, it was utf-8, but it can vary from program to program and in different operating systems.

Approach 2: Use Popen.communicate to Capture Output of subprocess in Python

Consider the following code:

import subprocess

pipe = subprocess.Popen("\\program\\pyramid.exe", stdout=subprocess.PIPE)

text = pipe.communicate()[0]

print(text)

Output:

b'        1 \r\n      2 3 2 \r\n    3 4 5 4 3 \r\n  4 5 6 7 6 5 4 \r\n5 6 7 8 9 8 7 6 5 \r\n'

Like before, a byte string is returned, which can easily be returned using the decode() function.

print(text.decode("utf-8"))

Output:

Instead of using check_output, we can also use the Popen.communicate() method, one of the main advantages of check_output is that it allows one to redirect both stdout and stderr to our desired variables.

It works that Popen returns a tuple of stdout and stderr, so when you access [0], you’re just grabbing the stdout. You could also do text, err = pipe.communicate(), and then the text will have what is required.

Just like check_output, Popen.communicate() also returns a byte string which can be easily converted to a normal string using the decode method after passing the compatible and suitable encoding scheme to the decode method.