Since os.popen is being replaced by subprocess.popen, I was wondering how would I convert

os.popen('swfdump /tmp/filename.swf/ -d')

to subprocess.popen()

I tried:

subprocess.Popen("swfdump /tmp/filename.swf -d")
subprocess.Popen("swfdump %s -d" % (filename))  # NOTE: filename is a variable
                                                # containing /tmp/filename.swf

But I guess I’m not properly writing this out. Any help would be appreciated. Thanks

subprocess.Popen takes a list of arguments:

from subprocess import Popen, PIPE

process = Popen(['swfdump', '/tmp/filename.swf', '-d'], stdout=PIPE, stderr=PIPE)
stdout, stderr = process.communicate()

There’s even a section of the documentation devoted to helping users migrate from os.popen to subprocess.

Use sh, it’ll make things a lot easier:

import sh
print sh.swfdump("/tmp/filename.swf", "-d")

Using Subprocess in easiest way!!

import subprocess
cmd = 'pip install numpy'.split()  #replace with your command
subprocess.call(cmd)

Is there a way to specify the running directory of command in Python’s subprocess.Popen()?

For example:

Popen('c:\mytool\tool.exe', workingdir='d:\test\local')

My Python script is located in C:\programs\python

Is is possible to run C:\mytool\tool.exe in the directory D:\test\local?

How do I set the working directory for a sub-process?

subprocess.Popen takes a cwd argument to set the Current Working Directory; you’ll also want to escape your backslashes ('d:\\test\\local'), or use r'd:\test\local' so that the backslashes aren’t interpreted as escape sequences by Python. The way you have it written, the \t part will be translated to a tab.

So, your new line should look like:

subprocess.Popen(r'c:\mytool\tool.exe', cwd=r'd:\test\local')

To use your Python script path as cwd, import os and define cwd using this:

os.path.dirname(os.path.realpath(__file__)) 

I want to call an external program from Python. I have used both Popen() and call() to do that.

What’s the difference between the two?

My specific goal is to run the following command from Python. I am not sure how redirects work.

./my_script.sh > output

I read the documentation and it says that call() is a convenience function or a shortcut function. Do we lose any power by using call() instead of Popen()?

There are two ways to do the redirect. Both apply to either subprocess.Popen or subprocess.call.

1. Set the keyword argument shell = True or executable = /path/to/the/shell and specify the command just as you have it there.

2. Since you’re just redirecting the output to a file, set the keyword argument

   stdout = an_open_writeable_file_object
   

   where the object points to the output file.

subprocess.Popen is more general than subprocess.call.

Popen doesn’t block, allowing you to interact with the process while it’s running, or continue with other things in your Python program. The call to Popen returns a Popen object.

call does block. While it supports all the same arguments as the Popen constructor, so you can still set the process’ output, environmental variables, etc., your script waits for the program to complete, and call returns a code representing the process’ exit status.

returncode = call(*args, **kwargs) 

is basically the same as calling

returncode = Popen(*args, **kwargs).wait()

call is just a convenience function. It’s implementation in CPython is in subprocess.py:

def call(*popenargs, timeout=None, **kwargs):
    """Run command with arguments.  Wait for command to complete or
    timeout, then return the returncode attribute.

    The arguments are the same as for the Popen constructor.  Example:

    retcode = call(["ls", "-l"])
    """
    with Popen(*popenargs, **kwargs) as p:
        try:
            return p.wait(timeout=timeout)
        except:
            p.kill()
            p.wait()
            raise

As you can see, it’s a thin wrapper around Popen.

The other answer is very complete, but here is a rule of thumb:

• call is blocking:

  call('notepad.exe')
  print('hello')  # only executed when notepad is closed
  

• Popen is non-blocking:

  Popen('notepad.exe')
  print('hello')  # immediately executed
  

I believe that running an external command with a slightly modified environment is a very common case. That’s how I tend to do it:

import subprocess, os
my_env = os.environ
my_env["PATH"] = "/usr/sbin:/sbin:" + my_env["PATH"]
subprocess.Popen(my_command, env=my_env)

I’ve got a gut feeling that there’s a better way; does it look alright?

I think os.environ.copy() is better if you don’t intend to modify the os.environ for the current process:

import subprocess, os
my_env = os.environ.copy()
my_env["PATH"] = "/usr/sbin:/sbin:" + my_env["PATH"]
subprocess.Popen(my_command, env=my_env)

That depends on what the issue is. If it’s to clone and modify the environment one solution could be:

subprocess.Popen(my_command, env=dict(os.environ, PATH="path"))

But that somewhat depends on that the replaced variables are valid python identifiers, which they most often are (how often do you run into environment variable names that are not alphanumeric+underscore or variables that starts with a number?).

Otherwise you’ll could write something like:

subprocess.Popen(my_command, env=dict(os.environ, 
                                      **{"Not valid python name":"value"}))

In the very odd case (how often do you use control codes or non-ascii characters in environment variable names?) that the keys of the environment are bytes you can’t (on python3) even use that construct.

As you can see the techniques (especially the first) used here benefits on the keys of the environment normally is valid python identifiers, and also known in advance (at coding time), the second approach has issues. In cases where that isn’t the case you should probably look for another approach.

you might use my_env.get("PATH", '') instead of my_env["PATH"] in case PATH somehow not defined in the original environment, but other than that it looks fine.

With Python 3.5 you could do it this way:

import os
import subprocess

my_env = {**os.environ, 'PATH': '/usr/sbin:/sbin:' + os.environ['PATH']}

subprocess.Popen(my_command, env=my_env)

Here we end up with a copy of os.environ and overridden PATH value.

It was made possible by PEP 448 (Additional Unpacking Generalizations).

Another example. If you have a default environment (i.e. os.environ), and a dict you want to override defaults with, you can express it like this:

my_env = {**os.environ, **dict_with_env_variables}

To temporarily set an environment variable without having to copy the os.envrion object etc, I do this:

process = subprocess.Popen(['env', 'RSYNC_PASSWORD=foobar', 'rsync', \
'rsync://username@foobar.com::'], stdout=subprocess.PIPE)

The env parameter accepts a dictionary. You can simply take os.environ, add a key (your desired variable) (to a copy of the dict if you must) to that and use it as a parameter to Popen.

I know this has been answered for some time, but there are some points that some may want to know about using PYTHONPATH instead of PATH in their environment variable. I have outlined an explanation of running python scripts with cronjobs that deals with the modified environment in a different way (found here). Thought it would be of some good for those who, like me, needed just a bit more than this answer provided.

In certain circumstances you may want to only pass down the environment variables your subprocess needs, but I think you’ve got the right idea in general (that’s how I do it too).