运行shell命令并捕获输出

问题:运行shell命令并捕获输出

我想编写一个函数,它将执行shell命令并以字符串形式返回其输出,无论它是错误还是成功消息。我只想获得与命令行相同的结果。

能做到这一点的代码示例是什么?

例如:

def run_command(cmd):
    # ??????

print run_command('mysqladmin create test -uroot -pmysqladmin12')
# Should output something like:
# mysqladmin: CREATE DATABASE failed; error: 'Can't create database 'test'; database exists'

I want to write a function that will execute a shell command and return its output as a string, no matter, is it an error or success message. I just want to get the same result that I would have gotten with the command line.

What would be a code example that would do such a thing?

For example:

def run_command(cmd):
    # ??????

print run_command('mysqladmin create test -uroot -pmysqladmin12')
# Should output something like:
# mysqladmin: CREATE DATABASE failed; error: 'Can't create database 'test'; database exists'

回答 0

这个问题的答案取决于您使用的Python版本。最简单的方法是使用以下subprocess.check_output功能:

>>> subprocess.check_output(['ls', '-l'])
b'total 0\n-rw-r--r--  1 memyself  staff  0 Mar 14 11:04 files\n'

check_output运行一个仅接受参数作为输入的程序。1它完全返回打印到的结果stdout。如果您需要将输入内容写入stdin,请跳至runPopen部分。如果要执行复杂的Shell命令,请参阅shell=True此答案末尾的注释。

check_output功能适用于仍在广泛使用的几乎所有版本的Python(2.7+)。2但对于较新的版本,不再推荐使用此方法。

现代版本的Python(3.5或更高版本): run

如果您使用的是Python 3.5或更高版本,并且不需要向后兼容,则建议使用run功能。它为该subprocess模块提供了一个非常通用的高级API 。要捕获程序的输出,请将subprocess.PIPE标志传递给stdout关键字参数。然后访问stdout返回CompletedProcess对象的属性:

>>> import subprocess
>>> result = subprocess.run(['ls', '-l'], stdout=subprocess.PIPE)
>>> result.stdout
b'total 0\n-rw-r--r--  1 memyself  staff  0 Mar 14 11:04 files\n'

返回值是一个bytes对象,因此,如果需要正确的字符串,则需要decode它。假设被调用的进程返回一个UTF-8编码的字符串:

>>> result.stdout.decode('utf-8')
'total 0\n-rw-r--r--  1 memyself  staff  0 Mar 14 11:04 files\n'

所有这些都可以压缩为单线:

>>> subprocess.run(['ls', '-l'], stdout=subprocess.PIPE).stdout.decode('utf-8')
'total 0\n-rw-r--r--  1 memyself  staff  0 Mar 14 11:04 files\n'

如果要将输入传递给流程的stdinbytes请将一个对象传递给input关键字参数:

>>> cmd = ['awk', 'length($0) > 5']
>>> input = 'foo\nfoofoo\n'.encode('utf-8')
>>> result = subprocess.run(cmd, stdout=subprocess.PIPE, input=input)
>>> result.stdout.decode('utf-8')
'foofoo\n'

您可以通过传递stderr=subprocess.PIPE(捕获到result.stderr)或stderr=subprocess.STDOUT(捕获到result.stdout常规输出)来捕获错误。如果不关心安全性,您还可以shell=True按照下面的说明通过传递来运行更复杂的Shell命令。

与旧的处理方式相比,这仅增加了一点复杂性。但是我认为值得这样做:现在,您仅需使用该run功能就可以完成几乎所有需要做的事情。

旧版本的Python(2.7-3.4): check_output

如果您使用的是旧版本的Python,或者需要适度的向后兼容性,则可以使用check_output上面简要介绍的函数。自python 2.7开始提供。

subprocess.check_output(*popenargs, **kwargs)  

它采用与Popen(请参见下文)相同的参数,并返回一个包含程序输出的字符串。该答案的开头有一个更详细的用法示例。在Python 3.5及更高版本中,check_output等效于run使用check=Truestdout=PIPE,仅返回stdout属性。

您可以通过stderr=subprocess.STDOUT确保错误信息包含在返回的输出-但在Python中通过一些版本stderr=subprocess.PIPE,以check_output可引起死锁。如果不关心安全性,您还可以shell=True按照下面的说明通过传递来运行更复杂的Shell命令。

如果您需要通过管道stderr传递输入或将输入传递给流程,check_output则将无法完成任务。Popen在这种情况下,请参见下面的示例。

复杂的应用程序和Python的旧版(2.6及以下版本): Popen

如果需要深入的向后兼容性,或者需要比check_output提供的功能更复杂的功能,则必须直接使用Popen对象,这些对象封装了用于子流程的低级API。

所述Popen构造器接受单个命令没有参数,或列表包含指令作为其第一项,其次是任意数量的参数,每个作为列表一个单独的项目。shlex.split可以帮助将字符串解析为格式正确的列表。Popen对象还接受用于进程IO管理和低级配置的许多不同参数

发送输入和捕获输出communicate几乎总是首选方法。如:

output = subprocess.Popen(["mycmd", "myarg"], 
                          stdout=subprocess.PIPE).communicate()[0]

要么

>>> import subprocess
>>> p = subprocess.Popen(['ls', '-a'], stdout=subprocess.PIPE, 
...                                    stderr=subprocess.PIPE)
>>> out, err = p.communicate()
>>> print out
.
..
foo

如果设置stdin=PIPEcommunicate还允许您通过以下方式将数据传递到流程stdin

>>> cmd = ['awk', 'length($0) > 5']
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
...                           stderr=subprocess.PIPE,
...                           stdin=subprocess.PIPE)
>>> out, err = p.communicate('foo\nfoofoo\n')
>>> print out
foofoo

艾伦·霍尔的回答,这表明在某些系统上,你可能需要设置stdoutstderr以及stdin所有PIPE(或DEVNULL)得到communicate工作的。

在极少数情况下,您可能需要复杂的实时输出捕获。Vartec的答案提出了一条前进的道路,但是communicate如果不谨慎使用,则其他方法都容易出现死锁。

与上述所有功能一样,当不考虑安全性时,可以通过传递运行更复杂的Shell命令shell=True

笔记

1.运行shell命令:shell=True参数

通常,对runcheck_outputPopen构造函数的每次调用都会执行一个程序。这意味着没有花哨的bash风格的管道。如果要运行复杂的Shell命令,则可以传递shell=True,这三个功能都支持。

但是,这样做会引起安全问题。如果您要做的不仅仅是轻脚本编写,那么最好单独调用每个进程,并将每个进程的输出作为输入通过以下方式传递给下一个进程:

run(cmd, [stdout=etc...], input=other_output)

要么

Popen(cmd, [stdout=etc...]).communicate(other_output)

直接连接管道的诱惑力很强;抵抗它。否则,您很可能会遇到僵局,或者不得不执行类似此类的骇人行为。

2. Unicode注意事项

check_output在Python 2中返回一个字符串,但bytes在Python 3中返回一个对象。如果您还没有花时间学习unicode,那么值得花一点时间。

The answer to this question depends on the version of Python you’re using. The simplest approach is to use the subprocess.check_output function:

>>> subprocess.check_output(['ls', '-l'])
b'total 0\n-rw-r--r--  1 memyself  staff  0 Mar 14 11:04 files\n'

check_output runs a single program that takes only arguments as input.1 It returns the result exactly as printed to stdout. If you need to write input to stdin, skip ahead to the run or Popen sections. If you want to execute complex shell commands, see the note on shell=True at the end of this answer.

The check_output function works on almost all versions of Python still in wide use (2.7+).2 But for more recent versions, it is no longer the recommended approach.

Modern versions of Python (3.5 or higher): run

If you’re using Python 3.5 or higher, and do not need backwards compatibility, the new run function is recommended. It provides a very general, high-level API for the subprocess module. To capture the output of a program, pass the subprocess.PIPE flag to the stdout keyword argument. Then access the stdout attribute of the returned CompletedProcess object:

>>> import subprocess
>>> result = subprocess.run(['ls', '-l'], stdout=subprocess.PIPE)
>>> result.stdout
b'total 0\n-rw-r--r--  1 memyself  staff  0 Mar 14 11:04 files\n'

The return value is a bytes object, so if you want a proper string, you’ll need to decode it. Assuming the called process returns a UTF-8-encoded string:

>>> result.stdout.decode('utf-8')
'total 0\n-rw-r--r--  1 memyself  staff  0 Mar 14 11:04 files\n'

This can all be compressed to a one-liner:

>>> subprocess.run(['ls', '-l'], stdout=subprocess.PIPE).stdout.decode('utf-8')
'total 0\n-rw-r--r--  1 memyself  staff  0 Mar 14 11:04 files\n'

If you want to pass input to the process’s stdin, pass a bytes object to the input keyword argument:

>>> cmd = ['awk', 'length($0) > 5']
>>> input = 'foo\nfoofoo\n'.encode('utf-8')
>>> result = subprocess.run(cmd, stdout=subprocess.PIPE, input=input)
>>> result.stdout.decode('utf-8')
'foofoo\n'

You can capture errors by passing stderr=subprocess.PIPE (capture to result.stderr) or stderr=subprocess.STDOUT (capture to result.stdout along with regular output). When security is not a concern, you can also run more complex shell commands by passing shell=True as described in the notes below.

This adds just a bit of complexity, compared to the old way of doing things. But I think it’s worth the payoff: now you can do almost anything you need to do with the run function alone.

Older versions of Python (2.7-3.4): check_output

If you are using an older version of Python, or need modest backwards compatibility, you can probably use the check_output function as briefly described above. It has been available since Python 2.7.

subprocess.check_output(*popenargs, **kwargs)  

It takes takes the same arguments as Popen (see below), and returns a string containing the program’s output. The beginning of this answer has a more detailed usage example. In Python 3.5 and greater, check_output is equivalent to executing run with check=True and stdout=PIPE, and returning just the stdout attribute.

You can pass stderr=subprocess.STDOUT to ensure that error messages are included in the returned output — but in some versions of Python passing stderr=subprocess.PIPE to check_output can cause deadlocks. When security is not a concern, you can also run more complex shell commands by passing shell=True as described in the notes below.

If you need to pipe from stderr or pass input to the process, check_output won’t be up to the task. See the Popen examples below in that case.

Complex applications & legacy versions of Python (2.6 and below): Popen

If you need deep backwards compatibility, or if you need more sophisticated functionality than check_output provides, you’ll have to work directly with Popen objects, which encapsulate the low-level API for subprocesses.

The Popen constructor accepts either a single command without arguments, or a list containing a command as its first item, followed by any number of arguments, each as a separate item in the list. shlex.split can help parse strings into appropriately formatted lists. Popen objects also accept a host of different arguments for process IO management and low-level configuration.

To send input and capture output, communicate is almost always the preferred method. As in:

output = subprocess.Popen(["mycmd", "myarg"], 
                          stdout=subprocess.PIPE).communicate()[0]

Or

>>> import subprocess
>>> p = subprocess.Popen(['ls', '-a'], stdout=subprocess.PIPE, 
...                                    stderr=subprocess.PIPE)
>>> out, err = p.communicate()
>>> print out
.
..
foo

If you set stdin=PIPE, communicate also allows you to pass data to the process via stdin:

>>> cmd = ['awk', 'length($0) > 5']
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
...                           stderr=subprocess.PIPE,
...                           stdin=subprocess.PIPE)
>>> out, err = p.communicate('foo\nfoofoo\n')
>>> print out
foofoo

Note Aaron Hall’s answer, which indicates that on some systems, you may need to set stdout, stderr, and stdin all to PIPE (or DEVNULL) to get communicate to work at all.

In some rare cases, you may need complex, real-time output capturing. Vartec‘s answer suggests a way forward, but methods other than communicate are prone to deadlocks if not used carefully.

As with all the above functions, when security is not a concern, you can run more complex shell commands by passing shell=True.

Notes

1. Running shell commands: the shell=True argument

Normally, each call to run, check_output, or the Popen constructor executes a single program. That means no fancy bash-style pipes. If you want to run complex shell commands, you can pass shell=True, which all three functions support.

However, doing so raises security concerns. If you’re doing anything more than light scripting, you might be better off calling each process separately, and passing the output from each as an input to the next, via

run(cmd, [stdout=etc...], input=other_output)

Or

Popen(cmd, [stdout=etc...]).communicate(other_output)

The temptation to directly connect pipes is strong; resist it. Otherwise, you’ll likely see deadlocks or have to do hacky things like this.

2. Unicode considerations

check_output returns a string in Python 2, but a bytes object in Python 3. It’s worth taking a moment to learn about unicode if you haven’t already.


回答 1

这很容易,但仅适用于Unix(包括Cygwin)和Python2.7。

import commands
print commands.getstatusoutput('wc -l file')

它返回带有(return_value,output)的元组。

对于适用于Python2和Python3的解决方案,请改用subprocess模块:

from subprocess import Popen, PIPE
output = Popen(["date"],stdout=PIPE)
response = output.communicate()
print response

This is way easier, but only works on Unix (including Cygwin) and Python2.7.

import commands
print commands.getstatusoutput('wc -l file')

It returns a tuple with the (return_value, output).

For a solution that works in both Python2 and Python3, use the subprocess module instead:

from subprocess import Popen, PIPE
output = Popen(["date"],stdout=PIPE)
response = output.communicate()
print response

回答 2

像这样:

def runProcess(exe):    
    p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    while(True):
        # returns None while subprocess is running
        retcode = p.poll() 
        line = p.stdout.readline()
        yield line
        if retcode is not None:
            break

请注意,我正在将stderr重定向到stdout,它可能并非您想要的,但我也想要错误消息。

此函数逐行产生(通常,您必须等待子进程完成才能获得整体输出)。

对于您的情况,用法是:

for line in runProcess('mysqladmin create test -uroot -pmysqladmin12'.split()):
    print line,

Something like that:

def runProcess(exe):    
    p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    while(True):
        # returns None while subprocess is running
        retcode = p.poll() 
        line = p.stdout.readline()
        yield line
        if retcode is not None:
            break

Note, that I’m redirecting stderr to stdout, it might not be exactly what you want, but I want error messages also.

This function yields line by line as they come (normally you’d have to wait for subprocess to finish to get the output as a whole).

For your case the usage would be:

for line in runProcess('mysqladmin create test -uroot -pmysqladmin12'.split()):
    print line,

回答 3

Vartec的答案无法读取所有行,因此我制作了一个可以读取的版本:

def run_command(command):
    p = subprocess.Popen(command,
                         stdout=subprocess.PIPE,
                         stderr=subprocess.STDOUT)
    return iter(p.stdout.readline, b'')

用法与接受的答案相同:

command = 'mysqladmin create test -uroot -pmysqladmin12'.split()
for line in run_command(command):
    print(line)

Vartec’s answer doesn’t read all lines, so I made a version that did:

def run_command(command):
    p = subprocess.Popen(command,
                         stdout=subprocess.PIPE,
                         stderr=subprocess.STDOUT)
    return iter(p.stdout.readline, b'')

Usage is the same as the accepted answer:

command = 'mysqladmin create test -uroot -pmysqladmin12'.split()
for line in run_command(command):
    print(line)

回答 4

这是一个棘手超级简单的解决方案,可在许多情况下使用:

import os
os.system('sample_cmd > tmp')
print open('tmp', 'r').read()

使用命令的输出创建一个临时文件(这里是tmp),您可以从中读取所需的输出。

注释中的额外说明:如果是一次性作业,则可以删除tmp文件。如果您需要多次执行此操作,则无需删除tmp。

os.remove('tmp')

This is a tricky but super simple solution which works in many situations:

import os
os.system('sample_cmd > tmp')
print open('tmp', 'r').read()

A temporary file(here is tmp) is created with the output of the command and you can read from it your desired output.

Extra note from the comments: You can remove the tmp file in the case of one-time job. If you need to do this several times, there is no need to delete the tmp.

os.remove('tmp')

回答 5

我遇到了同样的问题,但是想出了一种非常简单的方法:

import subprocess
output = subprocess.getoutput("ls -l")
print(output)

希望能帮上忙

注意:此解决方案特定subprocess.getoutput()于Python3,因为在Python2中不起作用

I had the same problem but figured out a very simple way of doing this:

import subprocess
output = subprocess.getoutput("ls -l")
print(output)

Hope it helps out

Note: This solution is Python3 specific as subprocess.getoutput() doesn’t work in Python2


回答 6

您可以使用以下命令来运行任何shell命令。我在ubuntu上使用过它们。

import os
os.popen('your command here').read()

注意:自python 2.6起不推荐使用。现在,您必须使用subprocess.Popen。以下是示例

import subprocess

p = subprocess.Popen("Your command", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()[0]
print p.split("\n")

You can use following commands to run any shell command. I have used them on ubuntu.

import os
os.popen('your command here').read()

Note: This is deprecated since python 2.6. Now you must use subprocess.Popen. Below is the example

import subprocess

p = subprocess.Popen("Your command", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()[0]
print p.split("\n")

回答 7

您的里程可能会有所不同,我尝试使用@senderle在Windows 2.6.5上的Windows中使用Vartec的解决方案,但我遇到了错误,并且没有其他解决方案起作用。我的错误是:WindowsError: [Error 6] The handle is invalid

我发现必须将PIPE分配给每个句柄才能使其返回我期望的输出-以下内容对我有用。

import subprocess

def run_command(cmd):
    """given shell command, returns communication tuple of stdout and stderr"""
    return subprocess.Popen(cmd, 
                            stdout=subprocess.PIPE, 
                            stderr=subprocess.PIPE, 
                            stdin=subprocess.PIPE).communicate()

并像这样调用([0]获取元组的第一个元素stdout):

run_command('tracert 11.1.0.1')[0]

学习更多之后,我相信我需要这些管道参数,因为我正在使用不同句柄的自定义系统上工作,因此必须直接控制所有std。

要停止控制台弹出窗口(在Windows中),请执行以下操作:

def run_command(cmd):
    """given shell command, returns communication tuple of stdout and stderr"""
    # instantiate a startupinfo obj:
    startupinfo = subprocess.STARTUPINFO()
    # set the use show window flag, might make conditional on being in Windows:
    startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
    # pass as the startupinfo keyword argument:
    return subprocess.Popen(cmd,
                            stdout=subprocess.PIPE, 
                            stderr=subprocess.PIPE, 
                            stdin=subprocess.PIPE, 
                            startupinfo=startupinfo).communicate()

run_command('tracert 11.1.0.1')

Your Mileage May Vary, I attempted @senderle’s spin on Vartec’s solution in Windows on Python 2.6.5, but I was getting errors, and no other solutions worked. My error was: WindowsError: [Error 6] The handle is invalid.

I found that I had to assign PIPE to every handle to get it to return the output I expected – the following worked for me.

import subprocess

def run_command(cmd):
    """given shell command, returns communication tuple of stdout and stderr"""
    return subprocess.Popen(cmd, 
                            stdout=subprocess.PIPE, 
                            stderr=subprocess.PIPE, 
                            stdin=subprocess.PIPE).communicate()

and call like this, ([0] gets the first element of the tuple, stdout):

run_command('tracert 11.1.0.1')[0]

After learning more, I believe I need these pipe arguments because I’m working on a custom system that uses different handles, so I had to directly control all the std’s.

To stop console popups (with Windows), do this:

def run_command(cmd):
    """given shell command, returns communication tuple of stdout and stderr"""
    # instantiate a startupinfo obj:
    startupinfo = subprocess.STARTUPINFO()
    # set the use show window flag, might make conditional on being in Windows:
    startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
    # pass as the startupinfo keyword argument:
    return subprocess.Popen(cmd,
                            stdout=subprocess.PIPE, 
                            stderr=subprocess.PIPE, 
                            stdin=subprocess.PIPE, 
                            startupinfo=startupinfo).communicate()

run_command('tracert 11.1.0.1')

回答 8

对于以下问题,我对同一问题的口味略有不同:

  1. 当STDOUT消息在STDOUT缓冲区中累积时(即实时)捕获并返回它们。
    • @vartec通过使用生成器和
      上面的’yield’ 关键字以Python方式解决了这个问题
  2. 打印所有STDOUT行(即使在可以完全读取STDOUT缓冲区之前退出进程
  3. 不要浪费CPU周期以高频率轮询进程
  4. 检查子流程的返回码
  5. 如果得到非零错误返回码,则打印STDERR(与STDOUT分开)。

我结合并调整了先前的答案,以得出以下结论:

import subprocess
from time import sleep

def run_command(command):
    p = subprocess.Popen(command,
                         stdout=subprocess.PIPE,
                         stderr=subprocess.PIPE,
                         shell=True)
    # Read stdout from subprocess until the buffer is empty !
    for line in iter(p.stdout.readline, b''):
        if line: # Don't print blank lines
            yield line
    # This ensures the process has completed, AND sets the 'returncode' attr
    while p.poll() is None:                                                                                                                                        
        sleep(.1) #Don't waste CPU-cycles
    # Empty STDERR buffer
    err = p.stderr.read()
    if p.returncode != 0:
       # The run_command() function is responsible for logging STDERR 
       print("Error: " + str(err))

此代码将与以前的答案相同地执行:

for line in run_command(cmd):
    print(line)

I had a slightly different flavor of the same problem with the following requirements:

  1. Capture and return STDOUT messages as they accumulate in the STDOUT buffer (i.e. in realtime).
    • @vartec solved this Pythonically with his use of generators and the ‘yield’
      keyword above
  2. Print all STDOUT lines (even if process exits before STDOUT buffer can be fully read)
  3. Don’t waste CPU cycles polling the process at high-frequency
  4. Check the return code of the subprocess
  5. Print STDERR (separate from STDOUT) if we get a non-zero error return code.

I’ve combined and tweaked previous answers to come up with the following:

import subprocess
from time import sleep

def run_command(command):
    p = subprocess.Popen(command,
                         stdout=subprocess.PIPE,
                         stderr=subprocess.PIPE,
                         shell=True)
    # Read stdout from subprocess until the buffer is empty !
    for line in iter(p.stdout.readline, b''):
        if line: # Don't print blank lines
            yield line
    # This ensures the process has completed, AND sets the 'returncode' attr
    while p.poll() is None:                                                                                                                                        
        sleep(.1) #Don't waste CPU-cycles
    # Empty STDERR buffer
    err = p.stderr.read()
    if p.returncode != 0:
       # The run_command() function is responsible for logging STDERR 
       print("Error: " + str(err))

This code would be executed the same as previous answers:

for line in run_command(cmd):
    print(line)

回答 9

拆分初始命令 subprocess可能会很棘手且麻烦。

采用 shlex.split()帮助自己。

样例命令

git log -n 5 --since "5 years ago" --until "2 year ago"

编码

from subprocess import check_output
from shlex import split

res = check_output(split('git log -n 5 --since "5 years ago" --until "2 year ago"'))
print(res)
>>> b'commit 7696ab087a163e084d6870bb4e5e4d4198bdc61a\nAuthor: Artur Barseghyan...'

没有shlex.split()代码的话看起来如下

res = check_output([
    'git', 
    'log', 
    '-n', 
    '5', 
    '--since', 
    '5 years ago', 
    '--until', 
    '2 year ago'
])
print(res)
>>> b'commit 7696ab087a163e084d6870bb4e5e4d4198bdc61a\nAuthor: Artur Barseghyan...'

Splitting the initial command for the subprocess might be tricky and cumbersome.

Use shlex.split() to help yourself out.

Sample command

git log -n 5 --since "5 years ago" --until "2 year ago"

The code

from subprocess import check_output
from shlex import split

res = check_output(split('git log -n 5 --since "5 years ago" --until "2 year ago"'))
print(res)
>>> b'commit 7696ab087a163e084d6870bb4e5e4d4198bdc61a\nAuthor: Artur Barseghyan...'

Without shlex.split() the code would look as follows

res = check_output([
    'git', 
    'log', 
    '-n', 
    '5', 
    '--since', 
    '5 years ago', 
    '--until', 
    '2 year ago'
])
print(res)
>>> b'commit 7696ab087a163e084d6870bb4e5e4d4198bdc61a\nAuthor: Artur Barseghyan...'

回答 10

如果您需要在多个文件上运行一个shell命令,那么这对我就成功了。

import os
import subprocess

# Define a function for running commands and capturing stdout line by line
# (Modified from Vartec's solution because it wasn't printing all lines)
def runProcess(exe):    
    p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    return iter(p.stdout.readline, b'')

# Get all filenames in working directory
for filename in os.listdir('./'):
    # This command will be run on each file
    cmd = 'nm ' + filename

    # Run the command and capture the output line by line.
    for line in runProcess(cmd.split()):
        # Eliminate leading and trailing whitespace
        line.strip()
        # Split the output 
        output = line.split()

        # Filter the output and print relevant lines
        if len(output) > 2:
            if ((output[2] == 'set_program_name')):
                print filename
                print line

编辑:刚刚看到了JF Sebastian的建议的Max Persson的解决方案。继续前进,并纳入。

If you need to run a shell command on multiple files, this did the trick for me.

import os
import subprocess

# Define a function for running commands and capturing stdout line by line
# (Modified from Vartec's solution because it wasn't printing all lines)
def runProcess(exe):    
    p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    return iter(p.stdout.readline, b'')

# Get all filenames in working directory
for filename in os.listdir('./'):
    # This command will be run on each file
    cmd = 'nm ' + filename

    # Run the command and capture the output line by line.
    for line in runProcess(cmd.split()):
        # Eliminate leading and trailing whitespace
        line.strip()
        # Split the output 
        output = line.split()

        # Filter the output and print relevant lines
        if len(output) > 2:
            if ((output[2] == 'set_program_name')):
                print filename
                print line

Edit: Just saw Max Persson’s solution with J.F. Sebastian’s suggestion. Went ahead and incorporated that.


回答 11

根据@senderle,如果您像我一样使用python3.6:

def sh(cmd, input=""):
    rst = subprocess.run(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, input=input.encode("utf-8"))
    assert rst.returncode == 0, rst.stderr.decode("utf-8")
    return rst.stdout.decode("utf-8")
sh("ls -a")

就像您在bash中运行命令一样

According to @senderle, if you use python3.6 like me:

def sh(cmd, input=""):
    rst = subprocess.run(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, input=input.encode("utf-8"))
    assert rst.returncode == 0, rst.stderr.decode("utf-8")
    return rst.stdout.decode("utf-8")
sh("ls -a")

Will act exactly like you run the command in bash


回答 12

如果您使用 subprocess python模块,则可以分别处理STDOUT,STDERR和命令的返回代码。您可以看到完整的命令调用程序实现的示例。当然,您可以根据需要扩展它try..except

下面的函数返回STDOUT,STDERR和Return代码,因此您可以在其他脚本中处理它们。

import subprocess

def command_caller(command=None)
    sp = subprocess.Popen(command, stderr=subprocess.PIPE, stdout=subprocess.PIPE, shell=False)
    out, err = sp.communicate()
    if sp.returncode:
        print(
            "Return code: %(ret_code)s Error message: %(err_msg)s"
            % {"ret_code": sp.returncode, "err_msg": err}
            )
    return sp.returncode, out, err

If you use the subprocess python module, you are able to handle the STDOUT, STDERR and return code of command separately. You can see an example for the complete command caller implementation. Of course you can extend it with try..except if you want.

The below function returns the STDOUT, STDERR and Return code so you can handle them in the other script.

import subprocess

def command_caller(command=None)
    sp = subprocess.Popen(command, stderr=subprocess.PIPE, stdout=subprocess.PIPE, shell=False)
    out, err = sp.communicate()
    if sp.returncode:
        print(
            "Return code: %(ret_code)s Error message: %(err_msg)s"
            % {"ret_code": sp.returncode, "err_msg": err}
            )
    return sp.returncode, out, err

回答 13

例如,execute(’ls -ahl’)区分了三种/四种可能的收益和OS平台:

  1. 无输出,但运行成功
  2. 输出空行,运行成功
  3. 运行失败
  4. 输出一些东西,成功运行

功能如下

def execute(cmd, output=True, DEBUG_MODE=False):
"""Executes a bash command.
(cmd, output=True)
output: whether print shell output to screen, only affects screen display, does not affect returned values
return: ...regardless of output=True/False...
        returns shell output as a list with each elment is a line of string (whitespace stripped both sides) from output
        could be 
        [], ie, len()=0 --> no output;    
        [''] --> output empty line;     
        None --> error occured, see below

        if error ocurs, returns None (ie, is None), print out the error message to screen
"""
if not DEBUG_MODE:
    print "Command: " + cmd

    # https://stackoverflow.com/a/40139101/2292993
    def _execute_cmd(cmd):
        if os.name == 'nt' or platform.system() == 'Windows':
            # set stdin, out, err all to PIPE to get results (other than None) after run the Popen() instance
            p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
        else:
            # Use bash; the default is sh
            p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, executable="/bin/bash")

        # the Popen() instance starts running once instantiated (??)
        # additionally, communicate(), or poll() and wait process to terminate
        # communicate() accepts optional input as stdin to the pipe (requires setting stdin=subprocess.PIPE above), return out, err as tuple
        # if communicate(), the results are buffered in memory

        # Read stdout from subprocess until the buffer is empty !
        # if error occurs, the stdout is '', which means the below loop is essentially skipped
        # A prefix of 'b' or 'B' is ignored in Python 2; 
        # it indicates that the literal should become a bytes literal in Python 3 
        # (e.g. when code is automatically converted with 2to3).
        # return iter(p.stdout.readline, b'')
        for line in iter(p.stdout.readline, b''):
            # # Windows has \r\n, Unix has \n, Old mac has \r
            # if line not in ['','\n','\r','\r\n']: # Don't print blank lines
                yield line
        while p.poll() is None:                                                                                                                                        
            sleep(.1) #Don't waste CPU-cycles
        # Empty STDERR buffer
        err = p.stderr.read()
        if p.returncode != 0:
            # responsible for logging STDERR 
            print("Error: " + str(err))
            yield None

    out = []
    for line in _execute_cmd(cmd):
        # error did not occur earlier
        if line is not None:
            # trailing comma to avoid a newline (by print itself) being printed
            if output: print line,
            out.append(line.strip())
        else:
            # error occured earlier
            out = None
    return out
else:
    print "Simulation! The command is " + cmd
    print ""

eg, execute(‘ls -ahl’) differentiated three/four possible returns and OS platforms:

  1. no output, but run successfully
  2. output empty line, run successfully
  3. run failed
  4. output something, run successfully

function below

def execute(cmd, output=True, DEBUG_MODE=False):
"""Executes a bash command.
(cmd, output=True)
output: whether print shell output to screen, only affects screen display, does not affect returned values
return: ...regardless of output=True/False...
        returns shell output as a list with each elment is a line of string (whitespace stripped both sides) from output
        could be 
        [], ie, len()=0 --> no output;    
        [''] --> output empty line;     
        None --> error occured, see below

        if error ocurs, returns None (ie, is None), print out the error message to screen
"""
if not DEBUG_MODE:
    print "Command: " + cmd

    # https://stackoverflow.com/a/40139101/2292993
    def _execute_cmd(cmd):
        if os.name == 'nt' or platform.system() == 'Windows':
            # set stdin, out, err all to PIPE to get results (other than None) after run the Popen() instance
            p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
        else:
            # Use bash; the default is sh
            p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, executable="/bin/bash")

        # the Popen() instance starts running once instantiated (??)
        # additionally, communicate(), or poll() and wait process to terminate
        # communicate() accepts optional input as stdin to the pipe (requires setting stdin=subprocess.PIPE above), return out, err as tuple
        # if communicate(), the results are buffered in memory

        # Read stdout from subprocess until the buffer is empty !
        # if error occurs, the stdout is '', which means the below loop is essentially skipped
        # A prefix of 'b' or 'B' is ignored in Python 2; 
        # it indicates that the literal should become a bytes literal in Python 3 
        # (e.g. when code is automatically converted with 2to3).
        # return iter(p.stdout.readline, b'')
        for line in iter(p.stdout.readline, b''):
            # # Windows has \r\n, Unix has \n, Old mac has \r
            # if line not in ['','\n','\r','\r\n']: # Don't print blank lines
                yield line
        while p.poll() is None:                                                                                                                                        
            sleep(.1) #Don't waste CPU-cycles
        # Empty STDERR buffer
        err = p.stderr.read()
        if p.returncode != 0:
            # responsible for logging STDERR 
            print("Error: " + str(err))
            yield None

    out = []
    for line in _execute_cmd(cmd):
        # error did not occur earlier
        if line is not None:
            # trailing comma to avoid a newline (by print itself) being printed
            if output: print line,
            out.append(line.strip())
        else:
            # error occured earlier
            out = None
    return out
else:
    print "Simulation! The command is " + cmd
    print ""

回答 14

可以将输出重定向到文本文件,然后将其读回。

import subprocess
import os
import tempfile

def execute_to_file(command):
    """
    This function execute the command
    and pass its output to a tempfile then read it back
    It is usefull for process that deploy child process
    """
    temp_file = tempfile.NamedTemporaryFile(delete=False)
    temp_file.close()
    path = temp_file.name
    command = command + " > " + path
    proc = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
    if proc.stderr:
        # if command failed return
        os.unlink(path)
        return
    with open(path, 'r') as f:
        data = f.read()
    os.unlink(path)
    return data

if __name__ == "__main__":
    path = "Somepath"
    command = 'ecls.exe /files ' + path
    print(execute(command))

The output can be redirected to a text file and then read it back.

import subprocess
import os
import tempfile

def execute_to_file(command):
    """
    This function execute the command
    and pass its output to a tempfile then read it back
    It is usefull for process that deploy child process
    """
    temp_file = tempfile.NamedTemporaryFile(delete=False)
    temp_file.close()
    path = temp_file.name
    command = command + " > " + path
    proc = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
    if proc.stderr:
        # if command failed return
        os.unlink(path)
        return
    with open(path, 'r') as f:
        data = f.read()
    os.unlink(path)
    return data

if __name__ == "__main__":
    path = "Somepath"
    command = 'ecls.exe /files ' + path
    print(execute(command))

回答 15

刚刚写了一个小的bash脚本来使用curl做到这一点

https://gist.github.com/harish2704/bfb8abece94893c53ce344548ead8ba5

#!/usr/bin/env bash

# Usage: gdrive_dl.sh <url>

urlBase='https://drive.google.com'
fCookie=tmpcookies

curl="curl -L -b $fCookie -c $fCookie"
confirm(){
    $curl "$1" | grep jfk-button-action | sed -e 's/.*jfk-button-action" href="\(\S*\)".*/\1/' -e 's/\&amp;/\&/g'
}

$curl -O -J "${urlBase}$(confirm $1)"

just wrote a small bash script to do this using curl

https://gist.github.com/harish2704/bfb8abece94893c53ce344548ead8ba5

#!/usr/bin/env bash

# Usage: gdrive_dl.sh <url>

urlBase='https://drive.google.com'
fCookie=tmpcookies

curl="curl -L -b $fCookie -c $fCookie"
confirm(){
    $curl "$1" | grep jfk-button-action | sed -e 's/.*jfk-button-action" href="\(\S*\)".*/\1/' -e 's/\&amp;/\&/g'
}

$curl -O -J "${urlBase}$(confirm $1)"