python“ with”语句的目的是什么?

问题:python“ with”语句的目的是什么?

with今天是第一次遇到Python 语句。我已经使用Python几个月了,甚至不知道它的存在!考虑到它的地位有些晦涩,我认为值得一问:

  1. Python with语句旨在用于什么?
  2. 你用它来做什么?
  3. 我需要了解任何陷阱,还是与其使用相关的常见反模式?有什么try..finally比这更好用的情况with吗?
  4. 为什么没有更广泛地使用它?
  5. 哪些标准库类与之兼容?

I came across the Python with statement for the first time today. I’ve been using Python lightly for several months and didn’t even know of its existence! Given its somewhat obscure status, I thought it would be worth asking:

  1. What is the Python with statement designed to be used for?
  2. What do you use it for?
  3. Are there any gotchas I need to be aware of, or common anti-patterns associated with its use? Any cases where it is better use try..finally than with?
  4. Why isn’t it used more widely?
  5. Which standard library classes are compatible with it?

回答 0

  1. 我相信这已经被我之前的其他用户回答了,所以我仅出于完整性的考虑添加该with语句:该语句通过将常见的准备和清理任务封装在所谓的上下文管理器中来简化异常处理。可以在PEP 343中找到更多详细信息。例如,该open语句本身就是一个上下文管理器,它使您可以打开文件,只要with在使用它的语句上下文中执行该文件,就可以保持打开状态,并在离开上下文后立即将其关闭,无论您是由于异常还是在常规控制流程中离开了它。with因此,可以使用类似于C ++中的RAII模式的方式使用该语句:with语句并在您离开with上下文时释放。

  2. 一些示例是:使用打开文件,使用with open(filename) as fp:获取锁with lock:(在lock的实例threading.Lock)。您还可以使用中的contextmanager装饰器来构造自己的上下文管理器contextlib。例如,当我不得不临时更改当前目录然后返回到原来的位置时,我经常使用它:

    from contextlib import contextmanager
    import os
    
    @contextmanager
    def working_directory(path):
        current_dir = os.getcwd()
        os.chdir(path)
        try:
            yield
        finally:
            os.chdir(current_dir)
    
    with working_directory("data/stuff"):
        # do something within data/stuff
    # here I am back again in the original working directory
    

    这是另一个示例,该示例临时重定向sys.stdinsys.stdout并重定向sys.stderr到其他文件句柄并稍后将其还原:

    from contextlib import contextmanager
    import sys
    
    @contextmanager
    def redirected(**kwds):
        stream_names = ["stdin", "stdout", "stderr"]
        old_streams = {}
        try:
            for sname in stream_names:
                stream = kwds.get(sname, None)
                if stream is not None and stream != getattr(sys, sname):
                    old_streams[sname] = getattr(sys, sname)
                    setattr(sys, sname, stream)
            yield
        finally:
            for sname, stream in old_streams.iteritems():
                setattr(sys, sname, stream)
    
    with redirected(stdout=open("/tmp/log.txt", "w")):
         # these print statements will go to /tmp/log.txt
         print "Test entry 1"
         print "Test entry 2"
    # back to the normal stdout
    print "Back to normal stdout again"
    

    最后,另一个示例创建一个临时文件夹并在离开上下文时清理它:

    from tempfile import mkdtemp
    from shutil import rmtree
    
    @contextmanager
    def temporary_dir(*args, **kwds):
        name = mkdtemp(*args, **kwds)
        try:
            yield name
        finally:
            shutil.rmtree(name)
    
    with temporary_dir() as dirname:
        # do whatever you want
    
  1. I believe this has already been answered by other users before me, so I only add it for the sake of completeness: the with statement simplifies exception handling by encapsulating common preparation and cleanup tasks in so-called context managers. More details can be found in PEP 343. For instance, the open statement is a context manager in itself, which lets you open a file, keep it open as long as the execution is in the context of the with statement where you used it, and close it as soon as you leave the context, no matter whether you have left it because of an exception or during regular control flow. The with statement can thus be used in ways similar to the RAII pattern in C++: some resource is acquired by the with statement and released when you leave the with context.

  2. Some examples are: opening files using with open(filename) as fp:, acquiring locks using with lock: (where lock is an instance of threading.Lock). You can also construct your own context managers using the contextmanager decorator from contextlib. For instance, I often use this when I have to change the current directory temporarily and then return to where I was:

    from contextlib import contextmanager
    import os
    
    @contextmanager
    def working_directory(path):
        current_dir = os.getcwd()
        os.chdir(path)
        try:
            yield
        finally:
            os.chdir(current_dir)
    
    with working_directory("data/stuff"):
        # do something within data/stuff
    # here I am back again in the original working directory
    

    Here’s another example that temporarily redirects sys.stdin, sys.stdout and sys.stderr to some other file handle and restores them later:

    from contextlib import contextmanager
    import sys
    
    @contextmanager
    def redirected(**kwds):
        stream_names = ["stdin", "stdout", "stderr"]
        old_streams = {}
        try:
            for sname in stream_names:
                stream = kwds.get(sname, None)
                if stream is not None and stream != getattr(sys, sname):
                    old_streams[sname] = getattr(sys, sname)
                    setattr(sys, sname, stream)
            yield
        finally:
            for sname, stream in old_streams.iteritems():
                setattr(sys, sname, stream)
    
    with redirected(stdout=open("/tmp/log.txt", "w")):
         # these print statements will go to /tmp/log.txt
         print "Test entry 1"
         print "Test entry 2"
    # back to the normal stdout
    print "Back to normal stdout again"
    

    And finally, another example that creates a temporary folder and cleans it up when leaving the context:

    from tempfile import mkdtemp
    from shutil import rmtree
    
    @contextmanager
    def temporary_dir(*args, **kwds):
        name = mkdtemp(*args, **kwds)
        try:
            yield name
        finally:
            shutil.rmtree(name)
    
    with temporary_dir() as dirname:
        # do whatever you want
    

回答 1

我会建议两个有趣的讲座:

  • PEP 343 “ with”声明
  • Effbot了解Python的“ with”语句

1.with语句用于使用上下文管理器定义的方法来包装块的执行。这允许将常用try...except...finally用法模式封装起来,以方便重用。

2. 您可以执行以下操作:

with open("foo.txt") as foo_file:
    data = foo_file.read()

要么

from contextlib import nested
with nested(A(), B(), C()) as (X, Y, Z):
   do_something()

或(Python 3.1)

with open('data') as input_file, open('result', 'w') as output_file:
   for line in input_file:
     output_file.write(parse(line))

要么

lock = threading.Lock()
with lock:
    # Critical section of code

3. 我在这里看不到任何反模式。
引用Dive进入Python

试试..最终是好的。与更好。

4. 我想这与程序员使用try..catch..finally其他语言的语句的习惯有关。

I would suggest two interesting lectures:

  • PEP 343 The “with” Statement
  • Effbot Understanding Python’s “with” statement

1. The with statement is used to wrap the execution of a block with methods defined by a context manager. This allows common try...except...finally usage patterns to be encapsulated for convenient reuse.

2. You could do something like:

with open("foo.txt") as foo_file:
    data = foo_file.read()

OR

from contextlib import nested
with nested(A(), B(), C()) as (X, Y, Z):
   do_something()

OR (Python 3.1)

with open('data') as input_file, open('result', 'w') as output_file:
   for line in input_file:
     output_file.write(parse(line))

OR

lock = threading.Lock()
with lock:
    # Critical section of code

3. I don’t see any Antipattern here.
Quoting Dive into Python:

try..finally is good. with is better.

4. I guess it’s related to programmers’s habit to use try..catch..finally statement from other languages.


回答 2

Python with语句是Resource Acquisition Is InitializationC ++中常用的惯用语的内置语言支持。旨在允许安全获取和释放操作系统资源。

with语句在作用域/块内创建资源。您可以使用块中的资源编写代码。当该块退出时,无论该块中代码的结果如何(即该块是正常退出还是由于异常而退出),资源都会被干净地释放。

Python库中的许多资源都遵循该with语句所需的协议,因此可以立即使用。但是,任何人都可以通过实施有据可查的协议来制作可用于with语句的资源:PEP 0343

每当您在应用程序中获取必须明确放弃的资源(例如文件,网络连接,锁等)时,都应使用它。

The Python with statement is built-in language support of the Resource Acquisition Is Initialization idiom commonly used in C++. It is intended to allow safe acquisition and release of operating system resources.

The with statement creates resources within a scope/block. You write your code using the resources within the block. When the block exits the resources are cleanly released regardless of the outcome of the code in the block (that is whether the block exits normally or because of an exception).

Many resources in the Python library that obey the protocol required by the with statement and so can used with it out-of-the-box. However anyone can make resources that can be used in a with statement by implementing the well documented protocol: PEP 0343

Use it whenever you acquire resources in your application that must be explicitly relinquished such as files, network connections, locks and the like.


回答 3

再次为了完整性,我将添加最有用的with语句用例。

我进行了大量的科学计算,对于某些活动,我需要使用Decimal库来进行任意精度的计算。我的代码的某些部分需要高精度,而对于大多数其他部分,则需要较低的精度。

我将默认精度设置为一个较低的数字,然后使用with来获取某些部分的更精确答案:

from decimal import localcontext

with localcontext() as ctx:
    ctx.prec = 42   # Perform a high precision calculation
    s = calculate_something()
s = +s  # Round the final result back to the default precision

我在“超几何测试”中经常使用此功能,该测试需要将大量数字除以形式阶乘。在进行基因组比例计算时,必须注意舍入和溢出错误。

Again for completeness I’ll add my most useful use-case for with statements.

I do a lot of scientific computing and for some activities I need the Decimal library for arbitrary precision calculations. Some part of my code I need high precision and for most other parts I need less precision.

I set my default precision to a low number and then use with to get a more precise answer for some sections:

from decimal import localcontext

with localcontext() as ctx:
    ctx.prec = 42   # Perform a high precision calculation
    s = calculate_something()
s = +s  # Round the final result back to the default precision

I use this a lot with the Hypergeometric Test which requires the division of large numbers resulting form factorials. When you do genomic scale calculations you have to be careful of round-off and overflow errors.


回答 4

反模式的一个例子可能是使用with一个循环内时,它会更有效率有with外循环

例如

for row in lines:
    with open("outfile","a") as f:
        f.write(row)

with open("outfile","a") as f:
    for row in lines:
        f.write(row)

第一种方法是为每个文件打开和关闭文件,row而第二种方法是一次打开和关闭文件,这可能会导致性能问题。

An example of an antipattern might be to use the with inside a loop when it would be more efficient to have the with outside the loop

for example

for row in lines:
    with open("outfile","a") as f:
        f.write(row)

vs

with open("outfile","a") as f:
    for row in lines:
        f.write(row)

The first way is opening and closing the file for each row which may cause performance problems compared to the second way with opens and closes the file just once.


回答 5

参见PEP 343-‘with’语句,最后有一个示例部分。

… Python语言的新语句“ with”使排除try / finally语句的标准用法成为可能。

See PEP 343 – The ‘with’ statement, there is an example section at the end.

… new statement “with” to the Python language to make it possible to factor out standard uses of try/finally statements.


回答 6

第1、2和3点被合理地涵盖了:

4:它相对较新,仅在python2.6 +(或使用的python2.5 from __future__ import with_statement)中可用

points 1, 2, and 3 being reasonably well covered:

4: it is relatively new, only available in python2.6+ (or python2.5 using from __future__ import with_statement)


回答 7

with语句适用于所谓的上下文管理器:

http://docs.python.org/release/2.5.2/lib/typecontextmanager.html

这个想法是通过在离开“ with”块之后进行必要的清理来简化异常处理。一些python内置插件已经可以用作上下文管理器。

The with statement works with so-called context managers:

http://docs.python.org/release/2.5.2/lib/typecontextmanager.html

The idea is to simplify exception handling by doing the necessary cleanup after leaving the ‘with’ block. Some of the python built-ins already work as context managers.


回答 8

开箱即用支持的另一个示例是流行的数据库模块的对象,当您习惯于使用内置open()行为方式时,乍一看可能会感到困惑。connection

这些connection对象是上下文管理器,因此可以直接在中使用with-statement,但是在使用上述说明时,请注意:

with-block完成时,或者与异常或不,该连接不会关闭。万一with-block异常结束,则回滚事务,否则提交事务。

这意味着程序员必须注意自己关闭连接,但允许获取连接,并以多个方式使用它with-statements,如psycopg2 docs中所示:

conn = psycopg2.connect(DSN)

with conn:
    with conn.cursor() as curs:
        curs.execute(SQL1)

with conn:
    with conn.cursor() as curs:
        curs.execute(SQL2)

conn.close()

在上面的示例中,您会注意到的cursor对象psycopg2也是上下文管理器。从有关行为的相关文档中:

cursor出口退出时,with-block它将关闭,释放最终与之关联的任何资源。交易状态不受影响。

Another example for out-of-the-box support, and one that might be a bit baffling at first when you are used to the way built-in open() behaves, are connection objects of popular database modules such as:

The connection objects are context managers and as such can be used out-of-the-box in a with-statement, however when using the above note that:

When the with-block is finished, either with an exception or without, the connection is not closed. In case the with-block finishes with an exception, the transaction is rolled back, otherwise the transaction is commited.

This means that the programmer has to take care to close the connection themselves, but allows to acquire a connection, and use it in multiple with-statements, as shown in the psycopg2 docs:

conn = psycopg2.connect(DSN)

with conn:
    with conn.cursor() as curs:
        curs.execute(SQL1)

with conn:
    with conn.cursor() as curs:
        curs.execute(SQL2)

conn.close()

In the example above, you’ll note that the cursor objects of psycopg2 also are context managers. From the relevant documentation on the behavior:

When a cursor exits the with-block it is closed, releasing any resource eventually associated with it. The state of the transaction is not affected.


回答 9

在python中,通常使用“ with ”语句来打开文件,处理文件中存在的数据,以及不调用close()方法而关闭文件。“ with”语句通过提供清理活动使异常处理更加简单。

的一般形式:

with open(“file name”, mode”) as file-var:
    processing statements

注意:无需通过在file-var.close()上调用close()来关闭文件

In python generally “with” statement is used to open a file, process the data present in the file, and also to close the file without calling a close() method. “with” statement makes the exception handling simpler by providing cleanup activities.

General form of with:

with open(“file name”, “mode”) as file-var:
    processing statements

note: no need to close the file by calling close() upon file-var.close()