标签归档:with-statement

在try-except块中使用python“ with”语句

问题:在try-except块中使用python“ with”语句

这是将python“ with”语句与try-except块结合使用的正确方法吗?

try:
    with open("file", "r") as f:
        line = f.readline()
except IOError:
    <whatever>

如果是这样,请考虑使用旧的处理方式:

try:
    f = open("file", "r")
    line = f.readline()
except IOError:
    <whatever>
finally:
    f.close()

这里的“ with”语句的主要好处是我们可以摆脱三行代码吗?对于这个用例,这似乎并不吸引我(尽管我知道“ with”语句还有其他用途)。

编辑:以上两个代码块的功能是否相同?

EDIT2:前几个答案大体上谈及使用“ with”的好处,但是这里似乎没有什么好处。我们已经(或者应该已经)明确地调用f.close()多年了。我想一个好处是草率的编码器将从使用“ with”中受益。

Is this the right way to use the python “with” statement in combination with a try-except block?:

try:
    with open("file", "r") as f:
        line = f.readline()
except IOError:
    <whatever>

If it is, then considering the old way of doing things:

try:
    f = open("file", "r")
    line = f.readline()
except IOError:
    <whatever>
finally:
    f.close()

Is the primary benefit of the “with” statement here that we can get rid of three lines of code? It doesn’t seem that compelling to me for this use case (though I understand that the “with” statement has other uses).

EDIT: Is the functionality of the above two blocks of code identical?

EDIT2: The first few answers talk generally about the benefits of using “with”, but those seem of marginal benefit here. We’ve all been (or should have been) explicitly calling f.close() for years. I suppose one benefit is that sloppy coders will benefit from using “with”.


回答 0

  1. 您提供的两个代码块 相等
  2. 您描述为旧的工作方式的代码有一个严重的错误:如果打开文件失败,您将在finally子句中得到第二个异常, 因为f它没有绑定。

等效的旧样式代码为:

try:
    f = open("file", "r")
    try:
        line = f.readline()
    finally:
        f.close()
except IOError:
    <whatever>

如您所见,该with语句可以减少出错的可能性。在较新版本的Python(2.7,3.1)中,您还可以在一个with语句中组合多个表达式。例如:

with open("input", "r") as inp, open("output", "w") as out:
    out.write(inp.read())

除此之外,我个人认为尽早发现任何异常是一个坏习惯。这不是exceptions的目的。如果可能失败的IO功能是更复杂的操作的一部分,则在大多数情况下,IOError应该中止整个操作,因此应从外部进行处理。使用with语句,您可以消除try...finally内部所有这些语句。

  1. The two code blocks you gave are not equivalent
  2. The code you described as old way of doing things has a serious bug: in case opening the file fails you will get a second exception in the finally clause because f is not bound.

The equivalent old style code would be:

try:
    f = open("file", "r")
    try:
        line = f.readline()
    finally:
        f.close()
except IOError:
    <whatever>

As you can see, the with statement can make things less error prone. In newer versions of Python (2.7, 3.1), you can also combine multiple expressions in one with statement. For example:

with open("input", "r") as inp, open("output", "w") as out:
    out.write(inp.read())

Besides that, I personally regard it as bad habit to catch any exception as early as possible. This is not the purpose of exceptions. If the IO function that can fail is part of a more complicated operation, in most cases the IOError should abort the whole operation and so be handled at an outer level. Using with statements, you can get rid of all these try...finally statements at inner levels.


回答 1

如果finally块的内容由打开的文件对象的属性决定,那么为什么文件对象的实现者不应该是编写finally块的人呢?这就是with语句的好处,不仅仅是在此特定实例中为您节省三行代码。

是的,你结合的方式with,并try-except为几乎做到这一点的唯一方法,因为内造成特殊错误open本身不能内被捕获的语句with块。

If the contents of the finally block are determined by the properties of the file object being opened, why shouldn’t the implementer of the file object be the one to write the finally block? That’s the benefit of the with statement, much more than saving you three lines of code in this particular instance.

And yes, the way you’ve combined with and try-except is pretty much the only way to do it, as exceptional errors caused within the open statement itself can’t be caught within the with block.


回答 2

我认为您对“ with”语句的理解是错误的,因为它只会减少行数。它实际上进行初始化并处理拆除。

在您的情况下,“ with”确实

  • 打开一个文件,
  • 处理其内容,以及
  • 确保关闭它。

这是用于理解“ with”语句的链接:http : //effbot.org/zone/python-with-statement.htm

编辑:是的,您对“ with”的使用是正确的,并且两个代码块的功能相同。关于为什么要使用“ with”的问题?这是因为您从中受益。就像您提到的意外丢失f.close()一样。

I think you got it wrong about “with” statement that it only reduces lines. It actually does initialization and handle teardown.

In your case “with” does

  • open a file,
  • process its contents, and
  • make sure to close it.

Here is link for understanding “with” statement : http://effbot.org/zone/python-with-statement.htm

Edit: Yes your usage of “with” is correct and functionality of both blocks of code is identical. Question about why to use “with” ? it’s because of benefits you get with it. like you mentioned about accidentally missing f.close().


回答 3

以下代码的更多Pythonic方式是:

try:
    f = open("file", "r")
    try:
        line = f.readline()
    finally:
        f.close()
except IOError:
    <whatever>

try:
    f = open("file", "r")
except IOError:
    <whatever>
else:
    f.close()

The more Pythonic way for the following codes is:

try:
    f = open("file", "r")
    try:
        line = f.readline()
    finally:
        f.close()
except IOError:
    <whatever>

try:
    f = open("file", "r")
except IOError:
    <whatever>
else:
    f.close()

如何模拟with语句中使用的打开(使用Python中的Mock框架)?

问题:如何模拟with语句中使用的打开(使用Python中的Mock框架)?

如何使用模拟测试以下代码(使用模拟Michael Foord的Mock框架提供的补丁装饰器和哨兵):

def testme(filepath):
    with open(filepath, 'r') as f:
        return f.read()

How do I test the following code with unittest.mock:

def testme(filepath):
    with open(filepath) as f:
        return f.read()

回答 0

模拟0.7.0中的更改方式已更改,该模拟最终支持模拟python协议方法(魔术方法),尤其是使用MagicMock:

http://www.voidspace.org.uk/python/mock/magicmock.html

作为上下文管理器打开模拟的示例(来自模拟文档中的示例页面):

>>> open_name = '%s.open' % __name__
>>> with patch(open_name, create=True) as mock_open:
...     mock_open.return_value = MagicMock(spec=file)
...
...     with open('/some/path', 'w') as f:
...         f.write('something')
...
<mock.Mock object at 0x...>
>>> file_handle = mock_open.return_value.__enter__.return_value
>>> file_handle.write.assert_called_with('something')

The way to do this has changed in mock 0.7.0 which finally supports mocking the python protocol methods (magic methods), particularly using the MagicMock:

http://www.voidspace.org.uk/python/mock/magicmock.html

An example of mocking open as a context manager (from the examples page in the mock documentation):

>>> open_name = '%s.open' % __name__
>>> with patch(open_name, create=True) as mock_open:
...     mock_open.return_value = MagicMock(spec=file)
...
...     with open('/some/path', 'w') as f:
...         f.write('something')
...
<mock.Mock object at 0x...>
>>> file_handle = mock_open.return_value.__enter__.return_value
>>> file_handle.write.assert_called_with('something')

回答 1

mock_openmock框架的一部分,使用非常简单。patch用作上下文返回用于替换已修补对象的对象:您可以使用它使测试更简单。

Python 3.x

使用builtins代替__builtin__

from unittest.mock import patch, mock_open
with patch("builtins.open", mock_open(read_data="data")) as mock_file:
    assert open("path/to/open").read() == "data"
    mock_file.assert_called_with("path/to/open")

Python 2.7

mock不属于unittest您,您应该修补__builtin__

from mock import patch, mock_open
with patch("__builtin__.open", mock_open(read_data="data")) as mock_file:
    assert open("path/to/open").read() == "data"
    mock_file.assert_called_with("path/to/open")

装饰箱

如果将的结果用作的patch修饰符用作装饰器mock_open(),则new patch可能会有些奇怪。

在这种情况下,最好使用new_callable patch的参数并要记住,每个patch不使用的额外参数都会按照文档中的new_callable描述传递给函数。patch

patch()使用任意关键字参数。这些将在构造时传递给Mock(或new_callable)。

例如,Python 3.x的修饰版本为:

@patch("builtins.open", new_callable=mock_open, read_data="data")
def test_patch(mock_file):
    assert open("path/to/open").read() == "data"
    mock_file.assert_called_with("path/to/open")

请记住,在这种情况下,patch会将模拟对象添加为测试函数的参数。

Python 3

Patch builtins.open and use mock_open, which is part of the mock framework. patch used as a context manager returns the object used to replace the patched one:

from unittest.mock import patch, mock_open
with patch("builtins.open", mock_open(read_data="data")) as mock_file:
    assert open("path/to/open").read() == "data"
    mock_file.assert_called_with("path/to/open")

If you want to use patch as a decorator, using mock_open()‘s result as the new= argument to patch can be a little bit weird. Instead, use patch‘s new_callable= argument and remember that every extra argument that patch doesn’t use will be passed to the new_callable function, as described in the patch documentation:

patch() takes arbitrary keyword arguments. These will be passed to the Mock (or new_callable) on construction.

@patch("builtins.open", new_callable=mock_open, read_data="data")
def test_patch(mock_file):
    assert open("path/to/open").read() == "data"
    mock_file.assert_called_with("path/to/open")

Remember that in this case patch will pass the mocked object as an argument to your test function.

Python 2

You need to patch __builtin__.open instead of builtins.open and mock is not part of unittest, you need to pip install and import it separately:

from mock import patch, mock_open
with patch("__builtin__.open", mock_open(read_data="data")) as mock_file:
    assert open("path/to/open").read() == "data"
    mock_file.assert_called_with("path/to/open")

回答 2

使用最新版本的模拟,您可以使用真正有用的模拟_打开帮助器:

mock_open(模拟=无,读取数据=无)

一个辅助函数创建一个模拟来代替open的使用。它适用于直接调用或用作上下文管理器的open。

模拟参数是要配置的模拟对象。如果没有(默认),则将为您创建一个MagicMock,其API限于标准文件句柄上可用的方法或属性。

read_data是用于返回文件句柄的read方法的字符串。默认情况下,这是一个空字符串。

>>> from mock import mock_open, patch
>>> m = mock_open()
>>> with patch('{}.open'.format(__name__), m, create=True):
...    with open('foo', 'w') as h:
...        h.write('some stuff')

>>> m.assert_called_once_with('foo', 'w')
>>> handle = m()
>>> handle.write.assert_called_once_with('some stuff')

With the latest versions of mock, you can use the really useful mock_open helper:

mock_open(mock=None, read_data=None)

A helper function to create a mock to replace the use of open. It works for open called directly or used as a context manager.

The mock argument is the mock object to configure. If None (the default) then a MagicMock will be created for you, with the API limited to methods or attributes available on standard file handles.

read_data is a string for the read method of the file handle to return. This is an empty string by default.

>>> from mock import mock_open, patch
>>> m = mock_open()
>>> with patch('{}.open'.format(__name__), m, create=True):
...    with open('foo', 'w') as h:
...        h.write('some stuff')

>>> m.assert_called_once_with('foo', 'w')
>>> handle = m()
>>> handle.write.assert_called_once_with('some stuff')

回答 3

要对一个简单文件使用嘲笑read()(打开)(此页面上已经给出的原始嘲笑的片段更适合写入):

my_text = "some text to return when read() is called on the file object"
mocked_open_function = mock.mock_open(read_data=my_text)

with mock.patch("__builtin__.open", mocked_open_function):
    with open("any_string") as f:
        print f.read()

请注意,根据嘲笑文档的doc,这是专门针对的read(),因此不适for line in f用于像这样的常见模式。

使用python 2.6.6 /模拟1.0.1

To use mock_open for a simple file read() (the original mock_open snippet already given on this page is geared more for write):

my_text = "some text to return when read() is called on the file object"
mocked_open_function = mock.mock_open(read_data=my_text)

with mock.patch("__builtin__.open", mocked_open_function):
    with open("any_string") as f:
        print f.read()

Note as per docs for mock_open, this is specifically for read(), so won’t work with common patterns like for line in f, for example.

Uses python 2.6.6 / mock 1.0.1


回答 4

最佳答案很有用,但我对此进行了扩展。

如果要基于传递给此处的参数设置文件对象(fin as f)的值,open()这是一种实现方法:

def save_arg_return_data(*args, **kwargs):
    mm = MagicMock(spec=file)
    mm.__enter__.return_value = do_something_with_data(*args, **kwargs)
    return mm
m = MagicMock()
m.side_effect = save_arg_return_array_of_data

# if your open() call is in the file mymodule.animals 
# use mymodule.animals as name_of_called_file
open_name = '%s.open' % name_of_called_file

with patch(open_name, m, create=True):
    #do testing here

基本上,open()将返回一个对象并with调用__enter__()该对象。

为了正确模拟,我们必须模拟open()返回一个模拟对象。然后,该模拟对象应该模拟对其的__enter__()调用(MagicMock将为我们执行此操作),以返回我们想要的模拟数据/文件对象(因此mm.__enter__.return_value)。通过上面的方法使用2个模拟进行此操作,我们可以捕获传递给参数的参数open()并将其传递给我们的do_something_with_data方法。

我将整个模拟文件作为字符串传递给了open()我,do_something_with_data如下所示:

def do_something_with_data(*args, **kwargs):
    return args[0].split("\n")

这会将字符串转换为列表,因此您可以像处理普通文件一样执行以下操作:

for line in file:
    #do action

The top answer is useful but I expanded on it a bit.

If you want to set the value of your file object (the f in as f) based on the arguments passed to open() here’s one way to do it:

def save_arg_return_data(*args, **kwargs):
    mm = MagicMock(spec=file)
    mm.__enter__.return_value = do_something_with_data(*args, **kwargs)
    return mm
m = MagicMock()
m.side_effect = save_arg_return_array_of_data

# if your open() call is in the file mymodule.animals 
# use mymodule.animals as name_of_called_file
open_name = '%s.open' % name_of_called_file

with patch(open_name, m, create=True):
    #do testing here

Basically, open() will return an object and with will call __enter__() on that object.

To mock properly, we must mock open() to return a mock object. That mock object should then mock the __enter__() call on it (MagicMock will do this for us) to return the mock data/file object we want (hence mm.__enter__.return_value). Doing this with 2 mocks the way above allows us to capture the arguments passed to open() and pass them to our do_something_with_data method.

I passed an entire mock file as a string to open() and my do_something_with_data looked like this:

def do_something_with_data(*args, **kwargs):
    return args[0].split("\n")

This transforms the string into a list so you can do the following as you would with a normal file:

for line in file:
    #do action

回答 5

我可能对游戏有些迟了,但是当我调用open另一个模块而不必创建新文件时,这对我有用。

test.py

import unittest
from mock import Mock, patch, mock_open
from MyObj import MyObj

class TestObj(unittest.TestCase):
    open_ = mock_open()
    with patch.object(__builtin__, "open", open_):
        ref = MyObj()
        ref.save("myfile.txt")
    assert open_.call_args_list == [call("myfile.txt", "wb")]

MyObj.py

class MyObj(object):
    def save(self, filename):
        with open(filename, "wb") as f:
            f.write("sample text")

通过open__builtin__模块内部的功能修补到my mock_open(),可以模拟写入文件而无需创建文件。

注意:如果您正在使用使用cython的模块,或者您的程序以任何方式依赖cython,则需要通过在文件顶部包含来导入cython的__builtin__模块import __builtin____builtin__如果您使用cython,则将无法模拟通用。

I might be a bit late to the game, but this worked for me when calling open in another module without having to create a new file.

test.py

import unittest
from mock import Mock, patch, mock_open
from MyObj import MyObj

class TestObj(unittest.TestCase):
    open_ = mock_open()
    with patch.object(__builtin__, "open", open_):
        ref = MyObj()
        ref.save("myfile.txt")
    assert open_.call_args_list == [call("myfile.txt", "wb")]

MyObj.py

class MyObj(object):
    def save(self, filename):
        with open(filename, "wb") as f:
            f.write("sample text")

By patching the open function inside the __builtin__ module to my mock_open(), I can mock writing to a file without creating one.

Note: If you are using a module that uses cython, or your program depends on cython in any way, you will need to import cython’s __builtin__ module by including import __builtin__ at the top of your file. You will not be able to mock the universal __builtin__ if you are using cython.


回答 6

要使用unittest修补内置的open()函数:

这适用于读取json配置的补丁。

class ObjectUnderTest:
    def __init__(self, filename: str):
        with open(filename, 'r') as f:
            dict_content = json.load(f)

模拟对象是open()函数返回的io.TextIOWrapper对象。

@patch("<src.where.object.is.used>.open",
        return_value=io.TextIOWrapper(io.BufferedReader(io.BytesIO(b'{"test_key": "test_value"}'))))
    def test_object_function_under_test(self, mocker):

To patch the built-in open() function with unittest:

This worked for a patch to read a json config.

class ObjectUnderTest:
    def __init__(self, filename: str):
        with open(filename, 'r') as f:
            dict_content = json.load(f)

The mocked object is the io.TextIOWrapper object returned by the open() function

@patch("<src.where.object.is.used>.open",
        return_value=io.TextIOWrapper(io.BufferedReader(io.BytesIO(b'{"test_key": "test_value"}'))))
    def test_object_function_under_test(self, mocker):

回答 7

如果您不再需要任何文件,则可以装饰测试方法:

@patch('builtins.open', mock_open(read_data="data"))
def test_testme():
    result = testeme()
    assert result == "data"

If you don’t need any file further, you can decorate the test method:

@patch('builtins.open', mock_open(read_data="data"))
def test_testme():
    result = testeme()
    assert result == "data"

在Python中,如果我在“ with”块内返回文件,文件是否仍会关闭?

问题:在Python中,如果我在“ with”块内返回文件,文件是否仍会关闭?

考虑以下:

with open(path, mode) as f:
    return [line for line in f if condition]

文件将被正确关闭,还是使用return某种方式绕过上下文管理器

Consider the following:

with open(path, mode) as f:
    return [line for line in f if condition]

Will the file be closed properly, or does using return somehow bypass the context manager?


回答 0

是的,它的作用就像一个finally块接一个try块,也就是说,它总是执行(除非python进程以异常的方式终止)。

PEP-343的示例之一也提到了该with语句,它是该语句的规范:

with locked(myLock):
    # Code here executes with myLock held.  The lock is
    # guaranteed to be released when the block is left (even
    # if via return or by an uncaught exception).

但是,值得一提的是,如果open()不将整个with块放入try..except通常不是您想要的块中,就无法轻松捕获调用引发的异常。

Yes, it acts like the finally block after a try block, i.e. it always executes (unless the python process terminates in an unusual way of course).

It is also mentioned in one of the examples of PEP-343 which is the specification for the with statement:

with locked(myLock):
    # Code here executes with myLock held.  The lock is
    # guaranteed to be released when the block is left (even
    # if via return or by an uncaught exception).

Something worth mentioning is however, that you cannot easily catch exceptions thrown by the open() call without putting the whole with block inside a try..except block which is usually not what one wants.


回答 1

是。

def example(path, mode):
    with open(path, mode) as f:
        return [line for line in f if condition]

..几乎等同于:

def example(path, mode):
    f = open(path, mode)

    try:
        return [line for line in f if condition]
    finally:
        f.close()

更准确地说, __exit__总是在退出该块时调用上下文管理器中方法(无论异常,返回等如何)。文件对象的__exit__方法只是调用f.close()(例如,在CPython中

Yes.

def example(path, mode):
    with open(path, mode) as f:
        return [line for line in f if condition]

..is pretty much equivalent to:

def example(path, mode):
    f = open(path, mode)

    try:
        return [line for line in f if condition]
    finally:
        f.close()

More accurately, the __exit__ method in a context manager is always called when exiting the block (regardless of exceptions, returns etc). The file object’s __exit__ method just calls f.close() (e.g here in CPython)


回答 2

是。更一般地,如果在上下文内部发生__exit__了“ with”语句上下文管理器,该方法的确会被调用return。可以使用以下方法进行测试:

class MyResource:
    def __enter__(self):
        print('Entering context.')
        return self

    def __exit__(self, *exc):
        print('EXITING context.')

def fun():
    with MyResource():
        print('Returning inside with-statement.')
        return
    print('Returning outside with-statement.')

fun()

输出为:

Entering context.
Returning inside with-statement.
EXITING context.

上面的输出确认 __exit__尽管是早期的调用return。这样,上下文管理器不会被绕过。

Yes. More generally, the __exit__ method of a With Statement Context Manager will indeed be called in the event of a return from inside the context. This can be tested with the following:

class MyResource:
    def __enter__(self):
        print('Entering context.')
        return self

    def __exit__(self, *exc):
        print('EXITING context.')

def fun():
    with MyResource():
        print('Returning inside with-statement.')
        return
    print('Returning outside with-statement.')

fun()

The output is:

Entering context.
Returning inside with-statement.
EXITING context.

The output above confirms that __exit__ was called despite the early return. As such, the context manager is not bypassed.


回答 3

是的,但是在其他情况下可能会有一些副作用,因为它可能会在__exit__块中执行某些操作(例如刷新缓冲区)

import gzip
import io

def test(data):
    out = io.BytesIO()
    with gzip.GzipFile(fileobj=out, mode="wb") as f:
        f.write(data)
        return out.getvalue()

def test1(data):
    out = io.BytesIO()
    with gzip.GzipFile(fileobj=out, mode="wb") as f:
        f.write(data)
    return out.getvalue()

print(test(b"test"), test1(b"test"))

# b'\x1f\x8b\x08\x00\x95\x1b\xb3[\x02\xff' b'\x1f\x8b\x08\x00\x95\x1b\xb3[\x02\xff+I-.\x01\x00\x0c~\x7f\xd8\x04\x00\x00\x00'

Yes, but there may be some side effect in other cases, because it may should do something (like flushing buffer) in __exit__ block

import gzip
import io

def test(data):
    out = io.BytesIO()
    with gzip.GzipFile(fileobj=out, mode="wb") as f:
        f.write(data)
        return out.getvalue()

def test1(data):
    out = io.BytesIO()
    with gzip.GzipFile(fileobj=out, mode="wb") as f:
        f.write(data)
    return out.getvalue()

print(test(b"test"), test1(b"test"))

# b'\x1f\x8b\x08\x00\x95\x1b\xb3[\x02\xff' b'\x1f\x8b\x08\x00\x95\x1b\xb3[\x02\xff+I-.\x01\x00\x0c~\x7f\xd8\x04\x00\x00\x00'

解释Python的’__enter__’和’__exit__’

问题:解释Python的’__enter__’和’__exit__’

我在某人的代码中看到了这一点。这是什么意思?

    def __enter__(self):
        return self

    def __exit__(self, type, value, tb):
        self.stream.close()

from __future__ import with_statement#for python2.5 

class a(object):
    def __enter__(self):
        print 'sss'
        return 'sss111'
    def __exit__(self ,type, value, traceback):
        print 'ok'
        return False

with a() as s:
    print s


print s

I saw this in someone’s code. What does it mean?

    def __enter__(self):
        return self

    def __exit__(self, type, value, tb):
        self.stream.close()

from __future__ import with_statement#for python2.5 

class a(object):
    def __enter__(self):
        print 'sss'
        return 'sss111'
    def __exit__(self ,type, value, traceback):
        print 'ok'
        return False

with a() as s:
    print s


print s

回答 0

使用这些魔术方法(__enter____exit__)使您可以实现可轻松用于该with语句的对象。

这个想法是,它使得构建需要执行一些“清除”代码的代码(将其视为一个try-finally块)变得容易。这里有更多的解释

一个有用的例子是数据库连接对象(一旦对应的“ with”语句超出范围,它就会自动关闭连接):

class DatabaseConnection(object):

    def __enter__(self):
        # make a database connection and return it
        ...
        return self.dbconn

    def __exit__(self, exc_type, exc_val, exc_tb):
        # make sure the dbconnection gets closed
        self.dbconn.close()
        ...

如上所述,将此对象与with语句一起使用(from __future__ import with_statement如果您使用的是Python 2.5,则可能需要在文件顶部执行此操作)。

with DatabaseConnection() as mydbconn:
    # do stuff

PEP343-‘with’语句’也有不错的写法。

Using these magic methods (__enter__, __exit__) allows you to implement objects which can be used easily with the with statement.

The idea is that it makes it easy to build code which needs some ‘cleandown’ code executed (think of it as a try-finally block). Some more explanation here.

A useful example could be a database connection object (which then automagically closes the connection once the corresponding ‘with’-statement goes out of scope):

class DatabaseConnection(object):

    def __enter__(self):
        # make a database connection and return it
        ...
        return self.dbconn

    def __exit__(self, exc_type, exc_val, exc_tb):
        # make sure the dbconnection gets closed
        self.dbconn.close()
        ...

As explained above, use this object with the with statement (you may need to do from __future__ import with_statement at the top of the file if you’re on Python 2.5).

with DatabaseConnection() as mydbconn:
    # do stuff

PEP343 — The ‘with’ statement’ has a nice writeup as well.


回答 1

如果您知道上下文管理器是什么,那么您就不需要了解__enter__和使用__exit__魔术方法。让我们看一个非常简单的例子。

在此示例中,我将借助open函数打开myfile.txt。在尝试/终于块确保即使发生意外的异常myfile.txt的将被关闭。

fp=open(r"C:\Users\SharpEl\Desktop\myfile.txt")
try:
    for line in fp:
        print(line)
finally:
    fp.close()

现在我用with语句打开相同的文件:

with open(r"C:\Users\SharpEl\Desktop\myfile.txt") as fp:
    for line in fp:
        print(line) 

如果您查看代码,则我没有关闭文件,也没有try / finally块。因为with语句会自动关闭 myfile.txt。您甚至可以通过调用print(fp.closed)attribute 来检查它-返回True

这是因为open函数返回的文件对象(在我的示例中为fp)具有两个内置方法__enter____exit__。它也称为上下文管理器。__enter__方法在with块的开头__exit__ 被调用,方法在结尾被调用。注意:with语句仅适用于支持上下文管理协议的对象,即它们具有__enter____exit__方法。实现这两种方法的类称为上下文管理器类。

现在让我们定义自己的上下文管理器类。

 class Log:
    def __init__(self,filename):
        self.filename=filename
        self.fp=None    
    def logging(self,text):
        self.fp.write(text+'\n')
    def __enter__(self):
        print("__enter__")
        self.fp=open(self.filename,"a+")
        return self    
    def __exit__(self, exc_type, exc_val, exc_tb):
        print("__exit__")
        self.fp.close()

with Log(r"C:\Users\SharpEl\Desktop\myfile.txt") as logfile:
    print("Main")
    logfile.logging("Test1")
    logfile.logging("Test2")

我希望您现在对魔术方法__enter__和两者都有基本的了解__exit__

If you know what context managers are then you need nothing more to understand __enter__ and __exit__ magic methods. Lets see a very simple example.

In this example I am opening myfile.txt with help of open function. The try/finally block ensures that even if an unexpected exception occurs myfile.txt will be closed.

fp=open(r"C:\Users\SharpEl\Desktop\myfile.txt")
try:
    for line in fp:
        print(line)
finally:
    fp.close()

Now I am opening same file with with statement:

with open(r"C:\Users\SharpEl\Desktop\myfile.txt") as fp:
    for line in fp:
        print(line) 

If you look at the code, I didn’t close the file & there is no try/finally block. Because with statement automatically closes myfile.txt . You can even check it by calling print(fp.closed) attribute — which returns True.

This is because the file objects (fp in my example) returned by open function has two built-in methods __enter__ and __exit__. It is also known as context manager. __enter__ method is called at the start of with block and __exit__ method is called at the end. Note: with statement only works with objects that support the context mamangement protocol i.e. they have __enter__ and __exit__ methods. A class which implement both methods is known as context manager class.

Now lets define our own context manager class.

 class Log:
    def __init__(self,filename):
        self.filename=filename
        self.fp=None    
    def logging(self,text):
        self.fp.write(text+'\n')
    def __enter__(self):
        print("__enter__")
        self.fp=open(self.filename,"a+")
        return self    
    def __exit__(self, exc_type, exc_val, exc_tb):
        print("__exit__")
        self.fp.close()

with Log(r"C:\Users\SharpEl\Desktop\myfile.txt") as logfile:
    print("Main")
    logfile.logging("Test1")
    logfile.logging("Test2")

I hope now you have basic understanding of both __enter__ and __exit__ magic methods.


回答 2

我发现通过Googling 定位python文档__enter____exit__方法非常困难,因此,这里的链接可以帮助其他人:

https://docs.python.org/2/reference/datamodel.html#with-statement-context-managers
https://docs.python.org/3/reference/datamodel.html#with-statement-context-managers
(两个版本的细节相同)

object.__enter__(self)
输入与此对象相关的运行时上下文。该with语句会将此方法的返回值绑定到该语句的as子句中指定的目标(如果有)。

object.__exit__(self, exc_type, exc_value, traceback)
退出与此对象相关的运行时上下文。参数描述导致上下文退出的异常。如果上下文无exceptions地退出,则所有三个参数均为None

如果提供了异常,并且该方法希望抑制该异常(即,防止其传播),则它应返回一个真值。否则,将在退出此方法后正常处理异常。

注意,__exit__()方法不应该引发传入的异常。这是调用方法的责任。

我希望对__exit__方法参数进行清楚的描述。这是缺少的,但是我们可以推断出它们。

大概exc_type是异常的类。

它说您不应该重新引发传入的异常。这向我们表明,其中一个参数可能是实际的Exception实例……或者您应该自己从类型和值中实例化它?

我们可以通过看这篇文章来回答:http :
//effbot.org/zone/python-with-statement.htm

例如,以下__exit__方法可吞噬所有TypeError,但允许所有其他异常通过:

def __exit__(self, type, value, traceback):
    return isinstance(value, TypeError)

…显然value是Exception实例。

大概traceback是一个Python 回溯对象。

I found it strangely difficult to locate the python docs for __enter__ and __exit__ methods by Googling, so to help others here is the link:

https://docs.python.org/2/reference/datamodel.html#with-statement-context-managers
https://docs.python.org/3/reference/datamodel.html#with-statement-context-managers
(detail is the same for both versions)

object.__enter__(self)
Enter the runtime context related to this object. The with statement will bind this method’s return value to the target(s) specified in the as clause of the statement, if any.

object.__exit__(self, exc_type, exc_value, traceback)
Exit the runtime context related to this object. The parameters describe the exception that caused the context to be exited. If the context was exited without an exception, all three arguments will be None.

If an exception is supplied, and the method wishes to suppress the exception (i.e., prevent it from being propagated), it should return a true value. Otherwise, the exception will be processed normally upon exit from this method.

Note that __exit__() methods should not reraise the passed-in exception; this is the caller’s responsibility.

I was hoping for a clear description of the __exit__ method arguments. This is lacking but we can deduce them…

Presumably exc_type is the class of the exception.

It says you should not re-raise the passed-in exception. This suggests to us that one of the arguments might be an actual Exception instance …or maybe you’re supposed to instantiate it yourself from the type and value?

We can answer by looking at this article:
http://effbot.org/zone/python-with-statement.htm

For example, the following __exit__ method swallows any TypeError, but lets all other exceptions through:

def __exit__(self, type, value, traceback):
    return isinstance(value, TypeError)

…so clearly value is an Exception instance.

And presumably traceback is a Python traceback object.


回答 3

除了上述答案以例证调用顺序外,一个简单的运行示例

class myclass:
    def __init__(self):
        print("__init__")

    def __enter__(self): 
        print("__enter__")

    def __exit__(self, type, value, traceback):
        print("__exit__")

    def __del__(self):
        print("__del__")

with myclass(): 
    print("body")

产生输出:

__init__
__enter__
body
__exit__
__del__

提醒:使用语法时,在上述情况下with myclass() as mc,变量mc获取由返回的值!对于此类用途,需要定义返回值,例如:__enter__()None

def __enter__(self): 
    print('__enter__')
    return self

In addition to the above answers to exemplify invocation order, a simple run example

class myclass:
    def __init__(self):
        print("__init__")

    def __enter__(self): 
        print("__enter__")

    def __exit__(self, type, value, traceback):
        print("__exit__")

    def __del__(self):
        print("__del__")

with myclass(): 
    print("body")

Produces the output:

__init__
__enter__
body
__exit__
__del__

A reminder: when using the syntax with myclass() as mc, variable mc gets the value returned by __enter__(), in the above case None! For such use, need to define return value, such as:

def __enter__(self): 
    print('__enter__')
    return self

回答 4

尝试添加我的答案(我的学习思想):

__enter__[__exit__]两者都是在“ with语句 ” 主体(PEP 343)的入口处和出口处被调用的方法,并且两者的实现都称为上下文管理器。

with语句旨在隐藏try finally子句的流控制,并使代码难以理解。

with语句的语法为:

with EXPR as VAR:
    BLOCK

转换为(如PEP 343中所述):

mgr = (EXPR)
exit = type(mgr).__exit__  # Not calling it yet
value = type(mgr).__enter__(mgr)
exc = True
try:
    try:
        VAR = value  # Only if "as VAR" is present
        BLOCK
    except:
        # The exceptional case is handled here
        exc = False
        if not exit(mgr, *sys.exc_info()):
            raise
        # The exception is swallowed if exit() returns true
finally:
    # The normal and non-local-goto cases are handled here
    if exc:
        exit(mgr, None, None, None)

尝试一些代码:

>>> import logging
>>> import socket
>>> import sys

#server socket on another terminal / python interpreter
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> s.listen(5)
>>> s.bind((socket.gethostname(), 999))
>>> while True:
>>>    (clientsocket, addr) = s.accept()
>>>    print('get connection from %r' % addr[0])
>>>    msg = clientsocket.recv(1024)
>>>    print('received %r' % msg)
>>>    clientsocket.send(b'connected')
>>>    continue

#the client side
>>> class MyConnectionManager:
>>>     def __init__(self, sock, addrs):
>>>         logging.basicConfig(level=logging.DEBUG, format='%(asctime)s \
>>>         : %(levelname)s --> %(message)s')
>>>         logging.info('Initiating My connection')
>>>         self.sock = sock
>>>         self.addrs = addrs
>>>     def __enter__(self):
>>>         try:
>>>             self.sock.connect(addrs)
>>>             logging.info('connection success')
>>>             return self.sock
>>>         except:
>>>             logging.warning('Connection refused')
>>>             raise
>>>     def __exit__(self, type, value, tb):
>>>             logging.info('CM suppress exception')
>>>             return False
>>> addrs = (socket.gethostname())
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> with MyConnectionManager(s, addrs) as CM:
>>>     try:
>>>         CM.send(b'establishing connection')
>>>         msg = CM.recv(1024)
>>>         print(msg)
>>>     except:
>>>         raise
#will result (client side) :
2018-12-18 14:44:05,863         : INFO --> Initiating My connection
2018-12-18 14:44:05,863         : INFO --> connection success
b'connected'
2018-12-18 14:44:05,864         : INFO --> CM suppress exception

#result of server side
get connection from '127.0.0.1'
received b'establishing connection'

然后手动尝试(遵循翻译语法):

>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) #make new socket object
>>> mgr = MyConnection(s, addrs)
2018-12-18 14:53:19,331         : INFO --> Initiating My connection
>>> ext = mgr.__exit__
>>> value = mgr.__enter__()
2018-12-18 14:55:55,491         : INFO --> connection success
>>> exc = True
>>> try:
>>>     try:
>>>         VAR = value
>>>         VAR.send(b'establishing connection')
>>>         msg = VAR.recv(1024)
>>>         print(msg)
>>>     except:
>>>         exc = False
>>>         if not ext(*sys.exc_info()):
>>>             raise
>>> finally:
>>>     if exc:
>>>         ext(None, None, None)
#the result:
b'connected'
2018-12-18 15:01:54,208         : INFO --> CM suppress exception

服务器端的结果与以前相同

对不起,我的英语不好,解释不清楚,谢谢。

try adding my answers (my thought of learning) :

__enter__ and [__exit__] both are methods that are invoked on entry to and exit from the body of “the with statement” (PEP 343) and implementation of both is called context manager.

the with statement is intend to hiding flow control of try finally clause and make the code inscrutable.

the syntax of the with statement is :

with EXPR as VAR:
    BLOCK

which translate to (as mention in PEP 343) :

mgr = (EXPR)
exit = type(mgr).__exit__  # Not calling it yet
value = type(mgr).__enter__(mgr)
exc = True
try:
    try:
        VAR = value  # Only if "as VAR" is present
        BLOCK
    except:
        # The exceptional case is handled here
        exc = False
        if not exit(mgr, *sys.exc_info()):
            raise
        # The exception is swallowed if exit() returns true
finally:
    # The normal and non-local-goto cases are handled here
    if exc:
        exit(mgr, None, None, None)

try some code:

>>> import logging
>>> import socket
>>> import sys

#server socket on another terminal / python interpreter
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> s.listen(5)
>>> s.bind((socket.gethostname(), 999))
>>> while True:
>>>    (clientsocket, addr) = s.accept()
>>>    print('get connection from %r' % addr[0])
>>>    msg = clientsocket.recv(1024)
>>>    print('received %r' % msg)
>>>    clientsocket.send(b'connected')
>>>    continue

#the client side
>>> class MyConnectionManager:
>>>     def __init__(self, sock, addrs):
>>>         logging.basicConfig(level=logging.DEBUG, format='%(asctime)s \
>>>         : %(levelname)s --> %(message)s')
>>>         logging.info('Initiating My connection')
>>>         self.sock = sock
>>>         self.addrs = addrs
>>>     def __enter__(self):
>>>         try:
>>>             self.sock.connect(addrs)
>>>             logging.info('connection success')
>>>             return self.sock
>>>         except:
>>>             logging.warning('Connection refused')
>>>             raise
>>>     def __exit__(self, type, value, tb):
>>>             logging.info('CM suppress exception')
>>>             return False
>>> addrs = (socket.gethostname())
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> with MyConnectionManager(s, addrs) as CM:
>>>     try:
>>>         CM.send(b'establishing connection')
>>>         msg = CM.recv(1024)
>>>         print(msg)
>>>     except:
>>>         raise
#will result (client side) :
2018-12-18 14:44:05,863         : INFO --> Initiating My connection
2018-12-18 14:44:05,863         : INFO --> connection success
b'connected'
2018-12-18 14:44:05,864         : INFO --> CM suppress exception

#result of server side
get connection from '127.0.0.1'
received b'establishing connection'

and now try manually (following translate syntax):

>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) #make new socket object
>>> mgr = MyConnection(s, addrs)
2018-12-18 14:53:19,331         : INFO --> Initiating My connection
>>> ext = mgr.__exit__
>>> value = mgr.__enter__()
2018-12-18 14:55:55,491         : INFO --> connection success
>>> exc = True
>>> try:
>>>     try:
>>>         VAR = value
>>>         VAR.send(b'establishing connection')
>>>         msg = VAR.recv(1024)
>>>         print(msg)
>>>     except:
>>>         exc = False
>>>         if not ext(*sys.exc_info()):
>>>             raise
>>> finally:
>>>     if exc:
>>>         ext(None, None, None)
#the result:
b'connected'
2018-12-18 15:01:54,208         : INFO --> CM suppress exception

the result of the server side same as before

sorry for my bad english and my unclear explanations, thank you….


回答 5

这称为上下文管理器,我只想补充一下,其他编程语言也存在类似的方法。比较它们可能有助于理解python中的上下文管理器。基本上,当我们处理一些需要初始化的资源(文件,网络,数据库),并且在某些时候需要拆除(配置)时,使用上下文管理器。在Java 7及更高版本中,我们具有以下形式的自动资源管理:

//Java code
try (Session session = new Session())
{
  // do stuff
}

请注意,会话需要实现AutoClosable其(许多)子接口之一。

C#中,我们使用using语句来管理资源,其形式为:

//C# code
using(Session session = new Session())
{
  ... do stuff.
}

Session应该在其中实施IDisposable

python中,我们使用的类应实现__enter____exit__。因此它采取以下形式:

#Python code
with Session() as session:
    #do stuff

正如其他人指出的那样,您始终可以在所有语言中使用try / finally语句来实现相同的机制。这只是语法糖。

This is called context manager and I just want to add that similar approaches exist for other programming languages. Comparing them could be helpful in understanding the context manager in python. Basically, a context manager is used when we are dealing with some resources (file, network, database) that need to be initialized and at some point, tear downed (disposed). In Java 7 and above we have automatic resource management that takes the form of:

//Java code
try (Session session = new Session())
{
  // do stuff
}

Note that Session needs to implement AutoClosable or one of its (many) sub-interfaces.

In C#, we have using statements for managing resources that takes the form of:

//C# code
using(Session session = new Session())
{
  ... do stuff.
}

In which Session should implement IDisposable.

In python, the class that we use should implement __enter__ and __exit__. So it takes the form of:

#Python code
with Session() as session:
    #do stuff

And as others pointed out, you can always use try/finally statement in all the languages to implement the same mechanism. This is just syntactic sugar.


python“ with”语句的目的是什么?

问题:python“ with”语句的目的是什么?

with今天是第一次遇到Python 语句。我已经使用Python几个月了,甚至不知道它的存在!考虑到它的地位有些晦涩,我认为值得一问:

  1. Python with语句旨在用于什么?
  2. 你用它来做什么?
  3. 我需要了解任何陷阱,还是与其使用相关的常见反模式?有什么try..finally比这更好用的情况with吗?
  4. 为什么没有更广泛地使用它?
  5. 哪些标准库类与之兼容?

I came across the Python with statement for the first time today. I’ve been using Python lightly for several months and didn’t even know of its existence! Given its somewhat obscure status, I thought it would be worth asking:

  1. What is the Python with statement designed to be used for?
  2. What do you use it for?
  3. Are there any gotchas I need to be aware of, or common anti-patterns associated with its use? Any cases where it is better use try..finally than with?
  4. Why isn’t it used more widely?
  5. Which standard library classes are compatible with it?

回答 0

  1. 我相信这已经被我之前的其他用户回答了,所以我仅出于完整性的考虑添加该with语句:该语句通过将常见的准备和清理任务封装在所谓的上下文管理器中来简化异常处理。可以在PEP 343中找到更多详细信息。例如,该open语句本身就是一个上下文管理器,它使您可以打开文件,只要with在使用它的语句上下文中执行该文件,就可以保持打开状态,并在离开上下文后立即将其关闭,无论您是由于异常还是在常规控制流程中离开了它。with因此,可以使用类似于C ++中的RAII模式的方式使用该语句:with语句并在您离开with上下文时释放。

  2. 一些示例是:使用打开文件,使用with open(filename) as fp:获取锁with lock:(在lock的实例threading.Lock)。您还可以使用中的contextmanager装饰器来构造自己的上下文管理器contextlib。例如,当我不得不临时更改当前目录然后返回到原来的位置时,我经常使用它:

    from contextlib import contextmanager
    import os
    
    @contextmanager
    def working_directory(path):
        current_dir = os.getcwd()
        os.chdir(path)
        try:
            yield
        finally:
            os.chdir(current_dir)
    
    with working_directory("data/stuff"):
        # do something within data/stuff
    # here I am back again in the original working directory
    

    这是另一个示例,该示例临时重定向sys.stdinsys.stdout并重定向sys.stderr到其他文件句柄并稍后将其还原:

    from contextlib import contextmanager
    import sys
    
    @contextmanager
    def redirected(**kwds):
        stream_names = ["stdin", "stdout", "stderr"]
        old_streams = {}
        try:
            for sname in stream_names:
                stream = kwds.get(sname, None)
                if stream is not None and stream != getattr(sys, sname):
                    old_streams[sname] = getattr(sys, sname)
                    setattr(sys, sname, stream)
            yield
        finally:
            for sname, stream in old_streams.iteritems():
                setattr(sys, sname, stream)
    
    with redirected(stdout=open("/tmp/log.txt", "w")):
         # these print statements will go to /tmp/log.txt
         print "Test entry 1"
         print "Test entry 2"
    # back to the normal stdout
    print "Back to normal stdout again"
    

    最后,另一个示例创建一个临时文件夹并在离开上下文时清理它:

    from tempfile import mkdtemp
    from shutil import rmtree
    
    @contextmanager
    def temporary_dir(*args, **kwds):
        name = mkdtemp(*args, **kwds)
        try:
            yield name
        finally:
            shutil.rmtree(name)
    
    with temporary_dir() as dirname:
        # do whatever you want
    
  1. I believe this has already been answered by other users before me, so I only add it for the sake of completeness: the with statement simplifies exception handling by encapsulating common preparation and cleanup tasks in so-called context managers. More details can be found in PEP 343. For instance, the open statement is a context manager in itself, which lets you open a file, keep it open as long as the execution is in the context of the with statement where you used it, and close it as soon as you leave the context, no matter whether you have left it because of an exception or during regular control flow. The with statement can thus be used in ways similar to the RAII pattern in C++: some resource is acquired by the with statement and released when you leave the with context.

  2. Some examples are: opening files using with open(filename) as fp:, acquiring locks using with lock: (where lock is an instance of threading.Lock). You can also construct your own context managers using the contextmanager decorator from contextlib. For instance, I often use this when I have to change the current directory temporarily and then return to where I was:

    from contextlib import contextmanager
    import os
    
    @contextmanager
    def working_directory(path):
        current_dir = os.getcwd()
        os.chdir(path)
        try:
            yield
        finally:
            os.chdir(current_dir)
    
    with working_directory("data/stuff"):
        # do something within data/stuff
    # here I am back again in the original working directory
    

    Here’s another example that temporarily redirects sys.stdin, sys.stdout and sys.stderr to some other file handle and restores them later:

    from contextlib import contextmanager
    import sys
    
    @contextmanager
    def redirected(**kwds):
        stream_names = ["stdin", "stdout", "stderr"]
        old_streams = {}
        try:
            for sname in stream_names:
                stream = kwds.get(sname, None)
                if stream is not None and stream != getattr(sys, sname):
                    old_streams[sname] = getattr(sys, sname)
                    setattr(sys, sname, stream)
            yield
        finally:
            for sname, stream in old_streams.iteritems():
                setattr(sys, sname, stream)
    
    with redirected(stdout=open("/tmp/log.txt", "w")):
         # these print statements will go to /tmp/log.txt
         print "Test entry 1"
         print "Test entry 2"
    # back to the normal stdout
    print "Back to normal stdout again"
    

    And finally, another example that creates a temporary folder and cleans it up when leaving the context:

    from tempfile import mkdtemp
    from shutil import rmtree
    
    @contextmanager
    def temporary_dir(*args, **kwds):
        name = mkdtemp(*args, **kwds)
        try:
            yield name
        finally:
            shutil.rmtree(name)
    
    with temporary_dir() as dirname:
        # do whatever you want
    

回答 1

我会建议两个有趣的讲座:

  • PEP 343 “ with”声明
  • Effbot了解Python的“ with”语句

1.with语句用于使用上下文管理器定义的方法来包装块的执行。这允许将常用try...except...finally用法模式封装起来,以方便重用。

2. 您可以执行以下操作:

with open("foo.txt") as foo_file:
    data = foo_file.read()

要么

from contextlib import nested
with nested(A(), B(), C()) as (X, Y, Z):
   do_something()

或(Python 3.1)

with open('data') as input_file, open('result', 'w') as output_file:
   for line in input_file:
     output_file.write(parse(line))

要么

lock = threading.Lock()
with lock:
    # Critical section of code

3. 我在这里看不到任何反模式。
引用Dive进入Python

试试..最终是好的。与更好。

4. 我想这与程序员使用try..catch..finally其他语言的语句的习惯有关。

I would suggest two interesting lectures:

  • PEP 343 The “with” Statement
  • Effbot Understanding Python’s “with” statement

1. The with statement is used to wrap the execution of a block with methods defined by a context manager. This allows common try...except...finally usage patterns to be encapsulated for convenient reuse.

2. You could do something like:

with open("foo.txt") as foo_file:
    data = foo_file.read()

OR

from contextlib import nested
with nested(A(), B(), C()) as (X, Y, Z):
   do_something()

OR (Python 3.1)

with open('data') as input_file, open('result', 'w') as output_file:
   for line in input_file:
     output_file.write(parse(line))

OR

lock = threading.Lock()
with lock:
    # Critical section of code

3. I don’t see any Antipattern here.
Quoting Dive into Python:

try..finally is good. with is better.

4. I guess it’s related to programmers’s habit to use try..catch..finally statement from other languages.


回答 2

Python with语句是Resource Acquisition Is InitializationC ++中常用的惯用语的内置语言支持。旨在允许安全获取和释放操作系统资源。

with语句在作用域/块内创建资源。您可以使用块中的资源编写代码。当该块退出时,无论该块中代码的结果如何(即该块是正常退出还是由于异常而退出),资源都会被干净地释放。

Python库中的许多资源都遵循该with语句所需的协议,因此可以立即使用。但是,任何人都可以通过实施有据可查的协议来制作可用于with语句的资源:PEP 0343

每当您在应用程序中获取必须明确放弃的资源(例如文件,网络连接,锁等)时,都应使用它。

The Python with statement is built-in language support of the Resource Acquisition Is Initialization idiom commonly used in C++. It is intended to allow safe acquisition and release of operating system resources.

The with statement creates resources within a scope/block. You write your code using the resources within the block. When the block exits the resources are cleanly released regardless of the outcome of the code in the block (that is whether the block exits normally or because of an exception).

Many resources in the Python library that obey the protocol required by the with statement and so can used with it out-of-the-box. However anyone can make resources that can be used in a with statement by implementing the well documented protocol: PEP 0343

Use it whenever you acquire resources in your application that must be explicitly relinquished such as files, network connections, locks and the like.


回答 3

再次为了完整性,我将添加最有用的with语句用例。

我进行了大量的科学计算,对于某些活动,我需要使用Decimal库来进行任意精度的计算。我的代码的某些部分需要高精度,而对于大多数其他部分,则需要较低的精度。

我将默认精度设置为一个较低的数字,然后使用with来获取某些部分的更精确答案:

from decimal import localcontext

with localcontext() as ctx:
    ctx.prec = 42   # Perform a high precision calculation
    s = calculate_something()
s = +s  # Round the final result back to the default precision

我在“超几何测试”中经常使用此功能,该测试需要将大量数字除以形式阶乘。在进行基因组比例计算时,必须注意舍入和溢出错误。

Again for completeness I’ll add my most useful use-case for with statements.

I do a lot of scientific computing and for some activities I need the Decimal library for arbitrary precision calculations. Some part of my code I need high precision and for most other parts I need less precision.

I set my default precision to a low number and then use with to get a more precise answer for some sections:

from decimal import localcontext

with localcontext() as ctx:
    ctx.prec = 42   # Perform a high precision calculation
    s = calculate_something()
s = +s  # Round the final result back to the default precision

I use this a lot with the Hypergeometric Test which requires the division of large numbers resulting form factorials. When you do genomic scale calculations you have to be careful of round-off and overflow errors.


回答 4

反模式的一个例子可能是使用with一个循环内时,它会更有效率有with外循环

例如

for row in lines:
    with open("outfile","a") as f:
        f.write(row)

with open("outfile","a") as f:
    for row in lines:
        f.write(row)

第一种方法是为每个文件打开和关闭文件,row而第二种方法是一次打开和关闭文件,这可能会导致性能问题。

An example of an antipattern might be to use the with inside a loop when it would be more efficient to have the with outside the loop

for example

for row in lines:
    with open("outfile","a") as f:
        f.write(row)

vs

with open("outfile","a") as f:
    for row in lines:
        f.write(row)

The first way is opening and closing the file for each row which may cause performance problems compared to the second way with opens and closes the file just once.


回答 5

参见PEP 343-‘with’语句,最后有一个示例部分。

… Python语言的新语句“ with”使排除try / finally语句的标准用法成为可能。

See PEP 343 – The ‘with’ statement, there is an example section at the end.

… new statement “with” to the Python language to make it possible to factor out standard uses of try/finally statements.


回答 6

第1、2和3点被合理地涵盖了:

4:它相对较新,仅在python2.6 +(或使用的python2.5 from __future__ import with_statement)中可用

points 1, 2, and 3 being reasonably well covered:

4: it is relatively new, only available in python2.6+ (or python2.5 using from __future__ import with_statement)


回答 7

with语句适用于所谓的上下文管理器:

http://docs.python.org/release/2.5.2/lib/typecontextmanager.html

这个想法是通过在离开“ with”块之后进行必要的清理来简化异常处理。一些python内置插件已经可以用作上下文管理器。

The with statement works with so-called context managers:

http://docs.python.org/release/2.5.2/lib/typecontextmanager.html

The idea is to simplify exception handling by doing the necessary cleanup after leaving the ‘with’ block. Some of the python built-ins already work as context managers.


回答 8

开箱即用支持的另一个示例是流行的数据库模块的对象,当您习惯于使用内置open()行为方式时,乍一看可能会感到困惑。connection

这些connection对象是上下文管理器,因此可以直接在中使用with-statement,但是在使用上述说明时,请注意:

with-block完成时,或者与异常或不,该连接不会关闭。万一with-block异常结束,则回滚事务,否则提交事务。

这意味着程序员必须注意自己关闭连接,但允许获取连接,并以多个方式使用它with-statements,如psycopg2 docs中所示:

conn = psycopg2.connect(DSN)

with conn:
    with conn.cursor() as curs:
        curs.execute(SQL1)

with conn:
    with conn.cursor() as curs:
        curs.execute(SQL2)

conn.close()

在上面的示例中,您会注意到的cursor对象psycopg2也是上下文管理器。从有关行为的相关文档中:

cursor出口退出时,with-block它将关闭,释放最终与之关联的任何资源。交易状态不受影响。

Another example for out-of-the-box support, and one that might be a bit baffling at first when you are used to the way built-in open() behaves, are connection objects of popular database modules such as:

The connection objects are context managers and as such can be used out-of-the-box in a with-statement, however when using the above note that:

When the with-block is finished, either with an exception or without, the connection is not closed. In case the with-block finishes with an exception, the transaction is rolled back, otherwise the transaction is commited.

This means that the programmer has to take care to close the connection themselves, but allows to acquire a connection, and use it in multiple with-statements, as shown in the psycopg2 docs:

conn = psycopg2.connect(DSN)

with conn:
    with conn.cursor() as curs:
        curs.execute(SQL1)

with conn:
    with conn.cursor() as curs:
        curs.execute(SQL2)

conn.close()

In the example above, you’ll note that the cursor objects of psycopg2 also are context managers. From the relevant documentation on the behavior:

When a cursor exits the with-block it is closed, releasing any resource eventually associated with it. The state of the transaction is not affected.


回答 9

在python中,通常使用“ with ”语句来打开文件,处理文件中存在的数据,以及不调用close()方法而关闭文件。“ with”语句通过提供清理活动使异常处理更加简单。

的一般形式:

with open(“file name”, mode”) as file-var:
    processing statements

注意:无需通过在file-var.close()上调用close()来关闭文件

In python generally “with” statement is used to open a file, process the data present in the file, and also to close the file without calling a close() method. “with” statement makes the exception handling simpler by providing cleanup activities.

General form of with:

with open(“file name”, “mode”) as file-var:
    processing statements

note: no need to close the file by calling close() upon file-var.close()


“ with”语句中有多个变量?

问题:“ with”语句中有多个变量?

使用withPython中的语句可以声明多个变量吗?

就像是:

from __future__ import with_statement

with open("out.txt","wt"), open("in.txt") as file_out, file_in:
    for line in file_in:
        file_out.write(line)

…还是同时清理两个资源是问题吗?

Is it possible to declare more than one variable using a with statement in Python?

Something like:

from __future__ import with_statement

with open("out.txt","wt"), open("in.txt") as file_out, file_in:
    for line in file_in:
        file_out.write(line)

… or is cleaning up two resources at the same time the problem?


回答 0

从v3.1Python 2.7 开始,Python 3中是可能的。新with语法支持多个上下文管理器:

with A() as a, B() as b, C() as c:
    doSomething(a,b,c)

不同于contextlib.nested,这保证了即使或方法引发异常,ab__exit__()调用C()__enter__()

您也可以在较新的定义中使用较早的变量(以下为h / t Ahmad):

with A() as a, B(a) as b, C(a, b) as c:
    doSomething(a, c)

It is possible in Python 3 since v3.1 and Python 2.7. The new with syntax supports multiple context managers:

with A() as a, B() as b, C() as c:
    doSomething(a,b,c)

Unlike the contextlib.nested, this guarantees that a and b will have their __exit__()‘s called even if C() or it’s __enter__() method raises an exception.

You can also use earlier variables in later definitions (h/t Ahmad below):

with A() as a, B(a) as b, C(a, b) as c:
    doSomething(a, c)

回答 1

contextlib.nested 支持这一点:

import contextlib

with contextlib.nested(open("out.txt","wt"), open("in.txt")) as (file_out, file_in):

   ...

更新:
要引用有关contextlib.nested以下内容的文档:

从2.7版开始不推荐使用:with语句现在直接支持此功能(不存在容易出错的易错问题)。

有关更多信息,请参见RafałDowgird的答案

contextlib.nested supports this:

import contextlib

with contextlib.nested(open("out.txt","wt"), open("in.txt")) as (file_out, file_in):

   ...

Update:
To quote the documentation, regarding contextlib.nested:

Deprecated since version 2.7: The with-statement now supports this functionality directly (without the confusing error prone quirks).

See Rafał Dowgird’s answer for more information.


回答 2

请注意,如果将变量分成几行,则必须使用反斜杠来包装换行符。

with A() as a, \
     B() as b, \
     C() as c:
    doSomething(a,b,c)

括号无效,因为Python会创建一个元组。

with (A(),
      B(),
      C()):
    doSomething(a,b,c)

由于元组缺少__enter__属性,因此会出现错误(描述性不够,并且无法标识类类型):

AttributeError: __enter__

如果尝试as在括号内使用,Python会在解析时捕获错误:

with (A() as a,
      B() as b,
      C() as c):
    doSomething(a,b,c)

SyntaxError:语法无效

https://bugs.python.org/issue12782似乎与此问题有关。

Note that if you split the variables into lines, you must use backslashes to wrap the newlines.

with A() as a, \
     B() as b, \
     C() as c:
    doSomething(a,b,c)

Parentheses don’t work, since Python creates a tuple instead.

with (A(),
      B(),
      C()):
    doSomething(a,b,c)

Since tuples lack a __enter__ attribute, you get an error (undescriptive and does not identify class type):

AttributeError: __enter__

If you try to use as within parentheses, Python catches the mistake at parse time:

with (A() as a,
      B() as b,
      C() as c):
    doSomething(a,b,c)

SyntaxError: invalid syntax

https://bugs.python.org/issue12782 seems to be related to this issue.


回答 3

我认为您想这样做:

from __future__ import with_statement

with open("out.txt","wt") as file_out:
    with open("in.txt") as file_in:
        for line in file_in:
            file_out.write(line)

I think you want to do this instead:

from __future__ import with_statement

with open("out.txt","wt") as file_out:
    with open("in.txt") as file_in:
        for line in file_in:
            file_out.write(line)

回答 4

因为Python 3.3,你可以使用类ExitStackcontextlib模块。

它可以管理动态数量的上下文感知对象,这意味着如果您不知道要处理多少文件,它将证明特别有用。

文档中提到的规范用例正在管理动态文件数量。

with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # All opened files will automatically be closed at the end of
    # the with statement, even if attempts to open files later
    # in the list raise an exception

这是一个通用示例:

from contextlib import ExitStack

class X:
    num = 1

    def __init__(self):
        self.num = X.num
        X.num += 1

    def __repr__(self):
        cls = type(self)
        return '{cls.__name__}{self.num}'.format(cls=cls, self=self)

    def __enter__(self):
        print('enter {!r}'.format(self))
        return self.num

    def __exit__(self, exc_type, exc_value, traceback):
        print('exit {!r}'.format(self))
        return True

xs = [X() for _ in range(3)]

with ExitStack() as stack:
    print(stack._exit_callbacks)
    nums = [stack.enter_context(x) for x in xs]
    print(stack._exit_callbacks)
print(stack._exit_callbacks)
print(nums)

输出:

deque([])
enter X1
enter X2
enter X3
deque([<function ExitStack._push_cm_exit.<locals>._exit_wrapper at 0x7f5c95f86158>, <function ExitStack._push_cm_exit.<locals>._exit_wrapper at 0x7f5c95f861e0>, <function ExitStack._push_cm_exit.<locals>._exit_wrapper at 0x7f5c95f86268>])
exit X3
exit X2
exit X1
deque([])
[1, 2, 3]

Since Python 3.3, you can use the class ExitStack from the contextlib module.

It can manage a dynamic number of context-aware objects, which means that it will prove especially useful if you don’t know how many files you are going to handle.

The canonical use-case that is mentioned in the documentation is managing a dynamic number of files.

with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # All opened files will automatically be closed at the end of
    # the with statement, even if attempts to open files later
    # in the list raise an exception

Here is a generic example:

from contextlib import ExitStack

class X:
    num = 1

    def __init__(self):
        self.num = X.num
        X.num += 1

    def __repr__(self):
        cls = type(self)
        return '{cls.__name__}{self.num}'.format(cls=cls, self=self)

    def __enter__(self):
        print('enter {!r}'.format(self))
        return self.num

    def __exit__(self, exc_type, exc_value, traceback):
        print('exit {!r}'.format(self))
        return True

xs = [X() for _ in range(3)]

with ExitStack() as stack:
    print(stack._exit_callbacks)
    nums = [stack.enter_context(x) for x in xs]
    print(stack._exit_callbacks)
print(stack._exit_callbacks)
print(nums)

Output:

deque([])
enter X1
enter X2
enter X3
deque([<function ExitStack._push_cm_exit.<locals>._exit_wrapper at 0x7f5c95f86158>, <function ExitStack._push_cm_exit.<locals>._exit_wrapper at 0x7f5c95f861e0>, <function ExitStack._push_cm_exit.<locals>._exit_wrapper at 0x7f5c95f86268>])
exit X3
exit X2
exit X1
deque([])
[1, 2, 3]

回答 5

在Python 3.1+中,您可以指定多个上下文表达式,它们将像with嵌套了多个语句一样进行处理:

with A() as a, B() as b:
    suite

相当于

with A() as a:
    with B() as b:
        suite

这也意味着您可以在第二个表达式中使用第一个表达式的别名(在使用数据库连接/游标时很有用):

with get_conn() as conn, conn.cursor() as cursor:
    cursor.execute(sql)

In Python 3.1+ you can specify multiple context expressions, and they will be processed as if multiple with statements were nested:

with A() as a, B() as b:
    suite

is equivalent to

with A() as a:
    with B() as b:
        suite

This also means that you can use the alias from the first expression in the second (useful when working with db connections/cursors):

with get_conn() as conn, conn.cursor() as cursor:
    cursor.execute(sql)