标签归档:pytest

如何正确断言pytest中引发了异常?

问题:如何正确断言pytest中引发了异常?

码:

# coding=utf-8
import pytest


def whatever():
    return 9/0

def test_whatever():
    try:
        whatever()
    except ZeroDivisionError as exc:
        pytest.fail(exc, pytrace=True)

输出:

================================ test session starts =================================
platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2
plugins: django, cov
collected 1 items 

pytest_test.py F

====================================== FAILURES ======================================
___________________________________ test_whatever ____________________________________

    def test_whatever():
        try:
            whatever()
        except ZeroDivisionError as exc:
>           pytest.fail(exc, pytrace=True)
E           Failed: integer division or modulo by zero

pytest_test.py:12: Failed
============================== 1 failed in 1.16 seconds ==============================

如何使pytest打印回溯,所以我会看到在whatever函数中引发异常的地方?

Code:

# coding=utf-8
import pytest


def whatever():
    return 9/0

def test_whatever():
    try:
        whatever()
    except ZeroDivisionError as exc:
        pytest.fail(exc, pytrace=True)

Output:

================================ test session starts =================================
platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2
plugins: django, cov
collected 1 items 

pytest_test.py F

====================================== FAILURES ======================================
___________________________________ test_whatever ____________________________________

    def test_whatever():
        try:
            whatever()
        except ZeroDivisionError as exc:
>           pytest.fail(exc, pytrace=True)
E           Failed: integer division or modulo by zero

pytest_test.py:12: Failed
============================== 1 failed in 1.16 seconds ==============================

How to make pytest print traceback, so I would see where in the whatever function an exception was raised?


回答 0

pytest.raises(Exception) 是您所需要的。

import pytest

def test_passes():
    with pytest.raises(Exception) as e_info:
        x = 1 / 0

def test_passes_without_info():
    with pytest.raises(Exception):
        x = 1 / 0

def test_fails():
    with pytest.raises(Exception) as e_info:
        x = 1 / 1

def test_fails_without_info():
    with pytest.raises(Exception):
        x = 1 / 1

# Don't do this. Assertions are caught as exceptions.
def test_passes_but_should_not():
    try:
        x = 1 / 1
        assert False
    except Exception:
        assert True

# Even if the appropriate exception is caught, it is bad style,
# because the test result is less informative
# than it would be with pytest.raises(e)
# (it just says pass or fail.)

def test_passes_but_bad_style():
    try:
        x = 1 / 0
        assert False
    except ZeroDivisionError:
        assert True

def test_fails_but_bad_style():
    try:
        x = 1 / 1
        assert False
    except ZeroDivisionError:
        assert True

输出量

============================================================================================= test session starts ==============================================================================================
platform linux2 -- Python 2.7.6 -- py-1.4.26 -- pytest-2.6.4
collected 7 items 

test.py ..FF..F

=================================================================================================== FAILURES ===================================================================================================
__________________________________________________________________________________________________ test_fails __________________________________________________________________________________________________

    def test_fails():
        with pytest.raises(Exception) as e_info:
>           x = 1 / 1
E           Failed: DID NOT RAISE

test.py:13: Failed
___________________________________________________________________________________________ test_fails_without_info ____________________________________________________________________________________________

    def test_fails_without_info():
        with pytest.raises(Exception):
>           x = 1 / 1
E           Failed: DID NOT RAISE

test.py:17: Failed
___________________________________________________________________________________________ test_fails_but_bad_style ___________________________________________________________________________________________

    def test_fails_but_bad_style():
        try:
            x = 1 / 1
>           assert False
E           assert False

test.py:43: AssertionError
====================================================================================== 3 failed, 4 passed in 0.02 seconds ======================================================================================

请注意,e_info将保存异常对象,以便您可以从中提取详细信息。例如,如果要检查异常调用堆栈或内部的另一个嵌套异常。

pytest.raises(Exception) is what you need.

Code

import pytest

def test_passes():
    with pytest.raises(Exception) as e_info:
        x = 1 / 0

def test_passes_without_info():
    with pytest.raises(Exception):
        x = 1 / 0

def test_fails():
    with pytest.raises(Exception) as e_info:
        x = 1 / 1

def test_fails_without_info():
    with pytest.raises(Exception):
        x = 1 / 1

# Don't do this. Assertions are caught as exceptions.
def test_passes_but_should_not():
    try:
        x = 1 / 1
        assert False
    except Exception:
        assert True

# Even if the appropriate exception is caught, it is bad style,
# because the test result is less informative
# than it would be with pytest.raises(e)
# (it just says pass or fail.)

def test_passes_but_bad_style():
    try:
        x = 1 / 0
        assert False
    except ZeroDivisionError:
        assert True

def test_fails_but_bad_style():
    try:
        x = 1 / 1
        assert False
    except ZeroDivisionError:
        assert True

Output

============================================================================================= test session starts ==============================================================================================
platform linux2 -- Python 2.7.6 -- py-1.4.26 -- pytest-2.6.4
collected 7 items 

test.py ..FF..F

=================================================================================================== FAILURES ===================================================================================================
__________________________________________________________________________________________________ test_fails __________________________________________________________________________________________________

    def test_fails():
        with pytest.raises(Exception) as e_info:
>           x = 1 / 1
E           Failed: DID NOT RAISE

test.py:13: Failed
___________________________________________________________________________________________ test_fails_without_info ____________________________________________________________________________________________

    def test_fails_without_info():
        with pytest.raises(Exception):
>           x = 1 / 1
E           Failed: DID NOT RAISE

test.py:17: Failed
___________________________________________________________________________________________ test_fails_but_bad_style ___________________________________________________________________________________________

    def test_fails_but_bad_style():
        try:
            x = 1 / 1
>           assert False
E           assert False

test.py:43: AssertionError
====================================================================================== 3 failed, 4 passed in 0.02 seconds ======================================================================================

Note that e_info saves the exception object so you can extract details from it. For example, if you want to check the exception call stack or another nested exception inside.


回答 1

您的意思是这样的吗:

def test_raises():
    with pytest.raises(Exception) as execinfo:   
        raise Exception('some info')
    # these asserts are identical; you can use either one   
    assert execinfo.value.args[0] == 'some info'
    assert str(execinfo.value) == 'some info'

Do you mean something like this:

def test_raises():
    with pytest.raises(Exception) as execinfo:   
        raise Exception('some info')
    # these asserts are identical; you can use either one   
    assert execinfo.value.args[0] == 'some info'
    assert str(execinfo.value) == 'some info'

回答 2

有两种方法可以在pytest中处理此类情况:

  • 使用pytest.raises功能

  • 使用pytest.mark.xfail装饰器

用途pytest.raises

def whatever():
    return 9/0
def test_whatever():
    with pytest.raises(ZeroDivisionError):
        whatever()

用途pytest.mark.xfail

@pytest.mark.xfail(raises=ZeroDivisionError)
def test_whatever():
    whatever()

输出pytest.raises

============================= test session starts ============================
platform linux2 -- Python 2.7.10, pytest-3.2.3, py-1.4.34, pluggy-0.4.0 -- 
/usr/local/python_2.7_10/bin/python
cachedir: .cache
rootdir: /home/user, inifile:
collected 1 item

test_fun.py::test_whatever PASSED


======================== 1 passed in 0.01 seconds =============================

pytest.xfail标记的输出:

============================= test session starts ============================
platform linux2 -- Python 2.7.10, pytest-3.2.3, py-1.4.34, pluggy-0.4.0 -- 
/usr/local/python_2.7_10/bin/python
cachedir: .cache
rootdir: /home/user, inifile:
collected 1 item

test_fun.py::test_whatever xfail

======================== 1 xfailed in 0.03 seconds=============================

文档所述

使用pytest.raises很可能是更好的为你在哪里测试异常你自己的代码被刻意提高的情况下,而使用@pytest.mark.xfail具有校验功能可能是更好的东西,像记录未修正的错误(如测试描述什么是“应该”发生)或错误的依赖。

There are two ways to handle these kind of cases in pytest:

  • Using pytest.raises function

  • Using pytest.mark.xfail decorator

As the documentation says:

Using pytest.raises is likely to be better for cases where you are testing exceptions your own code is deliberately raising, whereas using @pytest.mark.xfail with a check function is probably better for something like documenting unfixed bugs (where the test describes what “should” happen) or bugs in dependencies.

Usage of pytest.raises:

def whatever():
    return 9/0
def test_whatever():
    with pytest.raises(ZeroDivisionError):
        whatever()

Usage of pytest.mark.xfail:

@pytest.mark.xfail(raises=ZeroDivisionError)
def test_whatever():
    whatever()

Output of pytest.raises:

============================= test session starts ============================
platform linux2 -- Python 2.7.10, pytest-3.2.3, py-1.4.34, pluggy-0.4.0 -- 
/usr/local/python_2.7_10/bin/python
cachedir: .cache
rootdir: /home/user, inifile:
collected 1 item

test_fun.py::test_whatever PASSED


======================== 1 passed in 0.01 seconds =============================

Output of pytest.xfail marker:

============================= test session starts ============================
platform linux2 -- Python 2.7.10, pytest-3.2.3, py-1.4.34, pluggy-0.4.0 -- 
/usr/local/python_2.7_10/bin/python
cachedir: .cache
rootdir: /home/user, inifile:
collected 1 item

test_fun.py::test_whatever xfail

======================== 1 xfailed in 0.03 seconds=============================

回答 3

你可以试试

def test_exception():
    with pytest.raises(Exception) as excinfo:   
        function_that_raises_exception()   
    assert str(excinfo.value) == 'some info' 

you can try

def test_exception():
    with pytest.raises(Exception) as excinfo:   
        function_that_raises_exception()   
    assert str(excinfo.value) == 'some info' 

回答 4

pytest不断发展,并且随着最近的一次不错的变化,现在可以同时测试

  • 异常类型(严格测试)
  • 错误消息(使用正则表达式进行严格检查或宽松检查)

文档中的两个示例:

with pytest.raises(ValueError, match='must be 0 or None'):
    raise ValueError('value must be 0 or None')
with pytest.raises(ValueError, match=r'must be \d+$'):
    raise ValueError('value must be 42')

我已经在许多项目中使用了这种方法,并且非常喜欢它。

pytest constantly evolves and with one of the nice changes in the recent past it is now possible to simultaneously test for

  • the exception type (strict test)
  • the error message (strict or loose check using a regular expression)

Two examples from the documentation:

with pytest.raises(ValueError, match='must be 0 or None'):
    raise ValueError('value must be 0 or None')
with pytest.raises(ValueError, match=r'must be \d+$'):
    raise ValueError('value must be 42')

I have been using that approach in a number of projects and like it very much.


回答 5

正确的方法正在使用,pytest.raises但是我在这里的评论中找到了有趣的替代方法,并希望将其保存给以后这个问题的读者:

try:
    thing_that_rasises_typeerror()
    assert False
except TypeError:
    assert True

Right way is using pytest.raises but I found interesting alternative way in comments here and want to save it for future readers of this question:

try:
    thing_that_rasises_typeerror()
    assert False
except TypeError:
    assert True

回答 6

此解决方案是我们正在使用的解决方案:

def test_date_invalidformat():
    """
    Test if input incorrect data will raises ValueError exception
    """
    date = "06/21/2018 00:00:00"
    with pytest.raises(ValueError):
        app.func(date) #my function to be tested

请参阅pytest,https: //docs.pytest.org/en/latest/reference.html#pytest-raises

This solution is what we are using:

def test_date_invalidformat():
    """
    Test if input incorrect data will raises ValueError exception
    """
    date = "06/21/2018 00:00:00"
    with pytest.raises(ValueError):
        app.func(date) #my function to be tested

Please refer to pytest, https://docs.pytest.org/en/latest/reference.html#pytest-raises


回答 7

更好的做法是使用继承unittest.TestCase并运行self.assertRaises的类。

例如:

import unittest


def whatever():
    return 9/0


class TestWhatEver(unittest.TestCase):

    def test_whatever():
        with self.assertRaises(ZeroDivisionError):
            whatever()

然后,您可以通过运行以下命令来执行它:

pytest -vs test_path

Better practice will be using a class that inherit unittest.TestCase and running self.assertRaises.

For example:

import unittest


def whatever():
    return 9/0


class TestWhatEver(unittest.TestCase):

    def test_whatever():
        with self.assertRaises(ZeroDivisionError):
            whatever()

Then you would execute it by running:

pytest -vs test_path

回答 8

您是否尝试删除“ pytrace = True”?

pytest.fail(exc, pytrace=True) # before
pytest.fail(exc) # after

您是否尝试过使用’–fulltrace’吗?

Have you tried to remove “pytrace=True” ?

pytest.fail(exc, pytrace=True) # before
pytest.fail(exc) # after

Have you tried to run with ‘–fulltrace’ ?


如何查看pytest运行期间创建的正常打印输出?

问题:如何查看pytest运行期间创建的正常打印输出?

有时候,我只想在代码中插入一些打印语句,然后看看在执行该操作时打印出来的内容。我通常的“锻炼”方式是使用现有的pytest测试。但是,当我运行这些命令时,我似乎看不到任何标准输出(至少从我的IDE PyCharm内部)。

有没有一种简单的方法可以在pytest运行期间查看标准输出?

Sometimes I want to just insert some print statements in my code, and see what gets printed out when I exercise it. My usual way to “exercise” it is with existing pytest tests. But when I run these, I don’t seem able to see any standard output (at least from within PyCharm, my IDE).

Is there a simple way to see standard output during a pytest run?


回答 0

-s开关禁用每次测试捕获。

The -s switch disables per-test capturing.


回答 1

在对已接受答案的评论中,问:

有什么方法可以打印到控制台捕获输出,以便将其显示在junit报告中?

在UNIX中,这通常称为teeing。理想情况下,py.test默认是发球而不是捕获。尽管Python很少支持开箱即用,但py.test或任何现有的第三方py.test插件(无论如何我都知道)都不理想。

用Monkey修补py.test来做不受支持的事情并非易事。为什么?因为:

  • 大多数py.test功能都锁定在打算从外部导入的私有_pytest软件包后面。尝试这样做而不知道自己在做什么,通常会导致公共包在运行时引发模糊的异常。非常感谢py.test。真正可靠的体系结构。pytest
  • 即使您确实想出了如何以_pytest安全的方式对专用API 进行Monkey补丁的操作,也必须运行pytest由外部py.test命令运行的公共程序包之前这样做。您不能在插件中执行此操作(例如,conftest测试套件中的顶级模块)。到py.test懒惰地动态导入插件时,您想要进行Monkey补丁的任何py.test类都已被实例化很久了-并且您无权访问该实例。这意味着,如果您希望有意义地应用Monkey补丁,则无法再安全地运行外部py.test命令。相反,您必须使用自定义setuptools包装该命令的运行test 命令(按顺序):
    1. Monkey修补专用_pytestAPI。
    2. 调用public pytest.main()函数运行py.test命令。

这个答案是Monkey修补py.test -s--capture=no选项来捕获stderr而不是 stdout的。默认情况下,这些选项既不捕获stderr也不捕获stdout。当然,这还不够。但是,每一次伟大的旅程都是从一个繁琐的前传开始的,每个人在五年之内就忘记了。

为什么这样 我现在告诉你。我的py.test驱动的测试套件包含缓慢的功能测试。显示这些测试的标准输出是有益的,让人放心,防止leycec到达了killall -9 py.test当又一个长期运行的功能测试失败做几星期什么。但是,显示这些测试的stderr可以防止py.test报告有关测试失败的异常回溯。这是完全没有帮助的。因此,我们强制py.test捕获stderr 而不捕获stdout。

在开始之前,此答案假定您已经有一个test调用py.test 的自定义setuptools 命令。如果不这样做,请参阅py.test编写良好的“良好做法”页面的“ 手动集成”小节。

不要安装pytest亚军,第三方插件setuptools的提供自定义setuptools的test命令也调用py.test。如果已经安装pytest-runner,则可能需要卸载该pip3软件包,然后采用上面链接的手动方法。

假设您按照上面突出显示的“ 手动集成”中的说明进行操作,则您的代码库现在应包含一个PyTest.run_tests()方法。修改此方法,使其类似于:

class PyTest(TestCommand):
             .
             .
             .
    def run_tests(self):
        # Import the public "pytest" package *BEFORE* the private "_pytest"
        # package. While importation order is typically ignorable, imports can
        # technically have side effects. Tragicomically, that is the case here.
        # Importing the public "pytest" package establishes runtime
        # configuration required by submodules of the private "_pytest" package.
        # The former *MUST* always be imported before the latter. Failing to do
        # so raises obtuse exceptions at runtime... which is bad.
        import pytest
        from _pytest.capture import CaptureManager, FDCapture, MultiCapture

        # If the private method to be monkey-patched no longer exists, py.test
        # is either broken or unsupported. In either case, raise an exception.
        if not hasattr(CaptureManager, '_getcapture'):
            from distutils.errors import DistutilsClassError
            raise DistutilsClassError(
                'Class "pytest.capture.CaptureManager" method _getcapture() '
                'not found. The current version of py.test is either '
                'broken (unlikely) or unsupported (likely).'
            )

        # Old method to be monkey-patched.
        _getcapture_old = CaptureManager._getcapture

        # New method applying this monkey-patch. Note the use of:
        #
        # * "out=False", *NOT* capturing stdout.
        # * "err=True", capturing stderr.
        def _getcapture_new(self, method):
            if method == "no":
                return MultiCapture(
                    out=False, err=True, in_=False, Capture=FDCapture)
            else:
                return _getcapture_old(self, method)

        # Replace the old with the new method.
        CaptureManager._getcapture = _getcapture_new

        # Run py.test with all passed arguments.
        errno = pytest.main(self.pytest_args)
        sys.exit(errno)

要启用此Monkey补丁,请运行py.test,如下所示:

python setup.py test -a "-s"

现在将捕获Stderr而不是 stdout。好漂亮!

将上面的Monkey补丁扩展到tee stdout和stderr上,作为练习,留给读者一整桶的空闲时间。

In an upvoted comment to the accepted answer, Joe asks:

Is there any way to print to the console AND capture the output so that it shows in the junit report?

In UNIX, this is commonly referred to as teeing. Ideally, teeing rather than capturing would be the py.test default. Non-ideally, neither py.test nor any existing third-party py.test plugin (…that I know of, anyway) supports teeing – despite Python trivially supporting teeing out-of-the-box.

Monkey-patching py.test to do anything unsupported is non-trivial. Why? Because:

  • Most py.test functionality is locked behind a private _pytest package not intended to be externally imported. Attempting to do so without knowing what you’re doing typically results in the public pytest package raising obscure exceptions at runtime. Thanks alot, py.test. Really robust architecture you got there.
  • Even when you do figure out how to monkey-patch the private _pytest API in a safe manner, you have to do so before running the public pytest package run by the external py.test command. You cannot do this in a plugin (e.g., a top-level conftest module in your test suite). By the time py.test lazily gets around to dynamically importing your plugin, any py.test class you wanted to monkey-patch has long since been instantiated – and you do not have access to that instance. This implies that, if you want your monkey-patch to be meaningfully applied, you can no longer safely run the external py.test command. Instead, you have to wrap the running of that command with a custom setuptools test command that (in order):
    1. Monkey-patches the private _pytest API.
    2. Calls the public pytest.main() function to run the py.test command.

This answer monkey-patches py.test’s -s and --capture=no options to capture stderr but not stdout. By default, these options capture neither stderr nor stdout. This isn’t quite teeing, of course. But every great journey begins with a tedious prequel everyone forgets in five years.

Why do this? I shall now tell you. My py.test-driven test suite contains slow functional tests. Displaying the stdout of these tests is helpful and reassuring, preventing leycec from reaching for killall -9 py.test when yet another long-running functional test fails to do anything for weeks on end. Displaying the stderr of these tests, however, prevents py.test from reporting exception tracebacks on test failures. Which is completely unhelpful. Hence, we coerce py.test to capture stderr but not stdout.

Before we get to it, this answer assumes you already have a custom setuptools test command invoking py.test. If you don’t, see the Manual Integration subsection of py.test’s well-written Good Practices page.

Do not install pytest-runner, a third-party setuptools plugin providing a custom setuptools test command also invoking py.test. If pytest-runner is already installed, you’ll probably need to uninstall that pip3 package and then adopt the manual approach linked to above.

Assuming you followed the instructions in Manual Integration highlighted above, your codebase should now contain a PyTest.run_tests() method. Modify this method to resemble:

class PyTest(TestCommand):
             .
             .
             .
    def run_tests(self):
        # Import the public "pytest" package *BEFORE* the private "_pytest"
        # package. While importation order is typically ignorable, imports can
        # technically have side effects. Tragicomically, that is the case here.
        # Importing the public "pytest" package establishes runtime
        # configuration required by submodules of the private "_pytest" package.
        # The former *MUST* always be imported before the latter. Failing to do
        # so raises obtuse exceptions at runtime... which is bad.
        import pytest
        from _pytest.capture import CaptureManager, FDCapture, MultiCapture

        # If the private method to be monkey-patched no longer exists, py.test
        # is either broken or unsupported. In either case, raise an exception.
        if not hasattr(CaptureManager, '_getcapture'):
            from distutils.errors import DistutilsClassError
            raise DistutilsClassError(
                'Class "pytest.capture.CaptureManager" method _getcapture() '
                'not found. The current version of py.test is either '
                'broken (unlikely) or unsupported (likely).'
            )

        # Old method to be monkey-patched.
        _getcapture_old = CaptureManager._getcapture

        # New method applying this monkey-patch. Note the use of:
        #
        # * "out=False", *NOT* capturing stdout.
        # * "err=True", capturing stderr.
        def _getcapture_new(self, method):
            if method == "no":
                return MultiCapture(
                    out=False, err=True, in_=False, Capture=FDCapture)
            else:
                return _getcapture_old(self, method)

        # Replace the old with the new method.
        CaptureManager._getcapture = _getcapture_new

        # Run py.test with all passed arguments.
        errno = pytest.main(self.pytest_args)
        sys.exit(errno)

To enable this monkey-patch, run py.test as follows:

python setup.py test -a "-s"

Stderr but not stdout will now be captured. Nifty!

Extending the above monkey-patch to tee stdout and stderr is left as an exercise to the reader with a barrel-full of free time.


回答 2

运行测试时,请使用该-s选项。exampletest.py测试运行时,所有打印语句将在控制台上打印。

py.test exampletest.py -s

When running the test use the -s option. All print statements in exampletest.py would get printed on the console when test is run.

py.test exampletest.py -s

回答 3

根据pytest文档pytest的版本3可以临时禁用测试中的捕获:

def test_disabling_capturing(capsys):
    print('this output is captured')
    with capsys.disabled():
        print('output not captured, going directly to sys.stdout')
    print('this output is also captured')

According to pytest documentation, version 3 of pytest can temporary disable capture in a test:

def test_disabling_capturing(capsys):
    print('this output is captured')
    with capsys.disabled():
        print('output not captured, going directly to sys.stdout')
    print('this output is also captured')

回答 4

pytest捕获单个测试的标准输出,并仅在特定条件下显示它们,并默认显示其打印的测试摘要。

额外的摘要信息可以使用’-r’选项显示:

pytest -rP

显示已通过测试的捕获输出。

pytest -rx

显示捕获的失败测试输出(默认行为)。

-r的输出格式比-s的输出格式更漂亮。

pytest captures the stdout from individual tests and displays them only on certain conditions, along with the summary of the tests it prints by default.

Extra summary info can be shown using the ‘-r’ option:

pytest -rP

shows the captured output of passed tests.

pytest -rx

shows the captured output of failed tests (default behaviour).

The formatting of the output is prettier with -r than with -s.


回答 5


尝试 pytest -s -v test_login.py在控制台中获取更多信息。

-v 很短 --verbose

-s 表示“禁用所有捕获”




Try pytest -s -v test_login.py for more info in console.

-v it’s a short --verbose

-s means ‘disable all capturing’




回答 6

如果您使用的是PyCharm IDE,则可以使用“运行”工具栏运行该单个测试或所有测试。“运行”工具窗口显示由应用程序生成的输出,您可以在其中看到所有打印语句,作为测试输出的一部分。

If you are using PyCharm IDE, then you can run that individual test or all tests using Run toolbar. The Run tool window displays output generated by your application and you can see all the print statements in there as part of test output.


回答 7

pytest --capture=tee-sys是最近添加的。您可以捕获并查看stdout / err上的输出。

pytest --capture=tee-sys was recently added. You can capture as well as see the output on stdout/err.


回答 8

其他答案不起作用。查看捕获的输出的唯一方法是使用以下标志:

pytest-显示全部

The other answers don’t work. The only way to see the captured output is using the following flag:

pytest –show-capture all