有没有一种方法可以指定从文件运行哪些pytest测试?

问题:有没有一种方法可以指定从文件运行哪些pytest测试?

有没有办法选择pytest要从文件运行的测试?例如,一个foo.txt包含要执行的测试列表的文件:

tests_directory/foo.py::test_001
tests_directory/bar.py::test_some_other_test

或者,是否可以通过pytest从不同目录中选择多个测试,而这些测试的测试名称中没有相同的模式?

pytest -k <pattern> 允许使用单一模式。

一种选择是pytest.mark对每个测试使用a ,但是我的要求是对不同文件运行不同的测试组合。

有没有一种方法可以指定多个模式以及每个模式的测试文件名?

要么

有没有一种方法可以在文件中指定确切的测试路径,并将该文件作为输入输入pytest

要么

是否有可以用于此目的的挂钩函数?

Is there a way to select pytest tests to run from a file? For example, a file foo.txt containing a list of tests to be executed:

tests_directory/foo.py::test_001
tests_directory/bar.py::test_some_other_test

Or, is there a way to select multiple tests, having no common pattern in test name, from different directories with pytest?

pytest -k <pattern> allows a single pattern.

One option is to have a pytest.mark against each test, but my requirement is to run different combination of tests from different files.

Is there a way to specify multiple patterns and a test file name for each pattern?

Or

Is there a way to specify the exact test paths in a file and feed that file as an input to pytest?

Or

Is there a hook function that can be utilized for this purpose?


回答 0

您可以使用-koption运行具有不同模式的测试用例:

py.test tests_directory/foo.py tests_directory/bar.py -k 'test_001 or test_some_other_test'

这将运行名称为test_001test_some_other_test的测试用例从而取消选择其余的测试用例。

注意:这将选择任何以test_001或test_some_other_test开头的测试用例。例如,如果您有测试用例test_0012,则也将选择它。

You can use -k option to run test cases with different patterns:

py.test tests_directory/foo.py tests_directory/bar.py -k 'test_001 or test_some_other_test'

This will run test cases with name test_001 and test_some_other_test deselecting the rest of the test cases.

Note: This will select any test case starting with test_001 or test_some_other_test. For example, if you have test case test_0012 it will also be selected.


回答 1

指定测试/选择测试

Pytest支持从命令行运行和选择测试的几种方法。

在模块中运行测试

pytest test_mod.py

在目录中运行测试

pytest testing/

通过关键字表达式运行测试

pytest -k "MyClass and not method"

这将运行包含与给定字符串表达式匹配的名称的测试,其中可能包括使用文件名,类名和函数名作为变量的Python运算符。上面的示例将运行,TestMyClass.test_something但不会运行TestMyClass.test_method_simple

按节点ID运行测试

每个收集的测试都分配有一个唯一的nodeid名称,该名称由模块文件名后跟说明符(例如类名,函数名和参数化参数)组成,并用::字符分隔。

要在模块中运行特定的测试,请执行以下操作:

pytest test_mod.py::test_func

在命令行中指定测试方法的另一个示例:

pytest test_mod.py::TestClass::test_method

通过标记表达式运行测试

pytest -m slow

将运行用@pytest.mark.slow装饰器装饰的所有测试。

有关更多信息,请参见标记

从包运行测试

pytest --pyargs pkg.testing

这将导入pkg.testing并使用其文件系统位置来查找并运行测试。

来源:https//docs.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests

Specifying tests / selecting tests

Pytest supports several ways to run and select tests from the command-line.

Run tests in a module

pytest test_mod.py

Run tests in a directory

pytest testing/

Run tests by keyword expressions

pytest -k "MyClass and not method"

This will run tests which contain names that match the given string expression, which can include Python operators that use filenames, class names and function names as variables. The example above will run TestMyClass.test_something but not TestMyClass.test_method_simple.

Run tests by node ids

Each collected test is assigned a unique nodeid which consist of the module filename followed by specifiers like class names, function names and parameters from parametrization, separated by :: characters.

To run a specific test within a module:

pytest test_mod.py::test_func

Another example specifying a test method in the command line:

pytest test_mod.py::TestClass::test_method

Run tests by marker expressions

pytest -m slow

Will run all tests which are decorated with the @pytest.mark.slow decorator.

For more information see marks.

Run tests from packages

pytest --pyargs pkg.testing

This will import pkg.testing and use its filesystem location to find and run tests from.

Source: https://docs.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests


回答 2

我的答案提供了在不同情况下运行测试子集的方法。

运行项目中的所有测试

pytest

在单个目录中运行测试

要从一个目录运行所有测试,请将该目录用作以下参数 pytest

pytest tests/my-directory

在单个测试文件/模块中运行测试

要运行充满测试的文件,请列出文件,并将相对路径作为参数pytest

pytest tests/my-directory/test_demo.py

运行一个测试功能

要运行单个测试功能,请添加::和测试功能名称:

pytest -v tests/my-directory/test_demo.py::test_specific_function

-v 用于查看运行了哪个功能。

运行一个测试班

要只运行一个类,请像对函数和add一样进行操作::,然后将类名添加到file参数:

pytest -v tests/my-directory/test_demo.py::TestClassName

运行测试类的单一测试方法

如果您不想运行所有测试类,只需运行一个方法,只需添加另一个::方法名称即可:

pytest -v tests/my-directory/test_demo.py::TestClassName::test_specific_method

根据测试名称运行一组测试

-k选项使您可以传入表达式以运行具有由表达式指定为测试名称的子字符串的某些名称的测试。它可以使用,并没有创建复杂的表达式。

例如,要运行所有名称中带有_raises的函数:

pytest -v -k _raises

My answer provides a ways to run a subset of test in different scenarios.

Run all tests in a project

pytest

Run tests in a Single Directory

To run all the tests from one directory, use the directory as a parameter to pytest:

pytest tests/my-directory

Run tests in a Single Test File/Module

To run a file full of tests, list the file with the relative path as a parameter to pytest:

pytest tests/my-directory/test_demo.py

Run a Single Test Function

To run a single test function, add :: and the test function name:

pytest -v tests/my-directory/test_demo.py::test_specific_function

-v is used so you can see which function was run.

Run a Single Test Class

To run just a class, do like we did with functions and add ::, then the class name to the file parameter:

pytest -v tests/my-directory/test_demo.py::TestClassName

Run a Single Test Method of a Test Class

If you don’t want to run all of a test class, just one method, just add another :: and the method name:

pytest -v tests/my-directory/test_demo.py::TestClassName::test_specific_method

Run a Set of Tests Based on Test Name

The -k option enables you to pass in an expression to run tests that have certain names specified by the expression as a substring of the test name. It is possible to use and, or, and not to create complex expressions.

For example, to run all of the functions that have _raises in their name:

pytest -v -k _raises

回答 3

如果在两个不同的类中具有相同的方法名称,并且只想运行其中一个,则可以这样做:

pytest tests.py -k 'TestClassName and test_method_name'

If you have the same method name in two different classes and you just want to run one of them, this works:

pytest tests.py -k 'TestClassName and test_method_name'

回答 4

方法1:随机选择测试。长而丑。

python -m pytest test/stress/test_performance.py::TestPerformance::test_continuous_trigger test/integration/test_config.py::TestConfig::test_valid_config

方法2:使用关键字表达式。

注意:我正在按测试用例名称进行搜索。同样适用于TestClass名称。

情况1:无论找到什么,下面的内容都会运行。由于我们使用了’OR’。

python -m pytest -k 'test_password_valid or test_no_configuration'

可以说上述两个实际上是正确的,将运行2个测试。

情况2:现在一个不正确的名称和另一个正确的名称。

python -m pytest -k 'test_password_validzzzzzz or test_no_configuration' 

仅找到一个并运行。

情况3:如果要运行所有测试或不运行任何测试,请使用AND

python -m pytest -k 'test_password_valid and test_no_configuration'

如果正确或不执行,两者都将运行。

情况4:仅在一个文件夹中运行测试:

python -m pytest test/project1/integration -k 'test_password_valid or test_no_configuration'

情况5:仅从一个文件运行测试。

python -m pytest test/integration/test_authentication.py -k 'test_password_expiry or test_incorrect_password'

情况6:运行除匹配项外的所有测试。

python -m pytest test/integration/test_authentication.py -k 'not  test_incorrect_password'

Method 1: Randomly selected tests. Long and ugly.

python -m pytest test/stress/test_performance.py::TestPerformance::test_continuous_trigger test/integration/test_config.py::TestConfig::test_valid_config

Method 2: Use Keyword Expressions.

Note: I am searching by testcase names. Same is applicable to TestClass names.

Case 1: The below will run whichever is found. Since we have used ‘OR’ .

python -m pytest -k 'test_password_valid or test_no_configuration'

Lets say the two above are actually correct, 2 tests will be run.

Case 2: Now an incorrect name and another correct name.

python -m pytest -k 'test_password_validzzzzzz or test_no_configuration' 

Only one is found and run.

Case 3: If you want all tests to run or no one, then use AND

python -m pytest -k 'test_password_valid and test_no_configuration'

Both will be run if correct or none.

Case 4: Run test only in one folder:

python -m pytest test/project1/integration -k 'test_password_valid or test_no_configuration'

Case 5: Run test from only one file.

python -m pytest test/integration/test_authentication.py -k 'test_password_expiry or test_incorrect_password'

Case 6: Run all tests except the match.

python -m pytest test/integration/test_authentication.py -k 'not  test_incorrect_password'

回答 5

也许使用pytest_collect_file()钩子,您可以解析.txto.yaml文件的内容,在其中您可以根据需要指定测试,然后将它们返回到pytest核心。

pytest文档中显示了一个很好的示例。我认为您在寻找什么。

Maybe using pytest_collect_file() hook you can parse the content of a .txt o .yaml file where the tests are specify as you want, and return them to the pytest core.

A nice example is shown in the pytest documentation. I think what you are looking for.


回答 6

这是一个可能的部分答案,因为它仅允许选择测试脚本,而不能选择这些脚本中的单个测试。

而且它还受到我使用旧版兼容模式与unittest脚本的限制,因此不能保证它可以与本机pytest一起使用。

开始:

  1. 比如说创建一个新的字典subset_tests_directory
  2. ln -s tests_directory/foo.py
  3. ln -s tests_directory/bar.py

  4. 注意隐式假定文件位于的导入test_directory。我必须通过运行python foo.py,从内部进行subset_tests_directory并根据需要进行更正来修复其中的一些问题。

  5. 一旦测试脚本执行正确,公正cd subset_tests_directorypytest有。Pytest只会选择它看到的脚本。

另一种可能性是你当前的测试目录中的符号链接,说的ln -s foo.py subset_foo.py那么pytest subset*.py。这样可以避免调整导入,但在删除符号链接之前,事情会变得很混乱。也为我工作。

Here’s a possible partial answer, because it only allows selecting the test scripts, not individual tests within those scripts.

And it also limited by my using legacy compatibility mode vs unittest scripts, so not guaranteeing it would work with native pytest.

Here goes:

  1. create a new dictory, say subset_tests_directory.
  2. ln -s tests_directory/foo.py
  3. ln -s tests_directory/bar.py

  4. be careful about imports which implicitly assume files are in test_directory. I had to fix several of those by running python foo.py, from within subset_tests_directory and correcting as needed.

  5. Once the test scripts execute correctly, just cd subset_tests_directory and pytest there. Pytest will only pick up the scripts it sees.

Another possibility is symlinking within your current test directory, say as ln -s foo.py subset_foo.py then pytest subset*.py. That would avoid needing to adjust your imports, but it would clutter things up until you removed the symlinks. Worked for me as well.


回答 7

根据有关按节点ID运行测试的文档

由于您在foo.txt中拥有所有节点ID,因此只需运行

pytest `cat foo.txt | tr '\n' ' '`

这与以下命令相同(问题中包含文件内容)

pytest tests_directory/foo.py::test_001 tests_directory/bar.py::test_some_other_test

According to the doc about Run tests by node ids

since you have all node ids in foo.txt, just run

pytest `cat foo.txt | tr '\n' ' '`

this is same with below command (with file content in the question)

pytest tests_directory/foo.py::test_001 tests_directory/bar.py::test_some_other_test