缓存:使用交叉测试运行状态

使用

该插件提供了两个命令行选项,用于从上一个命令行重新运行失败。 pytest 调用:

  • --lf--last-failed -只重新运行故障。

  • --ff--failed-first -先运行失败,然后运行其余的测试。

对于清理(通常不需要),a --cache-clear 选项允许在测试运行之前删除所有跨会话缓存内容。

其他插件可以访问 config.cache 要设置/获取的对象 JSON可编程序 之间的价值 pytest 调用。

注解

此插件在默认情况下是启用的,但在需要时可以禁用:请参阅 按名称停用/注销插件 (此插件的内部名称为 cacheprovider

只重新运行失败或失败

首先,让我们创建50个测试调用,其中只有2个失败:

# content of test_50.py
import pytest


@pytest.mark.parametrize("i", range(50))
def test_num(i):
    if i in (17, 25):
        pytest.fail("bad luck")

如果您第一次运行它,您将看到两个失败:

$ pytest -q
.................F.......F........................                   [100%]
================================= FAILURES =================================
_______________________________ test_num[17] _______________________________

i = 17

    @pytest.mark.parametrize("i", range(50))
    def test_num(i):
        if i in (17, 25):
>           pytest.fail("bad luck")
E           Failed: bad luck

test_50.py:7: Failed
_______________________________ test_num[25] _______________________________

i = 25

    @pytest.mark.parametrize("i", range(50))
    def test_num(i):
        if i in (17, 25):
>           pytest.fail("bad luck")
E           Failed: bad luck

test_50.py:7: Failed
========================= short test summary info ==========================
FAILED test_50.py::test_num[17] - Failed: bad luck
FAILED test_50.py::test_num[25] - Failed: bad luck
2 failed, 48 passed in 0.12s

如果你那么用 --lf

$ pytest --lf
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
run-last-failure: rerun previous 2 failures

test_50.py FF                                                        [100%]

================================= FAILURES =================================
_______________________________ test_num[17] _______________________________

i = 17

    @pytest.mark.parametrize("i", range(50))
    def test_num(i):
        if i in (17, 25):
>           pytest.fail("bad luck")
E           Failed: bad luck

test_50.py:7: Failed
_______________________________ test_num[25] _______________________________

i = 25

    @pytest.mark.parametrize("i", range(50))
    def test_num(i):
        if i in (17, 25):
>           pytest.fail("bad luck")
E           Failed: bad luck

test_50.py:7: Failed
========================= short test summary info ==========================
FAILED test_50.py::test_num[17] - Failed: bad luck
FAILED test_50.py::test_num[25] - Failed: bad luck
============================ 2 failed in 0.12s =============================

您只运行了上次运行中的两个失败测试,而48个通过的测试没有运行(“取消选择”)。

现在,如果你和 --ff 选项,将运行所有测试,但将首先执行前面的第一个失败(从 FF 和点):

$ pytest --ff
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 50 items
run-last-failure: rerun previous 2 failures first

test_50.py FF................................................        [100%]

================================= FAILURES =================================
_______________________________ test_num[17] _______________________________

i = 17

    @pytest.mark.parametrize("i", range(50))
    def test_num(i):
        if i in (17, 25):
>           pytest.fail("bad luck")
E           Failed: bad luck

test_50.py:7: Failed
_______________________________ test_num[25] _______________________________

i = 25

    @pytest.mark.parametrize("i", range(50))
    def test_num(i):
        if i in (17, 25):
>           pytest.fail("bad luck")
E           Failed: bad luck

test_50.py:7: Failed
========================= short test summary info ==========================
FAILED test_50.py::test_num[17] - Failed: bad luck
FAILED test_50.py::test_num[25] - Failed: bad luck
======================= 2 failed, 48 passed in 0.12s =======================

新的 --nf--new-first 选项:首先运行新的测试,然后运行其余的测试,在这两种情况下,测试也按文件修改时间排序,最新的文件排在第一位。

上次运行中没有测试失败时的行为

上次运行中没有测试失败,或没有缓存时 lastfailed 数据被发现, pytest 可以配置为运行所有测试或不运行任何测试,使用 --last-failed-no-failures 选项,它采用以下值之一:

pytest --last-failed --last-failed-no-failures all    # run all tests (default behavior)
pytest --last-failed --last-failed-no-failures none   # run no tests and exit

新的config.cache对象

插件或conftest.py支持代码可以使用pytest获取缓存值。 config 对象。下面是一个实现 fixture 它跨pytest调用重用先前创建的状态:

# content of test_caching.py
import pytest
import time


def expensive_computation():
    print("running expensive computation...")


@pytest.fixture
def mydata(request):
    val = request.config.cache.get("example/value", None)
    if val is None:
        expensive_computation()
        val = 42
        request.config.cache.set("example/value", val)
    return val


def test_function(mydata):
    assert mydata == 23

如果第一次运行此命令,则可以看到print语句:

$ pytest -q
F                                                                    [100%]
================================= FAILURES =================================
______________________________ test_function _______________________________

mydata = 42

    def test_function(mydata):
>       assert mydata == 23
E       assert 42 == 23

test_caching.py:20: AssertionError
-------------------------- Captured stdout setup ---------------------------
running expensive computation...
========================= short test summary info ==========================
FAILED test_caching.py::test_function - assert 42 == 23
1 failed in 0.12s

如果再次运行该值,将从缓存中检索该值,并且不会打印任何内容:

$ pytest -q
F                                                                    [100%]
================================= FAILURES =================================
______________________________ test_function _______________________________

mydata = 42

    def test_function(mydata):
>       assert mydata == 23
E       assert 42 == 23

test_caching.py:20: AssertionError
========================= short test summary info ==========================
FAILED test_caching.py::test_function - assert 42 == 23
1 failed in 0.12s

config.cache fixture 了解更多详细信息。

正在检查缓存内容

您可以使用 --cache-show 命令行选项:

$ pytest --cache-show
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
cachedir: $PYTHON_PREFIX/.pytest_cache
--------------------------- cache values for '*' ---------------------------
cache/lastfailed contains:
  {'test_50.py::test_num[17]': True,
   'test_50.py::test_num[25]': True,
   'test_assert1.py::test_function': True,
   'test_assert2.py::test_set_comparison': True,
   'test_caching.py::test_function': True,
   'test_foocompare.py::test_compare': True}
cache/nodeids contains:
  ['test_50.py::test_num[0]',
   'test_50.py::test_num[10]',
   'test_50.py::test_num[11]',
   'test_50.py::test_num[12]',
   'test_50.py::test_num[13]',
   'test_50.py::test_num[14]',
   'test_50.py::test_num[15]',
   'test_50.py::test_num[16]',
   'test_50.py::test_num[17]',
   'test_50.py::test_num[18]',
   'test_50.py::test_num[19]',
   'test_50.py::test_num[1]',
   'test_50.py::test_num[20]',
   'test_50.py::test_num[21]',
   'test_50.py::test_num[22]',
   'test_50.py::test_num[23]',
   'test_50.py::test_num[24]',
   'test_50.py::test_num[25]',
   'test_50.py::test_num[26]',
   'test_50.py::test_num[27]',
   'test_50.py::test_num[28]',
   'test_50.py::test_num[29]',
   'test_50.py::test_num[2]',
   'test_50.py::test_num[30]',
   'test_50.py::test_num[31]',
   'test_50.py::test_num[32]',
   'test_50.py::test_num[33]',
   'test_50.py::test_num[34]',
   'test_50.py::test_num[35]',
   'test_50.py::test_num[36]',
   'test_50.py::test_num[37]',
   'test_50.py::test_num[38]',
   'test_50.py::test_num[39]',
   'test_50.py::test_num[3]',
   'test_50.py::test_num[40]',
   'test_50.py::test_num[41]',
   'test_50.py::test_num[42]',
   'test_50.py::test_num[43]',
   'test_50.py::test_num[44]',
   'test_50.py::test_num[45]',
   'test_50.py::test_num[46]',
   'test_50.py::test_num[47]',
   'test_50.py::test_num[48]',
   'test_50.py::test_num[49]',
   'test_50.py::test_num[4]',
   'test_50.py::test_num[5]',
   'test_50.py::test_num[6]',
   'test_50.py::test_num[7]',
   'test_50.py::test_num[8]',
   'test_50.py::test_num[9]',
   'test_assert1.py::test_function',
   'test_assert2.py::test_set_comparison',
   'test_caching.py::test_function',
   'test_foocompare.py::test_compare']
cache/stepwise contains:
  []
example/value contains:
  42

========================== no tests ran in 0.12s ===========================

--cache-show 采用可选参数指定用于筛选的全局模式:

$ pytest --cache-show example/*
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
cachedir: $PYTHON_PREFIX/.pytest_cache
----------------------- cache values for 'example/*' -----------------------
example/value contains:
  42

========================== no tests ran in 0.12s ===========================

清除缓存内容

通过添加 --cache-clear 这样的选择:

pytest --cache-clear

对于那些隔离和正确性比速度更重要的连续集成服务器的调用,建议这样做。

逐步的

作为替代 --lf -x 尤其是在您期望测试套件的大部分将失败的情况下, --sw--stepwise 允许您一次修复一个。测试套件将运行到第一次失败,然后停止。在下一次调用时,测试将从上次失败的测试继续进行,然后运行到下一次失败的测试。你可以使用 --stepwise-skip 选项忽略一个失败的测试并停止对第二个失败的测试的执行。如果你在一个失败的测试中陷入困境,并且只想在稍后忽略它,那么这很有用。