When pytest encounters a failing assert in a test, it stops its execution immediatelly. In many cases the tests should ideally continue running to get a complete picture of things which are failing. For example, when testing TLS-related functionality, it might be useful to test both the supported TLS versions and ciphers in a single run, instead of having to stop-and-fix if TLS version check fails first.

Luckily, this should be possible to easily achieve with the pytest-check plugin.

The following should be done:

• Introduce the pytest-check plugin into the project.
• Go through tests for all roles, and switch to using the pytest-check functionality where it makes sense.
• Update development instructions.
• Update role reference documentation.
• Update usage instructions.
• Update release noets.