Majic Projects
Gimmecert
Go to the previous open issue
Go to the previous issue (open or closed)
star_faded.png
Please log in to bookmark issues
icon_project.png Gimmecert / Closed Enhancement GC-14 Clean-up test runtime configuration and imrpove usability
Go to the next issue (open or closed)
Go to the next open issue
This issue has been closed with status "Closed" and resolution "RESOLVED".
Issue basics
  • Type of issue
    Enhancement
  • Category
    Not determined
  • Targetted for
    0.1.0
  • Status
    Closed
  • Progress
  • Priority
    Not determined
User pain
  • Type of bug
    Not triaged
  • Likelihood
    Not triaged
  • Effect
    Not triaged
Affected by this issue (0)
There are no items
People involved
Times and dates
  • Posted at
  • Last updated
  • Estimated time
    1 hour
  • Time spent
    3 hours
    Click here to see time logged against this issue
Issue details
  • Resolution
    RESOLVED
Attachments (0)
There is nothing attached to this issue
Duplicate issues (0)
This issue does not have any duplicates
Description

While working on one of the first major issues, and number of small annoyances have been popped-up in relation to test configuration and behaviour.

The following problems have been observed:

  • Running with pytest-flake8 (--flake8 option) will result in tests being shown as skipped. In particular, pytest-flake8 will skip performing linting tests against test files themselves.
  • Running with pytest-flake8 (--flake8 option) will result in a lot of noise during development due to styling issues being reported. Although useful in terms of keeping the code clean, it does get annoying after a while with some temporary imports and such.
  • pytest-flake8 ignores the .flake8 configuration file, resulting in linting issues being reported for the maximum line length.
  • Running with current configuration for coverage ends-up creating the coverage directory in a bunch of directories. This should most likely not be done, especially given that test coverage should be pretty high given the TDD-oriented development.
  • In order to see coverage issues, it is necessary to open HTML page. This is an additional step, and could be relatively easily missed.
  • Running tests against just the functional tests will produce invalid coverage report, since functional tests are run from a black-box perspective.
  • Running tests using tox will fail because coverage will report failure to produce report while running the functional tests.
  • Invoking pytest will run functional tests as well, which can be a bit slow, and documentation implies the functional tests are not being run automatically.

The following should be done:

  • Remove use of pytest-flake8. This includes configuration, dependencies, and documentation. Keep the .flake8 configuration file, though.
  • Update development documentation with instructions on how to run the linter (flake8) directly instead.
  • Remove generation of coverage report in HTML format. Instead replace the default invocation to use --cov-report=term-missing:skip-covered. This should provide immediate output and info about what lines are not being covered by tests.
  • Remove coverage runs from default target, and have it as an explicit step. Do run the coverage tests via tox, though.
  • Update the tox command to run both unit and functional tests using a single command.
  • Update the pytest.ini configuration to explicitly list just the unit tests path (tests/), so that invocation of pytest would run just the unit tests (without functional tests).
  • Update the development instructions to let the developers know about the behaviour implemented.
  • Update tox configuration to ensure the Sphinx documentation does not get built in cloned git directory.
  • Update coverage configuration to fail on anything below 100%.
Todos (0 / 0)
There are no comments