Add a Testing chapter for backend packages#2094
Open
jensens wants to merge 2 commits into
Open
Conversation
Plone 6 has no backend testing documentation. The Developer guide has no Testing section, and the only rendered HTML for plone.testing and plone.app.testing lives on the legacy 5.docs.plone.org domain. @davisagli sketched a full outline for such a chapter in a comment on plone/pytest-plone#16 two years ago. This picks up the two pieces of it that cover backend Python testing, and deliberately leaves the rest -- Volto, Robot Framework, testbrowser -- to build on top. - conceptual-guides/testing.md explains the kinds of tests, why Plone has testing layers at all, and how to choose between the two runners. - developer-guide/testing/ shows how to write and run tests with either zope.testrunner or pytest. The pytest guide links to the published pytest-plone documentation at plone.github.io/pytest-plone. Volto testing is cross-referenced rather than duplicated: it lives in the Volto repository. Every zope-testrunner command shown was executed against a real package before being documented. Vale reports 0 errors, 0 warnings, and 0 suggestions on the new pages; linkcheck confirms every external link resolves.
Writing the testing.py layer is the setup work both zope.testrunner and pytest-plone build on, and the chapter previously punted it to the large plone.app.testing README. That left the developer guide without its keystone: the one thing every backend test needs. Add developer-guide/testing/write-a-testing-layer.md, placed first in the sub-toctree because it is the shared prerequisite. It shows both forms: - the declarative PloneWithPackageLayer instance, for a simple add-on; - the PloneSandboxLayer subclass with setUpZope/setUpPloneSite, for anything more involved. Both forms, and the WSGI_SERVER_FIXTURE note for real-HTTP functional tests, are taken from real coredev packages (plone.app.dexterity, plone.volto) and the plone.app.testing README, and every import was checked against the installed packages. The runner guides and the conceptual guide now point here instead of at the README. Also drop the code-span from the zope.testrunner headings, which rendered as a heavy monospace box next to the plain 'pytest' heading.
gforcada
reviewed
Jul 17, 2026
Comment on lines
+77
to
+94
| `plone.app.testing` | ||
| `plone.testing` | ||
| `pytest-plone` | ||
| `zope.testrunner` | ||
| conftest | ||
| pytest | ||
| pytest-plone | ||
| testbrowser | ||
| plone.app.testing | ||
| plone.testing | ||
| zope.testrunner | ||
| PloneSandboxLayer | ||
| PloneWithPackageLayer | ||
| IntegrationTesting | ||
| FunctionalTesting | ||
| setUpZope | ||
| setUpPloneSite | ||
| applyProfile |
Member
There was a problem hiding this comment.
these should be sorted like the rest of the file
|
|
||
| Acceptance test | ||
| : Drives the whole system as a user would, through a browser. | ||
| In Plone this means Robot Framework or a Zope testbrowser for Classic UI, and Cypress for Volto. |
Member
There was a problem hiding this comment.
Should we rather use Blicca already? and Aurora? 🤔
I remember reading that volto/aurora at some point they want to switch (or already did) to playwright and not use cypress anymore?
| Choose `zope.testrunner` if you contribute to Plone core, or maintain a package whose tests already use it. | ||
| There is no reward for converting a working suite. | ||
|
|
||
| Choose pytest for new work, unless you have a reason not to. |
Member
There was a problem hiding this comment.
Suggested change
| Choose pytest for new work, unless you have a reason not to. | |
| Choose `pytest` for new work, unless you have a reason not to. |
Comment on lines
+123
to
+127
| ## Measure coverage | ||
|
|
||
| ```shell | ||
| pytest --cov=my.addon --cov-report term-missing | ||
| ``` |
Member
There was a problem hiding this comment.
This example will not work if you don't do a pip install pytest-cov, I guess it is worth mentioning.
| ``` | ||
|
|
||
| `-s` takes the dotted name of the package to search. | ||
| `--auto-path` adds that package's directory to the test search path, which is what you want for a package installed in development mode. |
Member
There was a problem hiding this comment.
I personally never had to use it. Not in zc.buildout contexts, nor in zc.buildout-free contexts (i.e. plone.meta's tox-based environments), there we use --test-path
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Plone 6 has no backend testing documentation. The Developer guide has no Testing section (only Volto testing, under Contributing), and the only rendered HTML for
plone.testingandplone.app.testinglives on the legacy5.docs.plone.orgdomain.@davisagli sketched a full outline for such a chapter in a comment on plone/pytest-plone#16 about two years ago, covering Volto/Jest/Cypress, unittest/
zope.testrunner, pytest, Robot, and testbrowser. This PR picks up the two pieces of that outline that cover backend Python testing and deliberately leaves the rest to build on top. Please treat the structure as a starting point — I expect there are opinions on placement, and I am happy to move things.What
Five pages, Diataxis-classified:
Explanation —
conceptual-guides/testing.md: The kinds of tests (unit / integration / functional / acceptance), why Plone has testing layers at all, and how to choose between the two runners.
How-to —
developer-guide/testing/: -
write-a-testing-layer.md— the keystone, placed first: writing thetesting.pylayer both runners share. Shows the declarativePloneWithPackageLayerform for simple add-ons and thePloneSandboxLayersubclass form for complex ones.: -
zope-testrunner.md— write and run tests withunittestandzope.testrunner.: -
pytest.md— write and run tests with pytest, linking to the published pytest-plone documentation.Scope
plone.app.testing/plone.testing; the how-to links there rather than reproducing it.Verification
zope-testrunnercommand shown was executed against a real package. The two layer forms are taken from real coredev packages (plone.app.dexterity,plone.volto) and theplone.app.testingREADME, with every import checked against the installed packages.make htmlproduces no new warnings, andlinkcheckfinds no broken links on the new pages.Related
Companion to the newly published pytest-plone documentation (plone/pytest-plone#56), which this chapter's pytest guide links to.