Implement Allure listeners #855

Context

People often need to somehow post-process Allure results according to their needs. Examples would be:

• filtering data they don't want in the report (parameters/labels/fixtures)
• customizing the default names, existing parameters, etc
• sending the results somewhere on the fly

Many such requirements are user-specific, and there is no point in implementing them here. Often, it's not even possible because many such requirements contradict each other.

Instead, it would be reasonable to have an easy-to-use extension mechanism that users may utilize for their purposes.

Existing solution

Currently, users can register custom plugins in allure_commons.plugin_manager. For example:

from allure_commons import hookimpl
from allure_commons import plugin_manager

class MyListener:
    @hookimpl(tryfirst=True)
    def report_result(self, result):
        # post-process the result here
        ...

def pytest_configure(config):

    def cleanup():
        name = plugin_manager.get_name(listener)
        plugin_manager.unregister(name=name)

    listener = MyListener()
    plugin_manager.register(listener)
    config.add_cleanup(cleanup)

While covering many cases, the solution has some drawbacks:

• Verbose API:
  • tryfirst is necessary to make sure the hook implementation is called after the default logger's one.
  • The cleanup needs to be added separately.
• Limited hook spec:
  • No steps and fixtures post-processing (only at the test result level, which complicates the processing).
  • Limited support for attachments post-processing.
  • No hooks for pre-processing.

Implementation

Lifecycle listeners can be implemented with pluggy as a plugin manager + hook specs + user-friendly API on top of this. We already use pluggy to implement the Allure API, so it's just a matter of defining one or more hook specs, calling them via the existing allure_commons.plugin_manager or a separate one.