simplebench.reporters.reporter_manager.manager module🔗

ReporterManager for benchmark results.

This module contains the ReporterManager class, which manages the registration, retrieval, and unregistration of Reporter classes and instances.

This registry is global. When a Reporter is registered, it is added to the global registry, and when it is unregistered, it is removed from the global registry.

class simplebench.reporters.reporter_manager.manager.ReporterManager(load_defaults: bool = True)[source]🔗

Bases: object

Manager for all available Reporter classes.

This class maintains a registry of Reporter instances and their associated CLI arguments. It allows for the registration, retrieval, and unregistration of Reporter instances.

Initially, it registers a set of built-in Reporter instances: CSVReporter, ScatterPlotReporter, and RichTableReporter.

New Reporter instances can be added using the register() method, and existing ones can be removed using the unregister() or unregister_by_name() methods.

Reporter instances are required to have unique names and CLI arguments in their Choices to avoid conflicts and the manager will raise exceptions if duplicates are detected during registration.

Registering a custom reporter example🔗

reporter_manager = ReporterManager()
my_custom_reporter = CustomReporter()
reporter_manager.register(my_custom_reporter)

Initialize the ReporterManager.

By default, this loads a set of predefined Reporter instances and then any reporters registered via the register_reporter() decorator.

If one of the predefined reporters’ dependencies are not installed, that reporter will be skipped and not registered.

If desired, this can be skipped by setting load_defaults to False. In which case, the manager starts with an empty registry and Reporter instances can be added via the register() method.

Parameters:: load_defaults (bool) – Whether to load the predefined reporters. Defaults to True.

add_reporters_to_argparse( parser: ArgumentParser, ) → None[source]🔗

Add all registered reporter choices to an ArgumentParser.

This method adds the CLI arguments for all registered reporters to the provided ArgumentParser instance using the add_flags_to_argparse() method of each Reporter.

Parameters:: parser (ArgumentParser) – The ArgumentParser to add the flags to.

all_reporters() → dict[str, Reporter][source]🔗

Return all available Reporter instances as a dictionary keyed by their names.

Returns:: A dictionary of all Reporter instances with their names as keys.
Return type:: dict[str, Reporter]

choice_for_arg( arg: str, ) → Choice | None[source]🔗

Return the Choice instance for a given CLI argument.

Parameters:: arg (str) – The CLI argument to look up.
Returns:: The corresponding Choice instance, or None if not found.
Return type:: Choice | None

property choices: Choices🔗

Return a Choices instance containing all registered reporter choices.

Returns:: A Choices instance with all registered reporter choices.
Return type:: Choices

register( reporter: Reporter, ) → None[source]🔗

This method adds a new Reporter to the registry. Unlike the predefined reporters and the register_reporter() decorator which both register by class, this method requires an instance of the Reporter.

This allows for more flexibility, such as registering Reporter instances with custom initialization parameters.

Parameters:

reporter (Reporter) – The Reporter to register.

Raises:

SimpleBenchValueError – If a Reporter with the same name or CLI argument is already registered.
SimpleBenchTypeError – If the provided reporter is not an instance of Reporter.

unregister( reporter: Reporter, ) → None[source]🔗

Unregister a Reporter by an instance exemplar.

It looks up the reporter by its name and removes it from the registry.

reporter_manager.unregister(MyCustomReporter())

Parameters:

reporter (Reporter) – Reporter instance exemplar to unregister.

Raises:

SimpleBenchKeyError – If no reporter with the given name is registered.
SimpleBenchTypeError – If the provided reporter is not an instance of Reporter.

unregister_all() → None[source]🔗

Unregister all Reporter instances.

This clears the entire registry of Reporter instances.

unregister_by_name( name: str, ) → None[source]🔗

Unregister a Reporter by its name.

Parameters:: name (str) – The name of the Reporter to unregister.
Raises:: SimpleBenchKeyError – If no reporter with the given name is registered.