simplebench.reporters.choice.choice module

Choice() for reporters.

class simplebench.reporters.choice.choice.Choice(

*,

reporter: Reporter,

choice_conf: ChoiceConf,

)

Bases: Hashable, ChoiceProtocol

Definition of a Choice option for live use by reporters.

A Choice represents a specific configuration of a Reporter subclass, including the sections to include in the report, the output targets, and the output formats.

The Choice class provides a structured way to define and manage different reporting options within the SimpleBench framework so that users can select from predefined configurations and developers can easily add new reporting options to the framework.

A Choice instance is immutable after creation to ensure consistency in reporting configurations.

The sections, targets, and formats are descriptive only; they do not enforce any behavior on the associated Reporter subclass. It is the responsibility of the Reporter subclass to implement the behavior corresponding to the specified sections, targets, and formats.

It is intended that multiple Choice instances can be created for a single Reporter subclass to represent different configurations of that reporter. For example, a JSON reporter might have one Choice that includes all sections and outputs to the filesystem, and another Choice that includes only the OPS section and outputs to the console. This allows users to select from a variety of predefined reporting configurations without needing to create multiple Reporter subclasses.

Choice instances are created by the Reporter subclass from ChoiceConf instances during a Reporter’s instantation process.

Separating ChoiceConf and Choice allows for a clear distinction between the configuration data (ChoiceConf) and the live, operational representation (Choice) used by the Reporter subclass.

Parameters:

• reporter (Reporter) – The Reporter subclass instance associated with the choice.

• choice_conf (ChoiceConf) – The ChoiceConf instance used to create the choice.

• flags (set[str]) – A set of command-line flags associated with the choice.

• flag_type (FlagType) – The type of command-line flag (e.g., boolean, target_list, etc.).

• name (str) – A unique name for the choice.

• description (str) – A brief description of the choice.

• sections (set[Section]) – A set of Section enums to include in the report.

• targets (set[Target]) – A set of Target enums for output.

• default_targets (set[Target] | None) – A set of Target enums representing the default targets for the choice. If None, no default targets are specified and the reporter’s defaults will be used when generating reports.

• output_format (Format) – A Format instance describing the output format.

• subdir (str | None) – An optional subdirectory for output files. If None, reports default to the reporter’s subdir.

• file_suffix (str | None) – An optional file suffix for output files. If None, reports default to the reporter’s file_suffix.

• file_unique (bool | None) – Whether to make output file names unique. If None, reports default to the reporter’s file_unique.

• file_append (bool | None) – Whether to append to existing output files. If None, reports default to the reporter’s file_append.

• options (ReporterOptions | None) – An optional ReporterOptions instance for additional configurations specific to a specific reporter. If None, reports default to the reporter’s default options.

• extra (Any | None) – Any additional metadata associated with the choice.

Construct a Choice instance from a Reporter and a ChoiceConf instance.

Parameters:

• reporter (Reporter) – An instance of a Reporter subclass.

• choice_conf (ChoiceConf) – An instance of ChoiceConf containing the configuration for the choice.

Raises:

• SimpleBenchTypeError – If any argument is of an incorrect type.

• SimpleBenchValueError – If any argument has an invalid value (e.g., empty strings or empty sequences).

property choice_conf: ChoiceConf

The ChoiceConf instance used to create the choice.

property default_targets: frozenset[Target] | None

Default output targets for the choice.

These are the default output targets that the associated Reporter subclass should use when this choice is selected, if no specific target

is provided by the user.

property description: str

Description of the choice.

This is used as help text for the command-line flags associated with the choice.

property extra: Any

Any additional metadata associated with the choice.

property file_append: bool | None

Whether to append to existing output files.

If specified, the associated Reporter subclass should use this setting for output files when this choice is selected. If None, the reporter’s default file_append setting should be used.

property file_suffix: str | None

An optional file suffix for output files.

If specified, the associated Reporter subclass should use this file suffix for output files when this choice is selected. If None, the reporter’s default file_suffix should be used.

property file_unique: bool | None

Whether to make output file names unique.

If specified, the associated Reporter subclass should use this setting for output files when this choice is selected. If None, the reporter’s default file_unique setting should be used.

property flag_type: FlagType

The type of command-line flag (e.g., BOOLEAN, TARGET_LIST, etc.).

property flags: frozenset[str]

Flags associated with the choice. These are used for command-line selection. They must be unique across all choices for all reporters. This is enforced by the ReporterManager when choices are registered.

The flags should be in the format used on the command line, typically starting with -- for long options.

['--json', '--json-full']

The description property of the Choice is used to provide help text for the flags when generating command-line help.

property name: str

Name of the choice.

property options: ReporterOptions | None

An optional ReporterOptions instance for additional configuration.

property output_format: Format

Output format for the choice.

This is the output format that the associated Reporter subclass is expected to use when this choice is selected.

Returns:

The output format.

Return type:

Format

property reporter: Reporter

The Reporter sub-class associated with the choice.

property sections: frozenset[Section]

Sections included in the choice.

These are the sections that the associated Reporter subclass is expected to include in its report when this choice is selected.

property subdir: str | None

An optional subdirectory for output files.

If specified, the associated Reporter subclass should use this subdirectory for output files when this choice is selected. If an empty string, no subdirectory should be used. If None, the reporter’s default subdir should be used.

property targets: frozenset[Target]

Output targets for the choice.

These are the output targets that the associated Reporter subclass is expected to use when this choice is selected.

simplebench.reporters.choice.choice.deferred_reporter_import() → None

Deferred import of Reporter to avoid circular imports during initialization.