simplebench.reporters.choice package

simplebench.reporters.choice package.

class simplebench.reporters.choice.Choice(

*,

reporter: Reporter,

choice_conf: ChoiceConf,

)

Bases: Hashable, ChoiceProtocol

Definition of a Choice option for live use by reporters.

A Choice represents a specific configuration of a Reporter subclass, including the sections to include in the report, the output targets, and the output formats.

The Choice class provides a structured way to define and manage different reporting options within the SimpleBench framework so that users can select from predefined configurations and developers can easily add new reporting options to the framework.

A Choice instance is immutable after creation to ensure consistency in reporting configurations.

The sections, targets, and formats are descriptive only; they do not enforce any behavior on the associated Reporter subclass. It is the responsibility of the Reporter subclass to implement the behavior corresponding to the specified sections, targets, and formats.

It is intended that multiple Choice instances can be created for a single Reporter subclass to represent different configurations of that reporter. For example, a JSON reporter might have one Choice that includes all sections and outputs to the filesystem, and another Choice that includes only the OPS section and outputs to the console. This allows users to select from a variety of predefined reporting configurations without needing to create multiple Reporter subclasses.

Choice instances are created by the Reporter subclass from ChoiceConf instances during a Reporter’s instantation process.

Separating ChoiceConf and Choice allows for a clear distinction between the configuration data (ChoiceConf) and the live, operational representation (Choice) used by the Reporter subclass.

Parameters:

• reporter (Reporter) – The Reporter subclass instance associated with the choice.

• choice_conf (ChoiceConf) – The ChoiceConf instance used to create the choice.

• flags (set[str]) – A set of command-line flags associated with the choice.

• flag_type (FlagType) – The type of command-line flag (e.g., boolean, target_list, etc.).

• name (str) – A unique name for the choice.

• description (str) – A brief description of the choice.

• sections (set[Section]) – A set of Section enums to include in the report.

• targets (set[Target]) – A set of Target enums for output.

• default_targets (set[Target] | None) – A set of Target enums representing the default targets for the choice. If None, no default targets are specified and the reporter’s defaults will be used when generating reports.

• output_format (Format) – A Format instance describing the output format.

• subdir (str | None) – An optional subdirectory for output files. If None, reports default to the reporter’s subdir.

• file_suffix (str | None) – An optional file suffix for output files. If None, reports default to the reporter’s file_suffix.

• file_unique (bool | None) – Whether to make output file names unique. If None, reports default to the reporter’s file_unique.

• file_append (bool | None) – Whether to append to existing output files. If None, reports default to the reporter’s file_append.

• options (ReporterOptions | None) – An optional ReporterOptions instance for additional configurations specific to a specific reporter. If None, reports default to the reporter’s default options.

• extra (Any | None) – Any additional metadata associated with the choice.

Construct a Choice instance from a Reporter and a ChoiceConf instance.

Parameters:

• reporter (Reporter) – An instance of a Reporter subclass.

• choice_conf (ChoiceConf) – An instance of ChoiceConf containing the configuration for the choice.

Raises:

• SimpleBenchTypeError – If any argument is of an incorrect type.

• SimpleBenchValueError – If any argument has an invalid value (e.g., empty strings or empty sequences).

property choice_conf: ChoiceConf

The ChoiceConf instance used to create the choice.

property default_targets: frozenset[Target] | None

Default output targets for the choice.

These are the default output targets that the associated Reporter subclass should use when this choice is selected, if no specific target

is provided by the user.

property description: str

Description of the choice.

This is used as help text for the command-line flags associated with the choice.

property extra: Any

Any additional metadata associated with the choice.

property file_append: bool | None

Whether to append to existing output files.

If specified, the associated Reporter subclass should use this setting for output files when this choice is selected. If None, the reporter’s default file_append setting should be used.

property file_suffix: str | None

An optional file suffix for output files.

If specified, the associated Reporter subclass should use this file suffix for output files when this choice is selected. If None, the reporter’s default file_suffix should be used.

property file_unique: bool | None

Whether to make output file names unique.

If specified, the associated Reporter subclass should use this setting for output files when this choice is selected. If None, the reporter’s default file_unique setting should be used.

property flag_type: FlagType

The type of command-line flag (e.g., BOOLEAN, TARGET_LIST, etc.).

property flags: frozenset[str]

Flags associated with the choice. These are used for command-line selection. They must be unique across all choices for all reporters. This is enforced by the ReporterManager when choices are registered.

The flags should be in the format used on the command line, typically starting with -- for long options.

['--json', '--json-full']

The description property of the Choice is used to provide help text for the flags when generating command-line help.

property name: str

Name of the choice.

property options: ReporterOptions | None

An optional ReporterOptions instance for additional configuration.

property output_format: Format

Output format for the choice.

This is the output format that the associated Reporter subclass is expected to use when this choice is selected.

Returns:

The output format.

Return type:

Format

property reporter: Reporter

The Reporter sub-class associated with the choice.

property sections: frozenset[Section]

Sections included in the choice.

These are the sections that the associated Reporter subclass is expected to include in its report when this choice is selected.

property subdir: str | None

An optional subdirectory for output files.

If specified, the associated Reporter subclass should use this subdirectory for output files when this choice is selected. If an empty string, no subdirectory should be used. If None, the reporter’s default subdir should be used.

property targets: frozenset[Target]

Output targets for the choice.

These are the output targets that the associated Reporter subclass is expected to use when this choice is selected.

class simplebench.reporters.choice.ChoiceConf(

*,

flags: Sequence[str],

flag_type: FlagType,

name: str,

description: str,

sections: Iterable[Section],

output_format: Format,

targets: Iterable[Target],

default_targets: Iterable[Target] | None = None,

subdir: str | None = None,

file_suffix: str | None = None,

file_unique: bool | None = None,

file_append: bool | None = None,

options: ReporterOptions | None = None,

extra: Any | None = None,

)

Bases: Hashable, ChoiceProtocol

Definition of a Choice configuration for reporters.

A ChoiceConf represents a specific configuration of an implied Choice that can be registered with a Reporter subclass. It defines the sections to include in the report, the output targets, the output format, and various other options related to reporting.

The ChoiceConf class provides a structured way to define different reporting options within the SimpleBench framework so that users can select from predefined configurations and developers can easily add new reporting options to the framework.

A ChoiceConf instance is immutable after creation to ensure consistency in reporting configurations.

The sections, targets, and formats are descriptive only; they do not enforce any behavior on the associated Reporter subclass. It is the responsibility of the Reporter subclass to implement the behavior corresponding to the specified sections, targets, and formats.

The ChoiceConf is intended to be used in defining configurations for reporters without depending on being embedded in the __init__() method for definition. This allows for more flexible and reusable reporter configurations. The live counterpart to ChoiceConf is the Choice class, which is used at runtime with actual Choice instances created by Reporter from ChoiceConf definitions.

Parameters:

• flags (set[str]) – A set of command-line flags associated with the choice.

• flag_type (FlagType) – The type of command-line flag (e.g., boolean, target_list, etc.).

• name (str) – A unique name for the choice.

• description (str) – A brief description of the choice.

• sections (set[Section]) – A set of Section enums to include in the report.

• targets (set[Target]) – A set of Target enums for output.

• default_targets (set[Target] | None) – A set of Target enums representing the default targets for the choice. If None, no default targets are specified and the reporter’s defaults will be used when generating reports.

• output_format (Format) – A Format instance describing the output format.

• subdir (str | None) – An optional subdirectory for output files. If None, reports default to the reporter’s subdir.

• file_suffix (str | None) – An optional file suffix for output files. If None, reports default to the reporter’s file_suffix.

• file_unique (bool | None) – Whether to make output file names unique. If None, reports default to the reporter’s file_unique.

• file_append (bool | None) – Whether to append to existing output files. If None, reports default to the reporter’s file_append.

• options (ReporterOptions | None) – An optional ReporterOptions instance for additional configurations specific to a specific reporter. If None, reports default to the reporter’s default options.

• extra (Any | None) – Any additional metadata associated with the choice.

Construct a ChoiceConf instance.

Parameters:

• flags (Iterable[str]) – An iterable of command-line flags associated with the choice.

• flag_type (FlagType) – The type of command-line flag (e.g., boolean, target_list, etc.).

• name (str) – A unique name for the choice.

• description (str) – A brief description of the choice.

• sections (Iterable[Section]) – An iterable of Section enums to include in the report. It must be non-empty, but NULL may be included to indicate no sections are specifically selected. The reporter is expected to include listed sections in its report.

• output_format (Format) – A Format instance describing the output format.

• targets (Iterable[Target]) – An iterable of Target enums for output. It must be non-empty. If multiple targets are specified, the reporter is expected to handle outputting to all specified targets when this choice is selected.

• default_targets (Iterable[Target] | None) – An optional iterable of default Target enums. The enums represent the default targets for the choice. If None, no default targets are specified and the reporter’s defaults will be used when generating reports.

• subdir (str | None) – An optional subdirectory for output files. If None, defaults to the reporter’s default subdir. It may only consist of alphanumeric characters (a-z, A-Z, 0-9) or be an empty string (to indicate no subdirectory). It cannot be longer than 64 characters and may be left as None to use the reporter’s default.

• file_suffix (str | None) – An optional file suffix for output files. If None, defaults to the reporter’s default file_suffix when generating reports. It may only consist of alphanumeric characters (a-z, A-Z, 0-9), and be no longer than 10 characters. It may be left as None to use the reporter’s default.

• file_unique (bool | None) – Whether to make output file names unique by appending a unique identifier. Mutually exclusive with file_append; both cannot be True or False at the same time. If None, defaults to the reporter’s default file_unique setting.

• file_append (bool | None) – Whether to append to existing output files instead of overwriting them. Mutually exclusive with file_unique; both cannot be True or False at the same time. If None, defaults to the reporter’s default file_append setting.

• options (ReporterOptions | None) – An optional ReporterOptions instance for additional configuration specific to a reporter. The option must be of the same type as that specified by the options_type property of the associated Reporter subclass.

• extra (Any) – Any additional metadata associated with the choice. This can be used to store custom information relevant to the choice and the core benchmarking framework does not interpret or enforce any structure on this data. Reporter subclasses may choose to utilize this field for their own purposes.

Raises:

• SimpleBenchTypeError – If any argument is of an incorrect type.

• SimpleBenchValueError – If any argument has an invalid value (e.g., empty strings or empty sequences).

property default_targets: frozenset[Target] | None

Default output targets for the choice.

These are the default output targets that the associated Reporter subclass should use when this choice is selected, if no specific target

is provided by the user.

property description: str

Description of the choice.

This is used as help text for the command-line flags associated with the choice.

property extra: Any

Any additional metadata associated with the choice.

property file_append: bool | None

Whether to append to existing output files.

If specified, the associated Reporter subclass should use this setting for output files when this choice is selected. If None, the reporter’s default file_append setting should be used.

property file_suffix: str | None

An optional file suffix for output files.

If specified, the associated Reporter subclass should use this file suffix for output files when this choice is selected. If None, the reporter’s default file_suffix should be used.

property file_unique: bool | None

Whether to make output file names unique.

If specified, the associated Reporter subclass should use this setting for output files when this choice is selected. If None, the reporter’s default file_unique setting should be used.

property flag_type: FlagType

The type of command-line flag (e.g., BOOLEAN, TARGET_LIST, etc.).

property flags: frozenset[str]

Flags associated with the choice. These are used for command-line selection. They must be unique across all choices for all reporters. This is enforced by the ReporterManager when choices are registered.

The flags should be in the format used on the command line, typically starting with -- for long options.

['--json', '--json-full']

The description property of the Choice is used to provide help text for the flags when generating command-line help.

property name: str

Name of the choice.

property options: ReporterOptions | None

An optional ReporterOptions instance for additional configuration.

property output_format: Format

Output format for the choice.

This is the output format that the associated Reporter subclass is expected to use when this choice is selected.

Returns:

The output format.

Return type:

Format

property sections: frozenset[Section]

Sections included in the choice.

These are the sections that the associated Reporter subclass is expected to include in its report when this choice is selected.

property subdir: str | None

An optional subdirectory for output files.

If specified, the associated Reporter subclass should use this subdirectory for output files when this choice is selected. If an empty string, no subdirectory should be used. If None, the reporter’s default subdir should be used.

property targets: frozenset[Target]

Output targets for the choice.

These are the output targets that the associated Reporter subclass is expected to use when this choice is selected.

Subpackages

• simplebench.reporters.choice.exceptions package
  • Submodules
    • simplebench.reporters.choice.exceptions.choice module
    • simplebench.reporters.choice.exceptions.choice_conf module

Submodules

• simplebench.reporters.choice.choice module
  • Choice
    • Choice.choice_conf
    • Choice.default_targets
    • Choice.description
    • Choice.extra
    • Choice.file_append
    • Choice.file_suffix
    • Choice.file_unique
    • Choice.flag_type
    • Choice.flags
    • Choice.name
    • Choice.options
    • Choice.output_format
    • Choice.reporter
    • Choice.sections
    • Choice.subdir
    • Choice.targets
  • deferred_reporter_import()
• simplebench.reporters.choice.choice_conf module
  • ChoiceConf
    • ChoiceConf.default_targets
    • ChoiceConf.description
    • ChoiceConf.extra
    • ChoiceConf.file_append
    • ChoiceConf.file_suffix
    • ChoiceConf.file_unique
    • ChoiceConf.flag_type
    • ChoiceConf.flags
    • ChoiceConf.name
    • ChoiceConf.options
    • ChoiceConf.output_format
    • ChoiceConf.sections
    • ChoiceConf.subdir
    • ChoiceConf.targets