simplebench.reporters.csv package🔗

CSV reporter package for simplebench.

Purpose is to provide a CSV reporter for the simplebench.reporters package. This package provides functionality for generating CSV reports from benchmark results.

Public API🔗

  • CSVField: Enum defining the fields available for CSV reporting.

  • CSVOptions: Options class for the CSV reporter.

  • CSVReporter: The CSV reporter class.

  • CSVConfig: Configuration class for the CSV reporter.

class simplebench.reporters.csv.CSVConfig(
*,
name: str | None = None,
description: str | None = None,
sections: Iterable[Section] | None = None,
targets: Iterable[Target] | None = None,
default_targets: Iterable[Target] | None = None,
formats: Iterable[Format] | None = None,
choices: ChoicesConf | None = None,
file_suffix: str | None = None,
file_unique: bool | None = None,
file_append: bool | None = None,
subdir: str | None = None,
)[source]🔗

Bases: ReporterConfig

Configuration for a CSVReporter.

This class inherits from ReporterConfig and provides a type-safe, discoverable interface for overriding the default settings of a CSVReporter.

By default, the CSVReporter is configured to output benchmark results to CSV files in the filesystem, with options to also output to console and via callback. The default sections included are OPS, TIMING, MEMORY, and PEAK_MEMORY.

Initialize the CSVReporter configuration.

Accepts keyword arguments to override any of the default configurations. All arguments are optional. If not provided, the default value for CSVReporter will be used.

class simplebench.reporters.csv.CSVField(value)[source]🔗

Bases: str, Enum

Fields available for CSV reporter output.

ELAPSED_SECONDS = 'Elapsed Seconds'🔗

The total elapsed time in seconds.

ITERATIONS = 'Iterations'🔗

The number of iterations performed.

MAX = 'max'🔗

The statistical maximum value.

MEAN = 'mean'🔗

The statistical mean value.

MEDIAN = 'median'🔗

The statistical median value.

MIN = 'min'🔗

The statistical minimum value.

N = 'N'🔗

The O() complexity value.

P5 = '5th'🔗

The statistical 5th percentile.

P95 = '95th'🔗

The statistical 95th percentile.

ROUNDS = 'Rounds'🔗

The number of rounds performed.

RSD_PERCENT = 'rsd%'🔗

The relative standard deviation percentage.

STD_DEV = 'std dev'🔗

The adjusted standard deviation of operation times.

class simplebench.reporters.csv.CSVOptions(
fields: Sequence[CSVField] | None = None,
variation_cols_last: bool = False,
)[source]🔗

Bases: ReporterOptions

Class for holding CSV reporter specific options in a Choice.

This class provides additional configuration options specific to the JSON reporter. It is accessed via the options attribute of a Choice instance.

Parameters:

  • fields –

    A tuple of CSV fields to include in the output. If none is specifically set, a predefined set of fields is used. The fields appear in the order specified in the sequence.

    If specified, all fields must be from the CSVField enum.

    The default fields, in order, are:

  • variation_cols_last – Whether to place the variation columns (if any) at the end of the rows. Defaults to False - which places the variation columns at the start of the rows.

Raises:

Initialize CSVOptions instance.

property fields: tuple[CSVField, ...]🔗

Return the fields, in order, to include in the CSV table when rendering.

Returns:

A tuple of CSVField enums representing the default fields.

property variation_cols_last: bool🔗

Return whether variation columns are placed at the end of the rows.

Returns:

True if variation columns are placed at the end, False if at the start.

class simplebench.reporters.csv.CSVReporter(
config: CSVConfig | None = None,
)[source]🔗

Bases: Reporter

Class for outputting benchmark results to CSV files.

It supports reporting statistics for various sections, either separately or together, to the filesystem, via a callback function, or to the console in CSV format.

The CSV files are tagged with metadata comments including the case title, description, and units for clarity.

Defined command-line flags:

–csv: {file, console, callback} (default=file) Outputs results to CSV.

program.py --csv               # Outputs results to CSV files in the filesystem (default).
program.py --csv filesystem    # Outputs results to CSV files in the filesystem.
program.py --csv console       # Outputs results to the console in CSV format.
program.py --csv callback      # Outputs results via a callback function in CSV format.
program.py --csv filesystem console  # Outputs results to both CSV files and the console.
Variables:

  • name (str) – The unique identifying name of the reporter.

  • description (str) – A brief description of the reporter.

  • choices (Iterable[ChoicesConf]) – Iterable of ChoicesConf instances defining the reporter instance, CLI flags, ChoiceConf name, supported Section objects, supported output Target objects, and supported output Format for the reporter.

  • targets (set[Target]) – The supported output targets for the reporter.

  • formats (set[Format]) – The supported output formats for the reporter.

Initialize the CSVReporter.

Note

The exception documentation below refers to validation of subclass configuration class variables _OPTIONS_TYPE and _OPTIONS_KWARGS. These must be correctly defined in any subclass of CSVReporter to ensure proper functionality.

Parameters:

config (CSVConfig | None) – An optional configuration object to override default reporter settings. If not provided, default settings will be used.

Raises:
render(
*,
case: Case,
section: Section,
options: ReporterOptions,
) → str[source]🔗

Renders the benchmark results as tagged CSV data and returns it as a string.

Parameters:

  • case – The Case instance representing the benchmarked code.

  • section – The section to output (eg. OPS or TIMING).

  • options – The options for the CSV report.

Returns:

The benchmark results formatted as tagged CSV data.

Raises:

SimpleBenchValueError – If the specified section is unsupported.

Subpackages🔗