simplebench.session module

Session management for SimpleBench.

class simplebench.session.Session(

*,

cases: Sequence[Case] | None = None,

verbosity: Verbosity = Verbosity.NORMAL,

default_runner: type[SimpleRunner] | None = None,

args_parser: ArgumentParser | None = None,

show_progress: bool = False,

output_path: Path | None = None,

console: Console | None = None,

timer: Callable[[], int] | None = None,

)

Bases: object

Container for session related information while running benchmarks.

The session is responsible for managing benchmark cases, command line arguments, progress display, and report generation.

This makes it the primary orchestrator for running benchmarks and generating reports.

Container and orchestrator for session related information while running benchmarks.

Parameters:

• cases – A Sequence of benchmark cases for the session. If None, an empty list will be created. Defaults to None.

• verbosity – The verbosity level for console output. Defaults to Verbosity.NORMAL.

• default_runner – The default runner class to use for Cases that do not specify a runner. If None, the default SimpleRunner is used. Defaults to None.

• args_parser – The ArgumentParser instance for the session. If None, a new ArgumentParser will be automatically created. Defaults to None.

• show_progress – Whether to show progress bars during execution. Defaults to False.

• output_path – The output path for reports. Defaults to None.

• console – A Rich Console instance for displaying output. If None, a new Console will be automatically created. Defaults to None.

• timer – A callable that returns the current time for timing benchmarks. If None, a default timer simplebench.defaults.DEFAULT_TIMER (perf_counter_ns) will be used. Defaults to None.

Raises:

SimpleBenchTypeError – If the arguments are of the wrong type.

add_case(

case: Case,

) → None

Add a Case to the Cases for this session.

Parameters:

case – benchmark case to add to the Session

Raises:

SimpleBenchTypeError – If the value is not a Case instance.

add_reporter_flags() → None

Add the command line flags for all registered reporters to the session’s ArgumentParser.

Any conflicts in flag names with already declared ArgumentParser flags will have to be handled by the reporters themselves.

It has its own method so that a user can customize the ArgumentParser before or after adding the reporter flags as needed.

It also allows the user to unregister reporters before adding the reporter flags if they want to omit specific built-in reporters entirely.

It is called internally by parse_args() if it has not already been called and does not re-add the reporter flags if called again.

Raises:

SimpleBenchArgumentError – If there is a conflict or other error in reporter flag names.

property args: Namespace | None

The command line arguments for the session. This will be None until the parse_args() method has been called.

property args_parser: ArgumentParser

The ArgumentParser instance for the session.

property cases: tuple[Case]

Tuple of Cases for this session.

property console: Console

The Rich Console instance for displaying output.

property default_runner: type[SimpleRunner] | None

The session scoped default runner class to use for Cases that do not specify a runner.

extend_cases(

cases: Sequence[Case],

) → None

Extend the Cases for this session.

Parameters:

cases (Sequence[Case]) – Sequence of Cases to add to the Session

Raises:

SimpleBenchTypeError – If the value is not a Sequence of Cases.

property output_path: Path | None

The output path for reports.

parse_args(

args: Sequence[str] | None = None,

) → None

Parse the command line arguments using the session’s ArgumentParser.

This method parses the command line arguments and stores them in the session’s args property. By default, it parses the arguments from sys.argv. If args is provided, it will parse the arguments from the provided sequence of strings instead.

This can be used to customize the command line arguments for testing or other purposes.

It automatically calls add_reporter_flags() to ensure that reporter flags are added to the ArgumentParser if it has not already been called.

If you wish to customize the ArgumentParser before or after adding the reporter flags, you can do so by calling add_reporter_flags() before calling this method.

Parameters:

args (Sequence[str], optional) – A list of command line arguments to parse. If None, the arguments will be taken from sys.argv. Defaults to None.

Raises:

SimpleBenchTypeError – If the args_parser is not set.

property progress: Progress

The Rich Progress instance for displaying progress bars.

report() → None

Generate reports for all benchmark cases in the session.

report_keys() → list[str]

Get a list of report keys for all reports to be generated in this session.

This filters the report choices based on the command line arguments that were set and parsed when the session was created and returns a list of report keys for the reports that should be generated.

The report keys correspond to the command line flags/args defined in the Choices of the registered reporters.

Note

This method requires that the session’s args property has been set by calling parse_args() prior to invoking this method.

Otherwise it will return an empty list.

Returns:

A list of report keys for all reports to be generated in this session.

property reporter_manager: ReporterManager

Returns the ReporterManager instance for managing reporters.

The reporter manager handles the registration and discovery of all reporters available to the session. This allows you to customize reporting behavior by adding your own reporters or removing default ones.

Returns:

The ReporterManager instance for managing reporters.

Return type:

ReporterManager

run() → None

Run all benchmark cases in the session.

This method iterates over all Case instances in the session’s cases and invokes their run() method to execute the benchmarks.

If the parse_args() method has not been called prior to invoking this method, it will be called authomatically with no arguments to parse from sys.argv.

If you wish to customize argument parsing or run the session entirely programmatically without command line args, you should call parse_args() before calling this method.

Raises:

• SimpleBenchTimeoutError – If a benchmark case times out during execution.

• SimpleBenchBenchmarkError – If an error occurs during the execution of a benchmark.

property show_progress: bool

Whether to show progress bars during execution.

property tasks: RichProgressTasks

The RichProgressTasks instance for managing progress tasks.

property timer: Callable[[], int]

The timer function used for benchmarking.

property verbosity: Verbosity

The Verbosity level for this session.