simplebench.cli module🔗

CLI script support for SimpleBench.

This module provides the main entry point for running benchmarks via a command-line interface (CLI).

It defines the main function, which sets up argument parsing, configures the benchmarking session, and executes the benchmarks based on user-specified options.

It provides functionality to list available benchmarks, select specific benchmarks to run, configure output verbosity, and specify output paths.

simplebench.cli.main(
benchmark_cases: Sequence[Case] | None = None,
*,
argv: list[str] | None = None,
extra_args: list[str] | None = None,
no_exit: bool = False,
) → ExitCode[source]🔗

Main entry point for running benchmarks via a command-line interface.

This function is responsible for setting up the command-line interface, parsing arguments, and executing the benchmark cases.

@benchmark decorated benchmark cases are automatically included and added to the list of benchmark cases passed to this function.

Call this function from a script or the command line to execute benchmarks.

If passed a list of Case instances via the benchmark_cases argument, those cases will be benchmarked in addition to any cases registered via the @benchmark decorator.

If argv is provided, it will be used as the list of command-line arguments instead of sys.argv.

If extra_args is provided, those arguments will be appended to the command-line arguments before parsing.

If the no_exit argument is set to True, the function will return the exit code instead of calling sys.exit(). This is useful for testing or embedding the CLI functionality in other applications.

Parameters:

  • benchmark_cases – A Sequence of SimpleBench.Case instances to be benchmarked.

  • argv – A list of command-line arguments to parse. If None, defaults to sys.argv.

  • extra_args – Additional command-line arguments to include.

  • no_exit – If True, the function will not call sys.exit() and will return the exit code instead.

Returns:

  • ExitCode.SUCCESS (0) on success

  • ExitCode.RUNTIME_ERROR (1) runtime errors during execution.

  • ExitCode.CLI_ARGUMENTS_ERROR (2) for errors during CLI argument processing

  • ExitCode.KEYBOARD_INTERRUPT (3) if interrupted by keyboard interrupt

  • ExitCode.BENCHMARK_TIMED_OUT (4) if a timeout occurs during execution of a benchmark.

  • ExitCode.BENCHMARK_ERROR (5) if an error occurs during execution of a benchmark.

Raises:

SimpleBenchTypeError – If the extra_args argument is not None or a list of strings or if argv is not None or a list of strings.