simplebench.stats package🔗

Stats module for SimpleBench benchmarking framework.

class simplebench.stats.MemoryUsage( *, iterations: Sequence[Iteration] | None = None, unit: str = 'bytes', scale: float = 1.0, rounds: int = 1, data: Sequence[int | float] | None = None, )[source]🔗

Bases: Stats

Container for the memory usage statistics of a benchmark.

Variables:

unit (str) – The unit of measurement for the memory usage (e.g., “MB”).
scale (float) – The scale factor for the memory usage (e.g., “1e6” for megabytes).
rounds (int) – The number of data points in the benchmark.
data (tuple[float | int, ...]) – Tuple of memory usage data points.
mean (float) – The mean memory usage.
median (float) – The median memory usage.
minimum (float) – The minimum memory usage.
maximum (float) – The maximum memory usage.
standard_deviation (float) – The standard deviation of the memory usage.
relative_standard_deviation (float) – The relative standard deviation of the memory usage.
percentiles (dict[int, float]) – Percentiles of memory usage.

Construct MemoryUsage stats from sequence of Iteration or raw memory data.

At least one of iterations or data must be provided.

If provided, iterations must be a sequence of Iteration objects and memory data will be extracted from each memory attribute. If data is also provided, the memory data extracted from the iterations will be appended to the provided data.

Parameters:

iterations – Optional list of Iteration objects to extract memory data from.
unit – Optional unit of measurement for the memory usage (e.g., “MB”). Defaults to ‘bytes’.
scale – Optional scale factor for the memory usage (e.g., “1e6” for megabytes). Defaults to 1.0.
rounds – The number of data points in the benchmark. Defaults to 1.
data – Optional list of memory usage data points. If not provided, memory data will be extracted from the iterations if available.

Raises:

SimpleBenchTypeError – If any of the arguments are of the wrong type.
SimpleBenchValueError – If any of the arguments have invalid values.

class simplebench.stats.MemoryUsageSummary( *, unit: str, scale: float, rounds: int, mean: float, median: float, minimum: float, maximum: float, standard_deviation: float, relative_standard_deviation: float, percentiles: tuple[float, ...], )[source]🔗

Bases: StatsSummary

Container for summary statistics of a MemoryUsage benchmark.

This class is exclusive of raw data points.

Variables:

unit (str) – The unit of measurement for the benchmark (e.g., “ops/s”).
scale (float) – The scale factor for the interval (e.g. 1 for seconds).
rounds (int) – The number of data points in the benchmark.
data (tuple[int | float, ...]) – Always an empty tuple as a StatsSummary object does not contain raw data points.
mean (float) – The mean operations per time interval.
median (float) – The median operations per time interval.
minimum (float) – The minimum operations per time interval.
maximum (float) – The maximum operations per time interval.
standard_deviation (float) – The standard deviation of operations per time interval.
relative_standard_deviation (float) – The relative standard deviation of ops per time interval.
percentiles (tuple[float, ...]) – Percentiles of operations per time interval.

Initialize the StatsSummary object.

Parameters:

unit (str) – The unit of measurement for the data (e.g., “ops/s”).
scale (float) – The scale factor the data (e.g. 1.0 for seconds).
rounds (int) – The number of rounds each data point represents.
mean (float) – The mean data point.
median (float) – The median data point.
minimum (float) – The minimum data point.
maximum (float) – The maximum data point.
standard_deviation (float) – The standard deviation of data.
relative_standard_deviation (float) – The relative standard deviation of data.
percentiles (tuple[float, ...]) – Percentiles of data.

Raises:

SimpleBenchTypeError – If any of the arguments are of the wrong type.
SimpleBenchValueError – If any of the arguments have invalid values.

class simplebench.stats.OperationTimings( *, iterations: Sequence[Iteration] | None = None, unit: str = 'ns', scale: float = 1e-09, rounds: int = 1, data: Sequence[int | float] | None = None, )[source]🔗

Bases: Stats

Container for the operation timing statistics of a benchmark.

Variables:

unit (str) – The unit of measurement for the timings (e.g., “ns”).
scale (float) – The scale factor for the timings (e.g., “1e-9” for nanoseconds).
rounds (int) – The number of data points in the benchmark.
data (tuple[int | float, ...]) – Tuple of timing data points.
mean (float) – The mean time per operation.
median (float) – The median time per operation.
minimum (float) – The minimum time per operation.
maximum (float) – The maximum time per operation.
standard_deviation (float) – The standard deviation of the time per operation.
relative_standard_deviation (float) – The relative standard deviation of the time per operation.
percentiles (dict[int, float]) – Percentiles of time per operation.

Construct OperationTimings stats from Iteration or raw timing data.

Parameters:

iterations – Sequence of Iteration objects to extract timing data from.
unit – The unit of measurement for the timings (e.g., “ns”).
scale – The scale factor for the timings (e.g., “1e-9” for nanoseconds).
rounds – The number of data points in the benchmark.
data – Optional sequence of timing data points. If not provided, timing data will be extracted from the iterations if available.

Raises:

SimpleBenchTypeError – If any of the arguments are of the wrong type.
SimpleBenchValueError – If any of the arguments have invalid values.

class simplebench.stats.OperationTimingsSummary( *, unit: str, scale: float, rounds: int, mean: float, median: float, minimum: float, maximum: float, standard_deviation: float, relative_standard_deviation: float, percentiles: tuple[float, ...], )[source]🔗

Bases: StatsSummary

Container for summary of operation timing statistics of a benchmark.

Variables:

unit (str) – The unit of measurement for the benchmark (e.g., “ops/s”).
scale (float) – The scale factor for the interval (e.g. 1 for seconds).
rounds (int) – The number of data points in the benchmark.
mean (float) – The mean operations per time interval.
median (float) – The median operations per time interval.
minimum (float) – The minimum operations per time interval.
maximum (float) – The maximum operations per time interval.
standard_deviation (float) – The standard deviation of operations per time interval.
relative_standard_deviation (float) – The relative standard deviation of ops per time interval.
percentiles (dict[int, float]) – Percentiles of operations per time interval.

Initialize the StatsSummary object.

Parameters:

unit (str) – The unit of measurement for the data (e.g., “ops/s”).
scale (float) – The scale factor the data (e.g. 1.0 for seconds).
rounds (int) – The number of rounds each data point represents.
mean (float) – The mean data point.
median (float) – The median data point.
minimum (float) – The minimum data point.
maximum (float) – The maximum data point.
standard_deviation (float) – The standard deviation of data.
relative_standard_deviation (float) – The relative standard deviation of data.
percentiles (tuple[float, ...]) – Percentiles of data.

Raises:

SimpleBenchTypeError – If any of the arguments are of the wrong type.
SimpleBenchValueError – If any of the arguments have invalid values.

class simplebench.stats.OperationsPerInterval( *, iterations: Sequence[Iteration] | None = None, unit: str = 'Ops/s', scale: float = 1.0, rounds: int = 1, data: Sequence[int | float] | None = None, )[source]🔗

Bases: Stats

Container for the operations per time interval statistics of a benchmark.

Variables:

unit (str) – The unit of measurement for the benchmark (e.g., “ops/s”).
scale (float) – The scale factor for the interval (e.g. 1 for seconds).
rounds (int) – The number of data points in the benchmark.
data (Sequence[int | float]) – List of data points.
mean (float) – The mean operations per time interval.
median (float) – The median operations per time interval.
minimum (float) – The minimum operations per time interval.
maximum (float) – The maximum operations per time interval.
standard_deviation (float) – The standard deviation of operations per time interval.
relative_standard_deviation (float) – The relative standard deviation of ops per time interval.
percentiles (dict[int, float]) – Percentiles of operations per time interval.

Construct OperationsPerInterval stats from Iteration or raw ops data.

Parameters:

iterations – List of Iteration objects to extract ops data from.
unit – The unit of measurement for the benchmark (e.g., “ops/s”).
scale – The scale factor for the interval (e.g. 1 for seconds).
rounds – The number of data points in the benchmark.
data – Optional list of ops data points. If not provided, ops data will be extracted from the iterations if available.

Raises:

SimpleBenchTypeError – If any of the arguments are of the wrong type.
SimpleBenchValueError – If any of the arguments have invalid values.

class simplebench.stats.OperationsPerIntervalSummary( *, unit: str, scale: float, rounds: int, mean: float, median: float, minimum: float, maximum: float, standard_deviation: float, relative_standard_deviation: float, percentiles: tuple[float, ...], )[source]🔗

Bases: StatsSummary

Summary of OperationsPerInterval statistics.

Variables:

unit (str) – The unit of measurement for the benchmark (e.g., “ops/s”).
scale (float) – The scale factor for the interval (e.g. 1 for seconds).
rounds (int) – The number of data points in the benchmark.
mean (float) – The mean operations per time interval.
median (float) – The median operations per time interval.
minimum (float) – The minimum operations per time interval.
maximum (float) – The maximum operations per time interval.
standard_deviation (float) – The standard deviation of operations per time interval.
relative_standard_deviation (float) – The relative standard deviation of ops per time interval.
percentiles (dict[int, float]) – Percentiles of operations per time interval.

Initialize the StatsSummary object.

Parameters:

unit (str) – The unit of measurement for the data (e.g., “ops/s”).
scale (float) – The scale factor the data (e.g. 1.0 for seconds).
rounds (int) – The number of rounds each data point represents.
mean (float) – The mean data point.
median (float) – The median data point.
minimum (float) – The minimum data point.
maximum (float) – The maximum data point.
standard_deviation (float) – The standard deviation of data.
relative_standard_deviation (float) – The relative standard deviation of data.
percentiles (tuple[float, ...]) – Percentiles of data.

Raises:

SimpleBenchTypeError – If any of the arguments are of the wrong type.
SimpleBenchValueError – If any of the arguments have invalid values.

class simplebench.stats.PeakMemoryUsage( *, iterations: Sequence[Iteration] | None = None, unit: str = 'bytes', scale: float = 1.0, rounds: int = 1, data: Sequence[int | float] | None = None, )[source]🔗

Bases: Stats

Container for the peak memory usage statistics of a benchmark.

Variables:

unit (str) – The unit of measurement for the memory usage (e.g., “MB”).
scale (float) – The scale factor for the memory usage (e.g., “1e6” for megabytes).
rounds (int) – The number of data points in the benchmark.
data (tuple[int | float, ...]) – Tuple of peak memory usage data points.
mean (float) – The mean memory usage.
median (float) – The median memory usage.
minimum (float) – The minimum memory usage.
maximum (float) – The maximum memory usage.
standard_deviation (float) – The standard deviation of the memory usage.
relative_standard_deviation (float) – The relative standard deviation of the memory usage.
percentiles (dict[int, float]) – Percentiles of memory usage.

Construct PeakMemoryUsage stats from Iteration or raw memory data.

Parameters:

iterations – Sequence of Iteration objects to extract peak memory data from.
unit – The unit of measurement for the memory usage (e.g., “MB”).
scale – The scale factor for the memory usage (e.g., “1e6” for megabytes).
rounds – The number of data points in the benchmark.
data – Optional Sequence of peak memory usage data points. If not provided, peak memory data will be extracted from the iterations if available.

Raises:

SimpleBenchTypeError – If any of the arguments are of the wrong type.
SimpleBenchValueError – If any of the arguments have invalid values.

class simplebench.stats.PeakMemoryUsageSummary( *, unit: str, scale: float, rounds: int, mean: float, median: float, minimum: float, maximum: float, standard_deviation: float, relative_standard_deviation: float, percentiles: tuple[float, ...], )[source]🔗

Bases: StatsSummary

Container for the summary of peak memory usage statistics of a benchmark.

Variables:

unit (str) – The unit of measurement for the memory usage (e.g., “MB”).
scale (float) – The scale factor for the memory usage (e.g., “1e6” for megabytes).
rounds (int) – The number of data points in the benchmark.
mean (float) – The mean memory usage.
median (float) – The median memory usage.
minimum (float) – The minimum memory usage.
maximum (float) – The maximum memory usage.
standard_deviation (float) – The standard deviation of the memory usage.
relative_standard_deviation (float) – The relative standard deviation of the memory usage.
percentiles (dict[int, float]) – Percentiles of memory usage.

Initialize the StatsSummary object.

Parameters:

unit (str) – The unit of measurement for the data (e.g., “ops/s”).
scale (float) – The scale factor the data (e.g. 1.0 for seconds).
rounds (int) – The number of rounds each data point represents.
mean (float) – The mean data point.
median (float) – The median data point.
minimum (float) – The minimum data point.
maximum (float) – The maximum data point.
standard_deviation (float) – The standard deviation of data.
relative_standard_deviation (float) – The relative standard deviation of data.
percentiles (tuple[float, ...]) – Percentiles of data.

Raises:

SimpleBenchTypeError – If any of the arguments are of the wrong type.
SimpleBenchValueError – If any of the arguments have invalid values.

class simplebench.stats.Stats( *, unit: str, scale: float, data: Sequence[int | float], rounds: int = 1, )[source]🔗

Bases: object

Generic container for statistics on a benchmark.

Variables:

unit (str) – The unit of measurement for the benchmark (e.g., “ops/s”). (read only)
scale (float) – The scale factor for the interval (e.g. 1 for seconds). (read only)
rounds (int) – The number of rounds each data point represents. (read only)
data (tuple[float | int, ...]) – Tuple of data points. (read only)
mean (float) – The mean operations per time interval. (read only)
median (float) – The median operations per time interval. (read only)
minimum (float) – The minimum operations per time interval. (read only)
maximum (float) – The maximum operations per time interval. (read only)
standard_deviation (float) – The standard deviation of operations per time interval. (read only)
relative_standard_deviation (float) – The relative standard deviation of ops per time interval. (read only)
percentiles (tuple[float, ...]) – Percentiles of operations per time interval. (read only)

Initialize the Stats object.

Parameters:

unit (str) – The unit of measurement for the benchmark (e.g., “ops/s”).
scale (float) – The scale factor for the interval (e.g. 1 for seconds).
data (Sequence[int | float]) – Sequence of data points.
rounds (int) – The number of rounds each data point represents.

Raises:

SimpleBenchTypeError – If any of the arguments are of the wrong type.
SimpleBenchValueError – If any of the arguments have invalid values.

property as_dict: dict[str, str | float | dict[int, float] | tuple[int | float, ...]]🔗

Returns the statistics and data as a JSON-serializable dictionary.

This includes all the statistics as well as the raw data points.

The data values are scaled according to the scale factor to provide human-readable values using the base unit rather than the scaled unit.

The unit is normalized to its SI base unit representation. (e.g., “ms” becomes “s”)

The dictionary is mutability-safe as all data is either a primitive or a copy.

Returns:: A dictionary containing the statistics and the scaled data points.

property data: tuple[int | float, ...]🔗: The data points.

classmethod from_dict( data: dict[str, Any], ) → Stats[source]🔗

Construct a Stats object from a dictionary.

Example

stats_dict = {
    "unit": "ops/s",
    "scale": 1,
    "data": [1000, 2000, 1500, 3000, 2500]
}
stats = Stats.from_dict(stats_dict)
print(stats.mean)  # Output: 2000.0

Parameters:

data (dict) – A dictionary containing the stats data. Must contain ‘data’ key with a non-empty sequence of data points consisting of integers or floats.

Returns:

A Stats object constructed from the provided dictionary.

Raises:

SimpleBenchTypeError – If the data, unit, or scale arguments are of the wrong type.
SimpleBenchKeyError – If the data dictionary does not contain a ‘unit’ key and no unit argument is provided.
SimpleBenchValueError – If the data dictionary does not contain a non-empty ‘data’ key with at least one data point, if the scale argument is not greater than zero, or if the unit argument is an empty string

property maximum: float🔗: The maximum of the data.

property mean: float🔗: The mean of the data.

property median: float🔗: The median of the data.

property minimum: float🔗: The minimum of the data.

property percentiles: tuple[float, ...]🔗

Percentiles of the data.

Returns the 0th through 100th percentiles of the data as an immutable tuple.

property relative_standard_deviation: float🔗: The relative standard deviation of the data.

property rounds: int🔗: The number of rounds each data point represents.

property scale: float🔗: The scale of the data.

property standard_deviation: float🔗: The standard deviation of the data.

property stats_summary: StatsSummary🔗

Returns a StatsSummary object created from this Stats object.

Returns:: A StatsSummary object containing the same statistics as this Stats object.

property unit: str🔗: The unit of the data.

class simplebench.stats.StatsSummary( *, unit: str, scale: float, rounds: int, mean: float, median: float, minimum: float, maximum: float, standard_deviation: float, relative_standard_deviation: float, percentiles: tuple[float, ...], )[source]🔗

Bases: object

Container for summary statistics of a benchmark, exclusive of raw data points.

This is a lightweight, data-only version of the Stats class, suitable for serialization or reporting when raw data points are not needed.

Variables:

unit (str) – The unit of measurement for the benchmark (e.g., “ops/s”). (read only)
scale (float) – The scale factor for the interval (e.g. 1 for seconds). (read only)
rounds (int) – The number of rounds each data point represents. (read only)
mean (float) – The mean operations per time interval. (read only)
median (float) – The median operations per time interval. (read only)
minimum (float) – The minimum operations per time interval. (read only)
maximum (float) – The maximum operations per time interval. (read only)
standard_deviation (float) – The standard deviation of operations per time interval. (read only)
relative_standard_deviation (float) – The relative standard deviation of ops per time interval. (read only)
percentiles (tuple[float, ...]) – Percentiles of operations per time interval. (read only)
data (tuple[int | float, ...]) – Always an empty tuple as StatsSummary does not contain raw data points. (read only)

Initialize the StatsSummary object.

Parameters:

unit (str) – The unit of measurement for the data (e.g., “ops/s”).
scale (float) – The scale factor the data (e.g. 1.0 for seconds).
rounds (int) – The number of rounds each data point represents.
mean (float) – The mean data point.
median (float) – The median data point.
minimum (float) – The minimum data point.
maximum (float) – The maximum data point.
standard_deviation (float) – The standard deviation of data.
relative_standard_deviation (float) – The relative standard deviation of data.
percentiles (tuple[float, ...]) – Percentiles of data.

Raises:

SimpleBenchTypeError – If any of the arguments are of the wrong type.
SimpleBenchValueError – If any of the arguments have invalid values.

property as_dict: dict[str, str | float | dict[int, float] | tuple[int | float, ...]]🔗

Returns the statistics as a JSON-serializable dictionary.

The data values are scaled according to the scale factor to provide human-readable values using the base unit rather than the scaled unit.

The unit is converted to its SI base unit representation. (e.g., “ms” becomes “s”)

This does not include raw data points, only the statistics.

The dictionary is mutability-safe as all data is either a primitive or a copy.

Returns:: A dictionary containing the statistics.

property data: tuple[int | float, ...]🔗

The data points.

This is always an empty tuple as a StatsSummary does not contain raw data points.

classmethod from_dict( data: dict[str, Any], ) → StatsSummary[source]🔗

Construct a StatsSummary object from a dictionary.

Example

stats_summary_dict = {
    "unit": "ops/s",
    "scale": 1.0,
    "rounds": 1,
    "mean": 2000.0,
    "median": 2000.0,
    "minimum": 1000.0,
    "maximum": 3000.0,
    "standard_deviation": 790.5694150420949,
    "relative_standard_deviation": 39.52847075252201,
    "percentiles": [1000.0, 1300.0, 1600.0, 1900.0, 2200.0,
                    2500.0, 2800.0, 3000.0, 3000.0]
}
stats_summary = StatsSummary.from_dict(stats_summary_dict)
print(stats_summary.mean)  # Output: 2000.0

Parameters:

data (dict) – A dictionary containing the stats data. Must contain ‘data’ key with a non-empty sequence of data points consisting of integers or floats.

Returns:

A StatsSummary object constructed from the provided dictionary.

Raises:

SimpleBenchTypeError – If the data, unit, or scale arguments are of the wrong type.
SimpleBenchKeyError – If the data dictionary does not contain a ‘unit’ key and no unit argument is provided.
SimpleBenchValueError – If the data dictionary does not contain a non-empty ‘data’ key with at least one data point, if the scale argument is not greater than zero, or if the unit argument is an empty string

classmethod from_stats( stats: Stats, ) → StatsSummary[source]🔗

Construct a new StatsSummary object from a Stats object.

Parameters:: stats (Stats) – The Stats object to derive the summary from.
Returns:: A new StatsSummary object containing the same statistics as the provided Stats object.
Raises:: SimpleBenchTypeError – If the stats argument is not a Stats object.

property maximum: float🔗: The maximum of the data.

property mean: float🔗: The mean of the data.

property median: float🔗: The median of the data.

property minimum: float🔗: The minimum of the data.

property percentiles: tuple[float, ...]🔗: Percentiles of the data.

property relative_standard_deviation: float🔗: The relative standard deviation of the data.

property rounds: int🔗: The number of rounds each data point represents.

property scale: float🔗: The scale of the data.

property standard_deviation: float🔗: The standard deviation of the data.

property unit: str🔗: The unit of the data.

Subpackages🔗

simplebench.stats.exceptions package
- Submodules

simplebench.stats package🔗

Subpackages🔗

Submodules🔗