simplebench.stats.stats module

Base benchmark statistics class.

class simplebench.stats.stats.Stats(

*,

unit: str,

scale: float,

data: Sequence[int | float],

rounds: int = 1,

)

Bases: object

Generic container for statistics on a benchmark.

Variables:

• unit (str) – The unit of measurement for the benchmark (e.g., “ops/s”). (read only)

• scale (float) – The scale factor for the interval (e.g. 1 for seconds). (read only)

• rounds (int) – The number of rounds each data point represents. (read only)

• data (tuple[float | int, ...]) – Tuple of data points. (read only)

• mean (float) – The mean operations per time interval. (read only)

• median (float) – The median operations per time interval. (read only)

• minimum (float) – The minimum operations per time interval. (read only)

• maximum (float) – The maximum operations per time interval. (read only)

• standard_deviation (float) – The standard deviation of operations per time interval. (read only)

• relative_standard_deviation (float) – The relative standard deviation of ops per time interval. (read only)

• percentiles (tuple[float, ...]) – Percentiles of operations per time interval. (read only)

Initialize the Stats object.

Parameters:

• unit (str) – The unit of measurement for the benchmark (e.g., “ops/s”).

• scale (float) – The scale factor for the interval (e.g. 1 for seconds).

• data (Sequence[int | float]) – Sequence of data points.

• rounds (int) – The number of rounds each data point represents.

Raises:

• SimpleBenchTypeError – If any of the arguments are of the wrong type.

• SimpleBenchValueError – If any of the arguments have invalid values.

property as_dict: dict[str, str | float | dict[int, float] | tuple[int | float, ...]]

Returns the statistics and data as a JSON-serializable dictionary.

This includes all the statistics as well as the raw data points.

The data values are scaled according to the scale factor to provide human-readable values using the base unit rather than the scaled unit.

The unit is normalized to its SI base unit representation. (e.g., “ms” becomes “s”)

The dictionary is mutability-safe as all data is either a primitive or a copy.

Returns:

A dictionary containing the statistics and the scaled data points.

property data: tuple[int | float, ...]

The data points.

classmethod from_dict(

data: dict[str, Any],

) → Stats

Construct a Stats object from a dictionary.

Example

stats_dict = {
    "unit": "ops/s",
    "scale": 1,
    "data": [1000, 2000, 1500, 3000, 2500]
}
stats = Stats.from_dict(stats_dict)
print(stats.mean)  # Output: 2000.0

Parameters:

data (dict) – A dictionary containing the stats data. Must contain ‘data’ key with a non-empty sequence of data points consisting of integers or floats.

Returns:

A Stats object constructed from the provided dictionary.

Raises:

• SimpleBenchTypeError – If the data, unit, or scale arguments are of the wrong type.

• SimpleBenchKeyError – If the data dictionary does not contain a ‘unit’ key and no unit argument is provided.

• SimpleBenchValueError – If the data dictionary does not contain a non-empty ‘data’ key with at least one data point, if the scale argument is not greater than zero, or if the unit argument is an empty string

property maximum: float

The maximum of the data.

property mean: float

The mean of the data.

property median: float

The median of the data.

property minimum: float

The minimum of the data.

property percentiles: tuple[float, ...]

Percentiles of the data.

Returns the 0th through 100th percentiles of the data as an immutable tuple.

property relative_standard_deviation: float

The relative standard deviation of the data.

property rounds: int

The number of rounds each data point represents.

property scale: float

The scale of the data.

property standard_deviation: float

The standard deviation of the data.

property stats_summary: StatsSummary

Returns a StatsSummary object created from this Stats object.

Returns:

A StatsSummary object containing the same statistics as this Stats object.

property unit: str

The unit of the data.

class simplebench.stats.stats.StatsSummary(

*,

unit: str,

scale: float,

rounds: int,

mean: float,

median: float,

minimum: float,

maximum: float,

standard_deviation: float,

relative_standard_deviation: float,

percentiles: tuple[float, ...],

)

Bases: object

Container for summary statistics of a benchmark, exclusive of raw data points.

This is a lightweight, data-only version of the Stats class, suitable for serialization or reporting when raw data points are not needed.

Variables:

• unit (str) – The unit of measurement for the benchmark (e.g., “ops/s”). (read only)

• scale (float) – The scale factor for the interval (e.g. 1 for seconds). (read only)

• rounds (int) – The number of rounds each data point represents. (read only)

• mean (float) – The mean operations per time interval. (read only)

• median (float) – The median operations per time interval. (read only)

• minimum (float) – The minimum operations per time interval. (read only)

• maximum (float) – The maximum operations per time interval. (read only)

• standard_deviation (float) – The standard deviation of operations per time interval. (read only)

• relative_standard_deviation (float) – The relative standard deviation of ops per time interval. (read only)

• percentiles (tuple[float, ...]) – Percentiles of operations per time interval. (read only)

• data (tuple[int | float, ...]) – Always an empty tuple as StatsSummary does not contain raw data points. (read only)

Initialize the StatsSummary object.

Parameters:

• unit (str) – The unit of measurement for the data (e.g., “ops/s”).

• scale (float) – The scale factor the data (e.g. 1.0 for seconds).

• rounds (int) – The number of rounds each data point represents.

• mean (float) – The mean data point.

• median (float) – The median data point.

• minimum (float) – The minimum data point.

• maximum (float) – The maximum data point.

• standard_deviation (float) – The standard deviation of data.

• relative_standard_deviation (float) – The relative standard deviation of data.

• percentiles (tuple[float, ...]) – Percentiles of data.

Raises:

• SimpleBenchTypeError – If any of the arguments are of the wrong type.

• SimpleBenchValueError – If any of the arguments have invalid values.

property as_dict: dict[str, str | float | dict[int, float] | tuple[int | float, ...]]

Returns the statistics as a JSON-serializable dictionary.

The data values are scaled according to the scale factor to provide human-readable values using the base unit rather than the scaled unit.

The unit is converted to its SI base unit representation. (e.g., “ms” becomes “s”)

This does not include raw data points, only the statistics.

The dictionary is mutability-safe as all data is either a primitive or a copy.

Returns:

A dictionary containing the statistics.

property data: tuple[int | float, ...]

The data points.

This is always an empty tuple as a StatsSummary does not contain raw data points.

classmethod from_dict(

data: dict[str, Any],

) → StatsSummary

Construct a StatsSummary object from a dictionary.

Example

stats_summary_dict = {
    "unit": "ops/s",
    "scale": 1.0,
    "rounds": 1,
    "mean": 2000.0,
    "median": 2000.0,
    "minimum": 1000.0,
    "maximum": 3000.0,
    "standard_deviation": 790.5694150420949,
    "relative_standard_deviation": 39.52847075252201,
    "percentiles": [1000.0, 1300.0, 1600.0, 1900.0, 2200.0,
                    2500.0, 2800.0, 3000.0, 3000.0]
}
stats_summary = StatsSummary.from_dict(stats_summary_dict)
print(stats_summary.mean)  # Output: 2000.0

Parameters:

data (dict) – A dictionary containing the stats data. Must contain ‘data’ key with a non-empty sequence of data points consisting of integers or floats.

Returns:

A StatsSummary object constructed from the provided dictionary.

Raises:

• SimpleBenchTypeError – If the data, unit, or scale arguments are of the wrong type.

• SimpleBenchKeyError – If the data dictionary does not contain a ‘unit’ key and no unit argument is provided.

• SimpleBenchValueError – If the data dictionary does not contain a non-empty ‘data’ key with at least one data point, if the scale argument is not greater than zero, or if the unit argument is an empty string

classmethod from_stats(

stats: Stats,

) → StatsSummary

Construct a new StatsSummary object from a Stats object.

Parameters:

stats (Stats) – The Stats object to derive the summary from.

Returns:

A new StatsSummary object containing the same statistics as the provided Stats object.

Raises:

SimpleBenchTypeError – If the stats argument is not a Stats object.

property maximum: float

The maximum of the data.

property mean: float

The mean of the data.

property median: float

The median of the data.

property minimum: float

The minimum of the data.

property percentiles: tuple[float, ...]

Percentiles of the data.

property relative_standard_deviation: float

The relative standard deviation of the data.

property rounds: int

The number of rounds each data point represents.

property scale: float

The scale of the data.

property standard_deviation: float

The standard deviation of the data.

property unit: str

The unit of the data.