fhelium.benchmarks.model

Data model for independent FHElium benchmark workloads and results.

ProgressCallback

data View source

ProgressCallback = Callable[[str], None]

BenchmarkRunner

data View source

BenchmarkRunner = Callable[['BenchmarkProfile', ProgressCallback], 'BenchmarkResult']

MetricDirection

type alias View source

MetricDirection = Literal['lower', 'higher', 'none']

BenchmarkProfile

class View source

BenchmarkProfile(name: str, description: str, parameters: Mapping[str, Any] = field(default_factory=dict))

Named parameter set exposed by the CLI and TUI.

parameters contains JSON-compatible defaults understood by the corresponding benchmark runner. with_overrides returns a new profile and does not mutate the registered definition.

Attributes

Name	Type	Default/value
name	str
description	str
parameters	Mapping[str, Any]	field(default_factory=dict)

with_overrides

method

def with_overrides(overrides: Mapping[str, Any]) -> BenchmarkProfile: ...

BenchmarkDefinition

class View source

BenchmarkDefinition(name: str, title: str, description: str, profiles: tuple[BenchmarkProfile, ...], runner: BenchmarkRunner, workload_id: str, category: str = 'workload', requirements: Mapping[str, Any] = field(default_factory=dict))

One discoverable benchmark workload and its execution profiles.

Attributes

Name	Type	Default/value
name	str
title	str
description	str
profiles	tuple[BenchmarkProfile, ...]
runner	BenchmarkRunner
workload_id	str
category	str	'workload'
requirements	Mapping[str, Any]	field(default_factory=dict)

profile

method

def profile(name: str | None=None) -> BenchmarkProfile: ...

BenchmarkTimedBoundary

class View source

BenchmarkTimedBoundary(id: str, description: str, includes: tuple[str, ...], excludes: tuple[str, ...], synchronization: str)

Stable definition of what one benchmark measurement includes.

Attributes

Name	Type	Default/value
id	str
description	str
includes	tuple[str, ...]
excludes	tuple[str, ...]
synchronization	str

to_dict

method

def to_dict() -> dict[str, Any]: ...

from_dict

method

def from_dict(payload: Mapping[str, Any]) -> BenchmarkTimedBoundary: ...

BenchmarkMetric

class View source

BenchmarkMetric(name: str, value: int | float, unit: str, statistic: str, direction: MetricDirection, dimensions: Mapping[str, Any] = field(default_factory=dict), samples: tuple[int | float, ...] = ())

One normalized finite benchmark measurement.

Attributes

Name	Type	Default/value
name	str
value	int | float
unit	str
statistic	str
direction	MetricDirection
dimensions	Mapping[str, Any]	field(default_factory=dict)
samples	tuple[int | float, ...]	()

to_dict

method

def to_dict() -> dict[str, Any]: ...

from_dict

method

def from_dict(payload: Mapping[str, Any]) -> BenchmarkMetric: ...

BenchmarkCheck

class View source

BenchmarkCheck(name: str, passed: bool, oracle: str, metric: str, observed: Any, comparison: str, limit: Any | None, unit: str | None = None, details: Mapping[str, Any] = field(default_factory=dict))

One recorded correctness observation and acceptance rule.

Attributes

Name	Type	Default/value
name	str
passed	bool
oracle	str
metric	str
observed	Any
comparison	str
limit	Any | None
unit	str | None	None
details	Mapping[str, Any]	field(default_factory=dict)

to_dict

method

def to_dict() -> dict[str, Any]: ...

from_dict

method

def from_dict(payload: Mapping[str, Any]) -> BenchmarkCheck: ...

BenchmarkResult

class View source

BenchmarkResult(benchmark: str, profile: str, workload_id: str, effective_parameters: Mapping[str, Any], timed_boundary: BenchmarkTimedBoundary, metrics: Sequence[BenchmarkMetric], correctness: Sequence[BenchmarkCheck], rows: list[dict[str, Any]] = field(default_factory=list), scalars: dict[str, Any] = field(default_factory=dict), metadata: dict[str, Any] = field(default_factory=dict), notes: list[str] = field(default_factory=list), evidence: list[dict[str, Any]] = field(default_factory=list))

Common result header plus workload-specific benchmark evidence.

Normalized metrics and correctness checks are the stable comparison and publication surface. rows, scalars, metadata, and evidence remain extensible for workload-specific diagnostics.

Attributes

Name	Type	Default/value
benchmark	str
profile	str
workload_id	str
effective_parameters	Mapping[str, Any]
timed_boundary	BenchmarkTimedBoundary
metrics	Sequence[BenchmarkMetric]
correctness	Sequence[BenchmarkCheck]
rows	list[dict[str, Any]]	field(default_factory=list)
scalars	dict[str, Any]	field(default_factory=dict)
metadata	dict[str, Any]	field(default_factory=dict)
notes	list[str]	field(default_factory=list)
evidence	list[dict[str, Any]]	field(default_factory=list)

correctness_passed

property

correctness_passed: bool

Whether at least one check exists and all checks passed.

to_dict

method

def to_dict() -> dict[str, Any]: ...

from_dict

method

def from_dict(payload: Mapping[str, Any]) -> BenchmarkResult: ...