FHElium

Appearance

fhelium.benchmarks.v1.model ​

Strict data model for the immutable FHElium Benchmark v1 specification.

BENCHMARK_VERSION ​

constant View source

python
BENCHMARK_VERSION = 'v1'

CaseStatus ​

class View source

python
CaseStatus()

Bases: StrEnum

Lifecycle state of one Benchmark v1 case.

Attributes ​

NameTypeDefault/value
PENDING'pending'
RUNNING'running'
MEASURED'measured'
UNAVAILABLE'unavailable'
FAILED'failed'
INTERRUPTED'interrupted'

ReportStatus ​

class View source

python
ReportStatus()

Bases: StrEnum

Lifecycle state of one Benchmark v1 report.

Attributes ​

NameTypeDefault/value
RUNNING'running'
COMPLETED'completed'
FAILED'failed'
INTERRUPTED'interrupted'

ExecutionBackend ​

class View source

python
ExecutionBackend()

Bases: StrEnum

Native execution backend selected for one complete v1 run.

Attributes ​

NameTypeDefault/value
CPU'cpu'
CUDA'cuda'

BenchmarkExecution ​

class View source

python
BenchmarkExecution(backend: ExecutionBackend, device: str)

Report-level backend and indexed device shared by every v1 case.

Attributes ​

NameTypeDefault/value
backendExecutionBackend
devicestr

to_dict ​

method

python
def to_dict() -> dict[str, str]: ...

from_dict ​

method

python
def from_dict(payload: Mapping[str, Any]) -> BenchmarkExecution: ...

BenchmarkCase ​

class View source

python
BenchmarkCase(id: str, title: str, category: str, benchmark: str, workload_id: str, profile: str | None = None, parameters: Mapping[str, Any] = field(default_factory=dict), requirements: Mapping[str, Any] = field(default_factory=dict), comparison: Mapping[str, Any] = field(default_factory=dict), unavailable_reason: str | None = None)

One fixed leaf-benchmark invocation in Benchmark v1.

parameters overrides the selected leaf profile. The v1 resolver records the resulting effective parameter object in its canonical manifest and in every case record. unavailable_reason declares a deterministic manifest-level absence; runners can additionally raise fhelium.benchmarks.v1.BenchmarkCaseUnavailable after preflight.

Attributes ​

NameTypeDefault/value
idstr
titlestr
categorystr
benchmarkstr
workload_idstr
profilestr | NoneNone
parametersMapping[str, Any]field(default_factory=dict)
requirementsMapping[str, Any]field(default_factory=dict)
comparisonMapping[str, Any]field(default_factory=dict)
unavailable_reasonstr | NoneNone

to_dict ​

method

python
def to_dict() -> dict[str, Any]: ...

from_dict ​

method

python
def from_dict(payload: Mapping[str, Any]) -> BenchmarkCase: ...

BenchmarkSpecification ​

class View source

python
BenchmarkSpecification(benchmark_version: str, title: str, description: str, cases: tuple[BenchmarkCase, ...])

The ordered, immutable Benchmark v1 case specification.

Attributes ​

NameTypeDefault/value
benchmark_versionstr
titlestr
descriptionstr
casestuple[BenchmarkCase, ...]

to_dict ​

method

python
def to_dict() -> dict[str, Any]: ...

from_dict ​

method

python
def from_dict(payload: Mapping[str, Any]) -> BenchmarkSpecification: ...

ProbeError ​

class View source

python
ProbeError(probe: str, error_type: str, message: str)

Non-fatal error from an optional platform-information probe.

Attributes ​

NameTypeDefault/value
probestr
error_typestr
messagestr

to_dict ​

method

python
def to_dict() -> dict[str, str]: ...

from_dict ​

method

python
def from_dict(payload: Mapping[str, Any]) -> ProbeError: ...

FHEliumBuildIdentity ​

class View source

python
FHEliumBuildIdentity(version: str, distribution: Mapping[str, Any], source_git: Mapping[str, Any], native: Mapping[str, Any])

FHElium distribution, source, and native-build provenance.

Attributes ​

NameTypeDefault/value
versionstr
distributionMapping[str, Any]
source_gitMapping[str, Any]
nativeMapping[str, Any]

to_dict ​

method

python
def to_dict() -> dict[str, Any]: ...

from_dict ​

method

python
def from_dict(payload: Mapping[str, Any]) -> FHEliumBuildIdentity: ...

PlatformSnapshot ​

class View source

python
PlatformSnapshot(system: Mapping[str, Any], cpu: Mapping[str, Any], memory: Mapping[str, Any], python: Mapping[str, Any], fhelium_build: FHEliumBuildIdentity, torch: Mapping[str, Any], cuda: Mapping[str, Any], environment: Mapping[str, Any], invocation: tuple[str, ...], probe_errors: tuple[ProbeError, ...] = ())

Normalized hardware, runtime, build, and invocation provenance.

Attributes ​

NameTypeDefault/value
systemMapping[str, Any]
cpuMapping[str, Any]
memoryMapping[str, Any]
pythonMapping[str, Any]
fhelium_buildFHEliumBuildIdentity
torchMapping[str, Any]
cudaMapping[str, Any]
environmentMapping[str, Any]
invocationtuple[str, ...]
probe_errorstuple[ProbeError, ...]()

to_dict ​

method

python
def to_dict() -> dict[str, Any]: ...

from_dict ​

method

python
def from_dict(payload: Mapping[str, Any]) -> PlatformSnapshot: ...

CaseUnavailable ​

class View source

python
CaseUnavailable(reason: str, details: Mapping[str, Any] = field(default_factory=dict))

Structured explanation for a case that cannot be measured.

Attributes ​

NameTypeDefault/value
reasonstr
detailsMapping[str, Any]field(default_factory=dict)

to_dict ​

method

python
def to_dict() -> dict[str, Any]: ...

from_dict ​

method

python
def from_dict(payload: Mapping[str, Any]) -> CaseUnavailable: ...

CaseFailure ​

class View source

python
CaseFailure(stage: str, error_type: str, message: str, traceback: tuple[str, ...])

Structured exception information retained in a Benchmark v1 run.

Attributes ​

NameTypeDefault/value
stagestr
error_typestr
messagestr
tracebacktuple[str, ...]

to_dict ​

method

python
def to_dict() -> dict[str, Any]: ...

from_dict ​

method

python
def from_dict(payload: Mapping[str, Any]) -> CaseFailure: ...

CaseRecord ​

class View source

python
CaseRecord(id: str, title: str, category: str, benchmark: str, workload_id: str, profile: str, parameters: Mapping[str, Any], requirements: Mapping[str, Any], comparison: Mapping[str, Any], status: CaseStatus = CaseStatus.PENDING, started_at: str | None = None, finished_at: str | None = None, result: BenchmarkResult | None = None, unavailable: CaseUnavailable | None = None, failure: CaseFailure | None = None)

Resolved case identity, execution state, and optional leaf result.

Attributes ​

NameTypeDefault/value
idstr
titlestr
categorystr
benchmarkstr
workload_idstr
profilestr
parametersMapping[str, Any]
requirementsMapping[str, Any]
comparisonMapping[str, Any]
statusCaseStatusCaseStatus.PENDING
started_atstr | NoneNone
finished_atstr | NoneNone
resultBenchmarkResult | NoneNone
unavailableCaseUnavailable | NoneNone
failureCaseFailure | NoneNone

validate_state ​

method

python
def validate_state() -> None: ...

Reject contradictory terminal-state payloads.

to_dict ​

method

python
def to_dict() -> dict[str, Any]: ...

from_dict ​

method

python
def from_dict(payload: Mapping[str, Any]) -> CaseRecord: ...

BenchmarkReport ​

class View source

python
BenchmarkReport(benchmark_version: str, manifest_sha256: str, execution: BenchmarkExecution, platform: PlatformSnapshot, started_at: str, finished_at: str | None, status: ReportStatus, cases: list[CaseRecord])

Outer report for one complete or checkpointed Benchmark v1 run.

benchmark_version versions the entire Benchmark specification. FHElium, Python, Torch, CUDA, and native-build versions remain platform provenance. No total or composite score is defined.

Attributes ​

NameTypeDefault/value
benchmark_versionstr
manifest_sha256str
executionBenchmarkExecution
platformPlatformSnapshot
started_atstr
finished_atstr | None
statusReportStatus
caseslist[CaseRecord]

requires_nonzero_exit ​

property

python
requires_nonzero_exit: bool

Whether a CLI should fail due to failure or interruption.

suggested_exit_code ​

property

python
suggested_exit_code: int

Return the decision for a CLI without exiting the process.

to_dict ​

method

python
def to_dict() -> dict[str, Any]: ...

validate_state ​

method

python
def validate_state() -> None: ...

Reject contradictory report lifecycle and outcome fields.

from_dict ​

method

python
def from_dict(payload: Mapping[str, Any]) -> BenchmarkReport: ...

Towards full-stack encrypted execution infrastructures

Released under the MIT License. Copyright VisualDust and FHElium contributors