FHElium

Appearance

fhelium.residency.manager ​

Managed value identity, transitions, plans, and accounting.

ResidencyManager ​

class View source

python
ResidencyManager(budgets: Mapping[ResidencyLocation, int] | None=None, *, trace_capacity: int=4096)

Own managed values and their local memory materializations.

The manager provides ensure, move, drop, and discard transitions across pageable, pinned, or indexed CUDA locations. Locations are recorded when first budgeted or used. Values registered through adopt transfer logical alias ownership to this manager under caller-enforced rules; callers must not retain or mutate the concrete value outside a ResidencyLease. The runtime cannot enforce Python alias destruction, so accounting covers only manager-owned storage while those rules are followed.

Materialization is synchronous. CUDA read lifetimes are not: lease release records consumer-stream events and retains protection until they complete. Plans and scopes compose the same primitive transitions and reserve headroom for unmanaged outputs or native workspace.

Registered source callbacks execute under the manager's transition lock. While a callback is active, the callback access guard rejects reentrant and concurrent public state access to that manager from every process thread; a callback must not call the manager or wait for work that does so.

Parameters

  • budgets: Optional strict managed-byte limits by location. Omitted locations remain unbudgeted and are recorded lazily when used.
  • trace_capacity: Maximum completed transition reports retained in the in-memory trace. Zero disables retention.

manager_id ​

property

python
manager_id: str

Identity embedded in every handle issued by this manager.

state_version ​

property

python
state_version: int

Monotonic version of placement, ownership, and lifetime state.

locations ​

property

python
locations: tuple[ResidencyLocation, ...]

Budgeted or observed locations in registration order.

adopt ​

method

python
def adopt(value: ValueT, *, at: ResidencyLocation | None=None, replica_mode: ReplicaMode=ReplicaMode.EXCLUSIVE) -> ResidencyHandle[ValueT]: ...

Transfer logical ownership of one live value into managed state.

The caller must stop retaining and using value after this call. Python cannot revoke existing aliases; violating this rule weakens accounting and immutability guarantees.

register_source ​

method

python
def register_source(spec: ResidencyValueSpec[ValueT], source: ResidencySource[ValueT], *, source_location: ResidencyLocation=PAGEABLE_HOST) -> ResidencyHandle[ValueT]: ...

Register a trusted reconstruction source without loading a value.

The source must reproduce the exact contents and CKKS state identified by the new handle. The manager later validates runtime type, declared location, logical bytes, and storage ceiling; the source owns content-identity correctness.

Parameters

  • spec: Immutable reconstructible value specification and accounting limits.
  • source: Synchronous trusted loader returning independent storage.
  • source_location: Valid local location charged during reconstruction.

Raises

  • ValueError: If location or recoverability is invalid.
  • TypeError: If source does not provide load().

ensure ​

method

python
def ensure(handle: ResidencyHandle[ValueT], at: ResidencyLocation, *, stream: torch.cuda.Stream | None=None) -> ResidencyTransitionReport: ...

Ensure one replica at at while retaining existing replicas.

copy ​

method

python
def copy(handle: ResidencyHandle[ValueT], to: ResidencyLocation, *, stream: torch.cuda.Stream | None=None) -> ResidencyTransitionReport: ...

Create or retain a replica at to without removing its source.

move ​

method

python
def move(handle: ResidencyHandle[ValueT], to: ResidencyLocation, *, from_location: ResidencyLocation | None=None, stream: torch.cuda.Stream | None=None) -> ResidencyTransitionReport: ...

Materialize at to and remove one selected source replica.

drop ​

method

python
def drop(handle: ResidencyHandle[Any], at: ResidencyLocation) -> ResidencyTransitionReport: ...

Remove one unprotected replica without ending its logical value.

discard ​

method

python
def discard(handle: ResidencyHandle[Any]) -> ResidencyTransitionReport: ...

End one managed value and remove all unprotected replicas.

execute_actions ​

method

python
def execute_actions(actions: Iterable[ResidencyAction], *, name: str='residency-actions', transfer_streams: Mapping[ResidencyLocation, torch.cuda.Stream] | None=None, expected_state_version: int | None=None) -> ResidencyPlanReport: ...

Preflight and execute an ordered primitive transition sequence.

Preflight is atomic with respect to manager state, but execution is not transactional. If a runtime source, allocation, copy, or later action fails, completed actions remain valid. ResidencyPlanExecutionError identifies the failed phase and action and carries their structured partial ResidencyPlanReport. Full success returns the complete report.

Parameters

  • actions: Ordered low-level actions to simulate and execute.
  • name: Non-empty diagnostic plan name.
  • transfer_streams: Optional destination-location to CUDA copy-stream mapping. Actions without a configured destination stream use the current stream on that destination device.
  • expected_state_version: Optional state-version precondition checked atomically before preflight or mutation.

Raises

  • ResidencyPlanError: If current-state preflight is infeasible.
  • ResidencyPlanExecutionError: If execution begins and a runtime source, allocation, copy, or residency invariant fails.

explain ​

method

python
def explain(plan: ResidencyPlan, *, expected_state_version: int | None=None) -> ResidencyPlanExplanation: ...

Dry-run one complete scope plan without changing manager state.

Simulation covers reservations, ordered entry and exit actions, deterministic source resolution, protection, budgets, and predicted charged peaks. The result is point-in-time evidence, not an admission lock; scope entry repeats preflight.

scope ​

method

python
def scope(plan: ResidencyPlan, *, transfer_streams: Mapping[ResidencyLocation, torch.cuda.Stream] | None=None, expected_state_version: int | None=None) -> ResidencyScope: ...

Create a single-use scope for ordered plan entry, body, and exit.

Entering executes reclaim actions, admits reservations, then executes entry actions. Closing runs exit actions and releases reservations. Completed transitions are never rolled back after a later failure.

expected_state_version is an optional state-version precondition checked atomically at entry before any plan mutation. Destination copy streams are selected from transfer_streams independently for each indexed CUDA location.

acquire ​

method

python
def acquire(handles: Iterable[ResidencyHandle[Any]], *, at: ResidencyLocation, consumer_stream: torch.cuda.Stream | None=None) -> ResidencyLease: ...

Borrow already-ready immutable materializations for evaluator reads.

CUDA leases require an explicit consumer_stream so release and finalization remain correct across Python threads. Additional consumer streams must be registered on the returned lease before release. This method is atomic across all handles and never materializes a missing value implicitly.

Parameters

  • handles: Non-empty managed values to borrow; duplicates collapse.
  • at: One valid local location shared by all requested values.
  • consumer_stream: Required initial stream for CUDA; omitted for CPU.

Raises

  • ValueError: If no handles are supplied or the location or CUDA stream requirements are not satisfied.
  • ResidencyUnavailableError: If any value is not already resident.
  • ResidencyHandleError: If any handle is foreign, stale, or unknown.

hold ​

method

python
def hold(handles: Iterable[ResidencyHandle[Any]], *, at: ResidencyLocation) -> ResidencyHold: ...

Retain a non-empty set of ready materializations outside active use.

reserve ​

method

python
def reserve(location: ResidencyLocation, nbytes: int, *, label: str) -> ResidencyReservation: ...

Record headroom and enforce its location budget when present.

snapshot ​

method

python
def snapshot() -> ResidencySnapshot[TensorResident]: ...

Return one atomic tensor-free view of values and location budgets.

trace ​

method

python
def trace() -> tuple[ResidencyTransitionReport, ...]: ...

Return retained completed transitions in completion order.

clear_trace ​

method

python
def clear_trace() -> None: ...

Remove retained transition reports without changing residency.

close ​

method

python
def close(*, wait: bool=True, force: bool=False) -> None: ...

Release manager-owned state after checking active lifetimes.

Parameters

  • wait: Synchronize already pending CUDA lease events before checking whether lifetime state remains.
  • force: Synchronize configured CUDA devices and clear active protections. This can invalidate public lifetime objects. force=True synchronizes configured CUDA devices and clears even leaked active protections. It is an explicit unsafe escape hatch for tests and process shutdown, not normal request cleanup.

Raises

  • ResidencyInUseError: If active or pending lifetimes remain without force=True.
  • ResidencyReentrancyError: If called from a source callback.

ResidencyScope ​

class View source

python
ResidencyScope(manager: ResidencyManager, plan: ResidencyPlan, *, transfer_streams: Mapping[ResidencyLocation, torch.cuda.Stream] | None=None, expected_state_version: int | None=None)

Single-use execution scope for one immutable residency plan.

exit_error retains a structured exit failure when a body exception remains the primary propagated exception.

manager ​

property

python
manager: ResidencyManager

Manager authority captured when this scope was constructed.

plan ​

property

python
plan: ResidencyPlan

Immutable plan captured when this scope was constructed.

expected_state_version ​

property

python
expected_state_version: int | None

State-version precondition captured at construction.

close ​

method

python
def close() -> None: ...

Execute exit actions and release reservations idempotently.

Towards full-stack encrypted execution infrastructures

Released under the MIT License. Copyright VisualDust and FHElium contributors