fhelium.residency

Managed local value placement, admission, lifetimes, and automation.

PAGEABLE_HOST

constant View source

PAGEABLE_HOST = ResidencyLocation('pageable-host', torch.device('cpu'))

PINNED_HOST

constant View source

PINNED_HOST = ResidencyLocation('pinned-host', torch.device('cpu'))

BorrowedValues

class View source

BorrowedValues(lease: ResidencyLease)

Bases: Mapping[ResidencyHandle[Any], TensorResident]

Concrete immutable materializations borrowed through one active lease.

The mapping validates its lease on every lookup and iteration. Returned exact values remain ordinary Python objects; callers must not mutate them or retain aliases beyond the lease. Managed memory accounting is strong only for manager-owned aliases that obey these borrowing rules.

DeterministicTieredLRU

class View source

DeterministicTieredLRU(fallback_tiers: Mapping[ResidencyLocation, Sequence[ResidencyLocation]] | None=None)

Deterministic priority-aware LRU ordering over configured tier edges.

Fallback locations are never inferred. An omitted source location has no spill path. Candidate order is lower priority, then older access epoch, then application stable key or manager registration order. Wall-clock time, UUID text, allocator free-memory readings, and random state are not inputs.

name

property

name: str

Stable diagnostic policy name.

config_identity

property

config_identity: tuple[tuple[str, object], ...]

Frozen fallback-tier configuration recorded in decisions.

fallback_locations

method

def fallback_locations(location: ResidencyLocation) -> tuple[ResidencyLocation, ...]: ...

Return configured destinations in caller-specified order.

order_candidates

method

def order_candidates(candidates: Sequence[ResidencyEvictionCandidate]) -> tuple[ResidencyEvictionCandidate, ...]: ...

Rank candidates without observing or mutating manager state.

DiscardValue

class View source

DiscardValue(handle: ResidencyHandle[ValueT_co])

Bases: Generic[ValueT_co]

End a managed value and remove all of its materializations.

Attributes

Name	Type	Default/value
handle	ResidencyHandle[ValueT_co]

DropResident

class View source

DropResident(handle: ResidencyHandle[ValueT_co], location: ResidencyLocation)

Bases: Generic[ValueT_co]

Remove one materialization without ending the managed value.

Attributes

Name	Type	Default/value
handle	ResidencyHandle[ValueT_co]
location	ResidencyLocation

EnsureResident

class View source

EnsureResident(handle: ResidencyHandle[ValueT_co], location: ResidencyLocation)

Bases: Generic[ValueT_co]

Require a managed value to be resident at location.

Attributes

Name	Type	Default/value
handle	ResidencyHandle[ValueT_co]
location	ResidencyLocation

MaterializationSnapshot

class View source

MaterializationSnapshot(location: ResidencyLocation, logical_nbytes: int, storage_nbytes: int, charged_nbytes: int, use_count: int, hold_count: int, pending_event_count: int)

Protection and byte state of one value at one residency location.

storage_nbytes is the current materialization's actual unique backing storage. charged_nbytes is the fixed conservative admission charge and can be larger after functional movement compacts a view-backed allocation. pending_event_count records completed lease closures whose CUDA work has not yet reached its recorded event. Such a materialization can have use_count == 0 while remaining protected from removal.

Attributes

Name	Type	Default/value
location	ResidencyLocation
logical_nbytes	int
storage_nbytes	int
charged_nbytes	int
use_count	int
hold_count	int
pending_event_count	int

MemoryReservation

class View source

MemoryReservation(location: ResidencyLocation, nbytes: int, label: str)

Named accounting headroom held for the lifetime of a plan scope.

Reservations account for future storage but do not allocate tensors or imply any value placement.

Attributes

Name	Type	Default/value
location	ResidencyLocation
nbytes	int
label	str

MoveResident

class View source

MoveResident(handle: ResidencyHandle[ValueT_co], to: ResidencyLocation, from_location: ResidencyLocation | None = None)

Bases: Generic[ValueT_co]

Move a managed value to to from an optional explicit source.

from_location=None delegates source resolution to the manager while preserving move semantics: successful completion does not retain the selected source materialization.

Attributes

Name	Type	Default/value
handle	ResidencyHandle[ValueT_co]
to	ResidencyLocation
from_location	ResidencyLocation | None	None

Recoverability

class View source

Recoverability()

Bases: StrEnum

Whether a managed value may lose its final materialization.

Attributes

Name	Type	Default/value
RECONSTRUCTIBLE		'reconstructible'
MUST_PRESERVE		'must-preserve'

ReplicaMode

class View source

ReplicaMode()

Bases: StrEnum

Constraint on simultaneous materializations of one managed value.

Attributes

Name	Type	Default/value
REPLICABLE		'replicable'
EXCLUSIVE		'exclusive'

ResidencyAction

data View source

ResidencyAction: TypeAlias = EnsureResident[TensorResident] | MoveResident[TensorResident] | DropResident[TensorResident] | DiscardValue[TensorResident]

ResidencyActionExplanation

class View source

ResidencyActionExplanation(action: ResidencyAction, executable: bool, no_op: bool, source: ResidencyLocation | None, destination: ResidencyLocation | None, logical_nbytes: int, storage_nbytes: int, reason: str | None = None)

Dry-run explanation of how one requested action would be resolved.

Attributes

Name	Type	Default/value
action	ResidencyAction
executable	bool
no_op	bool
source	ResidencyLocation | None
destination	ResidencyLocation | None
logical_nbytes	int
storage_nbytes	int
reason	str | None	None

ResidencyController

class View source

ResidencyController(manager: ResidencyManager, *, policy: ResidencyPolicy | None=None, search_state_limit: int=100000)

Deterministic optional automation bound to one residency manager.

The manager remains the only materialization owner and transition executor. This controller stores only policy metadata and logical access epochs. It decides only when called, emits an inspectable decision, and never runs background eviction, waits for protected victims, retries a failed source, rolls back completed actions, or silently replans a stale decision.

manager

property

manager: ResidencyManager

The sole state and materialization authority used by this controller.

policy

property

policy: ResidencyPolicy

Pure policy used to rank invariant-filtered candidates.

set_policy_metadata

method

def set_policy_metadata(handle: ResidencyHandle[Any], *, priority: int=0, stable_key: str | None=None) -> None: ...

Set controller-local eviction metadata for one known handle.

decide

method

def decide(request: ResidencyRequest) -> ResidencyDecision: ...

Derive an inspectable decision without executing residency actions.

scope

method

def scope(decision: ResidencyDecision, *, transfer_streams: TransferStreams | None=None) -> ResidencyScope: ...

Return a scope that version-checks and commits decision on entry.

use

method

def use(request: ResidencyRequest, *, consumer_streams: ConsumerStreams | None=None, transfer_streams: TransferStreams | None=None) -> ResidencyUse: ...

Return a context that derives, admits, and borrows one decision.

Values remain cached after the use scope. Later admission may reclaim them under policy. No exit eviction, hidden synchronization, or background work is performed.

ResidencyDecision

class View source

ResidencyDecision(manager_id: str, expected_state_version: int, explored_states: int, request: ResidencyRequest, plan: ResidencyPlan, policy_name: str, policy_config: tuple[tuple[str, object], ...], evictions: tuple[ResidencyEviction, ...], explanation: ResidencyPlanExplanation)

Immutable state-bound result of automatic residency decision-making.

A decision is valid only for the issuing manager at expected_state_version. It is process-local evidence, not a serialized or distributed execution artifact. Entering the associated manager scope atomically rechecks the version before any reclaim action or reservation.

Attributes

Name	Type	Default/value
manager_id	str
expected_state_version	int
explored_states	int
request	ResidencyRequest
plan	ResidencyPlan
policy_name	str
policy_config	tuple[tuple[str, object], ...]
evictions	tuple[ResidencyEviction, ...]
explanation	ResidencyPlanExplanation

ResidencyEviction

class View source

ResidencyEviction(action: DropResident[TensorResident] | MoveResident[TensorResident], rank: int, released_location: ResidencyLocation, released_nbytes: int, reason: str)

One controller-selected reclaim action and its policy evidence.

Attributes

Name	Type	Default/value
action	DropResident[TensorResident] | MoveResident[TensorResident]
rank	int
released_location	ResidencyLocation
released_nbytes	int
reason	str

ResidencyEvictionCandidate

class View source

ResidencyEvictionCandidate(handle: ResidencyHandle[Any], location: ResidencyLocation, charged_nbytes: int, registration_index: int, last_access_epoch: int | None, metadata: ResidencyPolicyMetadata)

One invariant-filtered materialization eligible for policy ranking.

Attributes

Name	Type	Default/value
handle	ResidencyHandle[Any]
location	ResidencyLocation
charged_nbytes	int
registration_index	int
last_access_epoch	int | None
metadata	ResidencyPolicyMetadata

ResidencyHandle

class View source

ResidencyHandle(manager_id: str, handle_id: str, value_type: type[ValueT_co])

Bases: Generic[ValueT_co]

Manager-issued tensor-free identity of one typed managed value.

Handles are immutable and hashable, and therefore remain valid dictionary keys independently of every materialization's device and lifetime. The value_type field preserves the expected runtime class without retaining a tensor-bearing value. manager_id prevents accidental cross-manager use, while handle_id is an opaque identity generated for each ResidencyManager.adopt or ResidencyManager.register_source call. Moving, copying, or dropping a materialization does not change the handle.

Parameters

• manager_id: Non-empty identity of the manager that issued the handle.
• handle_id: Non-empty opaque identity generated by the manager.
• value_type: Expected concrete or base TensorResident runtime class.

Attributes

Name	Type	Default/value
manager_id	str
handle_id	str
value_type	type[ValueT_co]

ResidencyHold

class View source

ResidencyHold(*, manager: ResidencyManager, token: object, handles: tuple[ResidencyHandle[Any], ...], location: ResidencyLocation)

Long-lived retention protection that exposes no concrete value.

Attributes

Name	Type	Default/value
close		release

active

property

active: bool

Whether this hold still protects its materializations.

handles

property

handles: tuple[ResidencyHandle[Any], ...]

Managed values retained by this hold.

location

property

location: ResidencyLocation

Location at which the materializations are retained.

release

method

def release() -> None: ...

Release exactly this retention protection idempotently.

ResidencyLease

class View source

ResidencyLease(*, manager: ResidencyManager, token: object, handles: tuple[ResidencyHandle[Any], ...], location: ResidencyLocation, consumer_streams: Sequence[torch.cuda.Stream])

Short read lifetime protected through CUDA consumer completion.

A lease is created only by ResidencyManager.acquire. CUDA consumer streams are registered rather than synchronized at Python scope exit. On release, the manager records one completion event per registered stream and retains every materialization until those events complete. CUDA acquisition requires one consumer stream; CPU leases release immediately.

Consumers using additional CUDA streams must call add_consumer_stream before release. Abandoning a lease synchronizes its registered streams before release; if that fails, a process-global strong root conservatively retains the manager and its storage.

Attributes

Name	Type	Default/value
close		release

active

property

active: bool

Whether the Python lease scope remains open.

handles

property

handles: tuple[ResidencyHandle[Any], ...]

Managed values protected by this lease.

location

property

location: ResidencyLocation

Location of every borrowed materialization.

values

property

values: BorrowedValues

Borrowed mapping valid until this lease is released.

add_consumer_stream

method

def add_consumer_stream(stream: torch.cuda.Stream) -> None: ...

Register another CUDA stream whose prior reads this lease protects.

release

method

def release(*, wait: bool=False) -> None: ...

Close the borrow and protect CUDA reads until completion.

Parameters

• wait: Synchronize recorded completion events before returning. With False, the manager retains a pending protection and reaps it after the events report completion.

ResidencyLocation

class View source

ResidencyLocation(kind: ResidencyLocationKind, device: torch.device)

Canonical identity of one managed memory tier.

Host locations always store the canonical unindexed cpu device. An indexed CPU spelling such as cpu:0 is accepted but normalized, which prevents two identities for the same host tier. CUDA locations require an device index because an ambient current device is not a stable placement identity.

Parameters

• kind: Storage class represented by the location.
• device: CPU for a host tier, or an indexed CUDA device.

Attributes

Name	Type	Default/value
kind	ResidencyLocationKind
device	torch.device

name

property

name: str

Return the stable diagnostic name of this location.

ResidencyLocationKind

type alias View source

ResidencyLocationKind = Literal['pageable-host', 'pinned-host', 'cuda']

ResidencyLocationSnapshot

class View source

ResidencyLocationSnapshot(location: ResidencyLocation, budget_bytes: int | None, used_bytes: int, reserved_bytes: int, remaining_budget_bytes: int | None, peak_used_bytes: int, peak_charged_bytes: int, value_count: int, reservation_count: int, use_count: int, hold_count: int, pending_event_count: int, allocator_allocated_bytes: int | None, allocator_reserved_bytes: int | None)

Budget, usage, and protection accounting for one location.

budget_bytes and remaining_budget_bytes are both None when the manager applies no admission limit at this location. With a budget, remaining_budget_bytes equals the budget minus current materialization and reservation charges. peak_used_bytes is the maximum materialization charge observed by the manager and excludes reservations; peak_charged_bytes includes MemoryReservation and temporary charges. CUDA locations additionally report process-wide PyTorch allocator allocated and reserved bytes at capture time; those metrics include storage outside this manager. Counts are aggregates over current materializations and named reservations at this location.

Attributes

Name	Type	Default/value
location	ResidencyLocation
budget_bytes	int | None
used_bytes	int
reserved_bytes	int
remaining_budget_bytes	int | None
peak_used_bytes	int
peak_charged_bytes	int
value_count	int
reservation_count	int
use_count	int
hold_count	int
pending_event_count	int
allocator_allocated_bytes	int | None
allocator_reserved_bytes	int | None

ResidencyManager

class View source

ResidencyManager(budgets: Mapping[ResidencyLocation, int] | None=None, *, trace_capacity: int=4096)

Own managed values and their local memory materializations.

The manager provides ensure, move, drop, and discard transitions across pageable, pinned, or indexed CUDA locations. Locations are recorded when first budgeted or used. Values registered through adopt transfer logical alias ownership to this manager under caller-enforced rules; callers must not retain or mutate the concrete value outside a ResidencyLease. The runtime cannot enforce Python alias destruction, so accounting covers only manager-owned storage while those rules are followed.

Materialization is synchronous. CUDA read lifetimes are not: lease release records consumer-stream events and retains protection until they complete. Plans and scopes compose the same primitive transitions and reserve headroom for unmanaged outputs or native workspace.

Registered source callbacks execute under the manager's transition lock. While a callback is active, the callback access guard rejects reentrant and concurrent public state access to that manager from every process thread; a callback must not call the manager or wait for work that does so.

Parameters

• budgets: Optional strict managed-byte limits by location. Omitted locations remain unbudgeted and are recorded lazily when used.
• trace_capacity: Maximum completed transition reports retained in the in-memory trace. Zero disables retention.

manager_id

property

manager_id: str

Identity embedded in every handle issued by this manager.

state_version

property

state_version: int

Monotonic version of placement, ownership, and lifetime state.

locations

property

locations: tuple[ResidencyLocation, ...]

Budgeted or observed locations in registration order.

adopt

method

def adopt(value: ValueT, *, at: ResidencyLocation | None=None, replica_mode: ReplicaMode=ReplicaMode.EXCLUSIVE) -> ResidencyHandle[ValueT]: ...

Transfer logical ownership of one live value into managed state.

The caller must stop retaining and using value after this call. Python cannot revoke existing aliases; violating this rule weakens accounting and immutability guarantees.

register_source

method

def register_source(spec: ResidencyValueSpec[ValueT], source: ResidencySource[ValueT], *, source_location: ResidencyLocation=PAGEABLE_HOST) -> ResidencyHandle[ValueT]: ...

Register a trusted reconstruction source without loading a value.

The source must reproduce the exact contents and CKKS state identified by the new handle. The manager later validates runtime type, declared location, logical bytes, and storage ceiling; the source owns content-identity correctness.

Parameters

• spec: Immutable reconstructible value specification and accounting limits.
• source: Synchronous trusted loader returning independent storage.
• source_location: Valid local location charged during reconstruction.

Raises

• ValueError: If location or recoverability is invalid.
• TypeError: If source does not provide load().

ensure

method

def ensure(handle: ResidencyHandle[ValueT], at: ResidencyLocation, *, stream: torch.cuda.Stream | None=None) -> ResidencyTransitionReport: ...

Ensure one replica at at while retaining existing replicas.

copy

method

def copy(handle: ResidencyHandle[ValueT], to: ResidencyLocation, *, stream: torch.cuda.Stream | None=None) -> ResidencyTransitionReport: ...

Create or retain a replica at to without removing its source.

move

method

def move(handle: ResidencyHandle[ValueT], to: ResidencyLocation, *, from_location: ResidencyLocation | None=None, stream: torch.cuda.Stream | None=None) -> ResidencyTransitionReport: ...

Materialize at to and remove one selected source replica.

drop

method

def drop(handle: ResidencyHandle[Any], at: ResidencyLocation) -> ResidencyTransitionReport: ...

Remove one unprotected replica without ending its logical value.

discard

method

def discard(handle: ResidencyHandle[Any]) -> ResidencyTransitionReport: ...

End one managed value and remove all unprotected replicas.

execute_actions

method

def execute_actions(actions: Iterable[ResidencyAction], *, name: str='residency-actions', transfer_streams: Mapping[ResidencyLocation, torch.cuda.Stream] | None=None, expected_state_version: int | None=None) -> ResidencyPlanReport: ...

Preflight and execute an ordered primitive transition sequence.

Preflight is atomic with respect to manager state, but execution is not transactional. If a runtime source, allocation, copy, or later action fails, completed actions remain valid. ResidencyPlanExecutionError identifies the failed phase and action and carries their structured partial ResidencyPlanReport. Full success returns the complete report.

Parameters

• actions: Ordered low-level actions to simulate and execute.
• name: Non-empty diagnostic plan name.
• transfer_streams: Optional destination-location to CUDA copy-stream mapping. Actions without a configured destination stream use the current stream on that destination device.
• expected_state_version: Optional state-version precondition checked atomically before preflight or mutation.

Raises

• ResidencyPlanError: If current-state preflight is infeasible.
• ResidencyPlanExecutionError: If execution begins and a runtime source, allocation, copy, or residency invariant fails.

explain

method

def explain(plan: ResidencyPlan, *, expected_state_version: int | None=None) -> ResidencyPlanExplanation: ...

Dry-run one complete scope plan without changing manager state.

Simulation covers reservations, ordered entry and exit actions, deterministic source resolution, protection, budgets, and predicted charged peaks. The result is point-in-time evidence, not an admission lock; scope entry repeats preflight.

scope

method

def scope(plan: ResidencyPlan, *, transfer_streams: Mapping[ResidencyLocation, torch.cuda.Stream] | None=None, expected_state_version: int | None=None) -> ResidencyScope: ...

Create a single-use scope for ordered plan entry, body, and exit.

Entering executes reclaim actions, admits reservations, then executes entry actions. Closing runs exit actions and releases reservations. Completed transitions are never rolled back after a later failure.

expected_state_version is an optional state-version precondition checked atomically at entry before any plan mutation. Destination copy streams are selected from transfer_streams independently for each indexed CUDA location.

acquire

method

def acquire(handles: Iterable[ResidencyHandle[Any]], *, at: ResidencyLocation, consumer_stream: torch.cuda.Stream | None=None) -> ResidencyLease: ...

Borrow already-ready immutable materializations for evaluator reads.

CUDA leases require an explicit consumer_stream so release and finalization remain correct across Python threads. Additional consumer streams must be registered on the returned lease before release. This method is atomic across all handles and never materializes a missing value implicitly.

Parameters

• handles: Non-empty managed values to borrow; duplicates collapse.
• at: One valid local location shared by all requested values.
• consumer_stream: Required initial stream for CUDA; omitted for CPU.

Raises

• ValueError: If no handles are supplied or the location or CUDA stream requirements are not satisfied.
• ResidencyUnavailableError: If any value is not already resident.
• ResidencyHandleError: If any handle is foreign, stale, or unknown.

hold

method

def hold(handles: Iterable[ResidencyHandle[Any]], *, at: ResidencyLocation) -> ResidencyHold: ...

Retain a non-empty set of ready materializations outside active use.

reserve

method

def reserve(location: ResidencyLocation, nbytes: int, *, label: str) -> ResidencyReservation: ...

Record headroom and enforce its location budget when present.

snapshot

method

def snapshot() -> ResidencySnapshot[TensorResident]: ...

Return one atomic tensor-free view of values and location budgets.

trace

method

def trace() -> tuple[ResidencyTransitionReport, ...]: ...

Return retained completed transitions in completion order.

clear_trace

method

def clear_trace() -> None: ...

Remove retained transition reports without changing residency.

close

method

def close(*, wait: bool=True, force: bool=False) -> None: ...

Release manager-owned state after checking active lifetimes.

Parameters

• wait: Synchronize already pending CUDA lease events before checking whether lifetime state remains.
• force: Synchronize configured CUDA devices and clear active protections. This can invalidate public lifetime objects. force=True synchronizes configured CUDA devices and clears even leaked active protections. It is an explicit unsafe escape hatch for tests and process shutdown, not normal request cleanup.

Raises

• ResidencyInUseError: If active or pending lifetimes remain without force=True.
• ResidencyReentrancyError: If called from a source callback.

ResidencyPlan

class View source

ResidencyPlan(name: str, enter: tuple[ResidencyAction, ...] = (), exit: tuple[ResidencyAction, ...] = (), reservations: tuple[MemoryReservation, ...] = (), reclaim: tuple[ResidencyAction, ...] = ())

Immutable ordered residency plan executed at scope entry and exit.

reclaim actions execute first and may free capacity needed by the scope. Reservations are admitted only after reclaim completes, then enter actions execute in tuple order. exit actions execute in tuple order when the plan scope closes. The tuples are normalized at construction so callers cannot mutate a plan by retaining an input list. Each action remains a manager request.

Parameters

• name: Non-empty diagnostic plan identity.
• reclaim: Ordered actions that establish capacity before reservations.
• enter: Ordered actions for plan-scope entry.
• exit: Ordered actions for plan-scope exit.
• reservations: Capacity reservations held across the plan scope.

Attributes

Name	Type	Default/value
name	str
enter	tuple[ResidencyAction, ...]	()
exit	tuple[ResidencyAction, ...]	()
reservations	tuple[MemoryReservation, ...]	()
reclaim	tuple[ResidencyAction, ...]	()

ResidencyPlanExplanation

class View source

ResidencyPlanExplanation(plan_name: str, actions: tuple[ResidencyActionExplanation, ...], reservations: tuple[MemoryReservation, ...], predicted_peak_bytes: tuple[tuple[ResidencyLocation, int], ...], feasible: bool, reason: str | None = None)

Immutable dry-run feasibility and predicted peak for one plan.

predicted_peak_bytes starts with the manager's tracked locations, then appends locations first referenced by the plan in reservation/action order. It represents the predicted peak of managed backing storage, MemoryReservation plan charges, and temporary source loading without mutating manager state. It is not CUDA caching-allocator reservation.

Attributes

Name	Type	Default/value
plan_name	str
actions	tuple[ResidencyActionExplanation, ...]
reservations	tuple[MemoryReservation, ...]
predicted_peak_bytes	tuple[tuple[ResidencyLocation, int], ...]
feasible	bool
reason	str | None	None

ResidencyPlanReport

class View source

ResidencyPlanReport(plan_name: str, transitions: tuple[ResidencyTransitionReport, ...], started_at_ns: int, completed_at_ns: int)

Completed transition sequence for one plan operation.

A successful execute_actions call or closed scope returns its complete report. A ResidencyPlanExecutionError carries a partial report containing only transitions committed before the runtime failure.

Attributes

Name	Type	Default/value
plan_name	str
transitions	tuple[ResidencyTransitionReport, ...]
started_at_ns	int
completed_at_ns	int

ResidencyPolicy

class View source

ResidencyPolicy()

Bases: Protocol

Pure deterministic policy interface used by a residency controller.

name

property

name: str

Stable diagnostic policy name.

config_identity

property

config_identity: tuple[tuple[str, object], ...]

Frozen tensor-free configuration evidence for decisions.

fallback_locations

method

def fallback_locations(location: ResidencyLocation) -> tuple[ResidencyLocation, ...]: ...

Return configured spill tiers for location.

order_candidates

method

def order_candidates(candidates: Sequence[ResidencyEvictionCandidate]) -> tuple[ResidencyEvictionCandidate, ...]: ...

Return every supplied candidate in deterministic eviction order.

ResidencyPolicyMetadata

class View source

ResidencyPolicyMetadata(priority: int = 0, stable_key: str | None = None)

Controller-owned eviction hints for one manager handle.

Higher priority values retain a value before lower-priority values. stable_key is an optional application-stable tie breaker and diagnostic identity. Neither field changes logical value identity or manager ownership.

Attributes

Name	Type	Default/value
priority	int	0
stable_key	str | None	None

ResidencyRequest

class View source

ResidencyRequest(name: str, requirements: tuple[ResidencyRequirement, ...], reservations: tuple[MemoryReservation, ...] = ())

Immutable declarative working-set and headroom request.

Each (handle, location) pair appears exactly once. A replicable value may therefore be required at several locations in one request. Whether a handle's replica mode permits those simultaneous postconditions is validated against manager state during controller decision-making.

Reservations are accounting headroom held while the admitted request is active. They neither allocate tensors nor name eviction victims.

Attributes

Name	Type	Default/value
name	str
requirements	tuple[ResidencyRequirement, ...]
reservations	tuple[MemoryReservation, ...]	()

ResidencyRequirement

class View source

ResidencyRequirement(handle: ResidencyHandle[Any], location: ResidencyLocation)

Require one managed logical value at one local location.

The requirement states a post-admission condition. It deliberately does not select a source materialization, transfer path, eviction victim, or execution stream; those choices belong to a controller decision.

Attributes

Name	Type	Default/value
handle	ResidencyHandle[Any]
location	ResidencyLocation

ResidencyReservation

class View source

ResidencyReservation(*, manager: ResidencyManager, token: object, location: ResidencyLocation, nbytes: int, label: str)

Idempotently releasable accounting reservation at one location.

Attributes

Name	Type	Default/value
close		release

active

property

active: bool

Whether this reservation still charges its location budget.

release

method

def release() -> None: ...

Return the reserved headroom to its location budget.

ResidencyReservationSnapshot

class View source

ResidencyReservationSnapshot(location: ResidencyLocation, nbytes: int, label: str)

One active named accounting reservation without an allocated tensor.

Attributes

Name	Type	Default/value
location	ResidencyLocation
nbytes	int
label	str

ResidencyScope

class View source

ResidencyScope(manager: ResidencyManager, plan: ResidencyPlan, *, transfer_streams: Mapping[ResidencyLocation, torch.cuda.Stream] | None=None, expected_state_version: int | None=None)

Single-use execution scope for one immutable residency plan.

exit_error retains a structured exit failure when a body exception remains the primary propagated exception.

manager

property

manager: ResidencyManager

Manager authority captured when this scope was constructed.

plan

property

plan: ResidencyPlan

Immutable plan captured when this scope was constructed.

expected_state_version

property

expected_state_version: int | None

State-version precondition captured at construction.

close

method

def close() -> None: ...

Execute exit actions and release reservations idempotently.

ResidencySnapshot

class View source

ResidencySnapshot(manager_id: str, state_version: int, values: tuple[ResidencyValueSnapshot[ValueT_co], ...], locations: tuple[ResidencyLocationSnapshot, ...], reservations: tuple[ResidencyReservationSnapshot, ...], captured_at_ns: int)

Bases: Generic[ValueT_co]

Hierarchical tensor-free observation of one manager at one instant.

state_version is the manager's monotonic mutation version at capture. captured_at_ns is a Unix timestamp in nanoseconds recorded while the manager holds the state lock used to construct both value and location observations.

Attributes

Name	Type	Default/value
manager_id	str
state_version	int
values	tuple[ResidencyValueSnapshot[ValueT_co], ...]
locations	tuple[ResidencyLocationSnapshot, ...]
reservations	tuple[ResidencyReservationSnapshot, ...]
captured_at_ns	int

ResidencySource

class View source

ResidencySource()

Bases: Protocol[ValueT_co]

Synchronous reconstruction source for one exact managed value.

Sources are registered directly with residency state and return a ready TensorResident value when invoked. The manager validates its runtime type, byte accounting, and initial location. The source is trusted to return the exact contents and CKKS state registered for that handle; the source owns content-identity correctness. Each call must return independent tensor storage whose logical alias ownership transfers to the manager; a source must not retain or later mutate the returned materialization.

load

method

def load() -> ValueT_co: ...

Synchronously reconstruct and return the exact registered value.

ResidencyTransitionReport

class View source

ResidencyTransitionReport(action: ResidencyAction, no_op: bool, source: ResidencyLocation | None, destination: ResidencyLocation | None, logical_nbytes: int, storage_nbytes: int, started_at_ns: int, completed_at_ns: int, allocator_device: torch.device | None = None, allocator_allocated_bytes_before: int | None = None, allocator_reserved_bytes_before: int | None = None, allocator_allocated_bytes_after: int | None = None, allocator_reserved_bytes_after: int | None = None, reason: str | None = None)

Measured result of one completed residency action.

action is the exact requested action. source records the resolved source when an action left it implicit, and destination records the target when applicable. CUDA allocator metrics are optional because host transitions and managers without allocator instrumentation cannot provide them. allocator_device identifies the optional allocator sample; a cross-device move samples its destination. Timestamps are Unix nanoseconds.

Attributes

Name	Type	Default/value
action	ResidencyAction
no_op	bool
source	ResidencyLocation | None
destination	ResidencyLocation | None
logical_nbytes	int
storage_nbytes	int
started_at_ns	int
completed_at_ns	int
allocator_device	torch.device | None	None
allocator_allocated_bytes_before	int | None	None
allocator_reserved_bytes_before	int | None	None
allocator_allocated_bytes_after	int | None	None
allocator_reserved_bytes_after	int | None	None
reason	str | None	None

ResidencyUse

class View source

ResidencyUse(*, controller: ResidencyController, request: ResidencyRequest, consumer_streams: ConsumerStreams | None, transfer_streams: TransferStreams | None)

Single-use automatic admission, borrow, and reservation lifetime.

decision

property

decision: ResidencyDecision

State-bound decision after successful context entry.

values

property

values: Mapping[ResidencyRequirement, TensorResident]

Borrowed values keyed by exact requirement during this context.

value

method

def value(handle: ResidencyHandle[Any], *, at: ResidencyLocation) -> TensorResident: ...

Return the borrow for one exact (handle, location) endpoint.

report

property

report: ResidencyPlanReport | None

Completed plan report after successful scope close.

exit_error

property

exit_error: BaseException | None

Structured plan cleanup failure retained by the underlying scope.

ResidencyValueSnapshot

class View source

ResidencyValueSnapshot(handle: ResidencyHandle[ValueT_co], spec: ResidencyValueSpec[ValueT_co], materializations: tuple[MaterializationSnapshot, ...], has_source: bool, source_location: ResidencyLocation | None, discarded: bool)

Bases: Generic[ValueT_co]

Public state of one registered or discarded managed value.

source_location identifies reconstruction placement when has_source is true. Discarded handles remain visible so diagnostics can explain why they are rejected; they have no source or live materializations.

Attributes

Name	Type	Default/value
handle	ResidencyHandle[ValueT_co]
spec	ResidencyValueSpec[ValueT_co]
materializations	tuple[MaterializationSnapshot, ...]
has_source	bool
source_location	ResidencyLocation | None
discarded	bool

ResidencyValueSpec

class View source

ResidencyValueSpec(value_type: type[ValueT_co], logical_nbytes: int, storage_nbytes: int, replica_mode: ReplicaMode, recoverability: Recoverability)

Bases: Generic[ValueT_co]

Immutable storage specification for one managed value.

logical_nbytes is the value's tensor payload size, while storage_nbytes is the fixed conservative charge used for each physical residency admission. A moved view may use less actual backing storage but may not exceed this registered ceiling; accounting still debits the full charge. This model requires the storage charge to be at least the logical payload size. Implementations with overlapping tensor views must therefore charge the full logical extent rather than deduct shared backing storage; this conservative rule makes admission accounting independent of alias analysis.

Parameters

• value_type: Expected TensorResident runtime class.
• logical_nbytes: Non-negative logical tensor payload bytes.
• storage_nbytes: Non-negative physical residency charge, greater than or equal to logical_nbytes.
• replica_mode: Whether simultaneous replicas are permitted.
• recoverability: Whether the final materialization may be discarded.

Attributes

Name	Type	Default/value
value_type	type[ValueT_co]
logical_nbytes	int
storage_nbytes	int
replica_mode	ReplicaMode
recoverability	Recoverability

cuda_location

function View source

def cuda_location(device: torch.device | str) -> ResidencyLocation: ...

Return the canonical location for one indexed CUDA device.

Parameters

• device: CUDA device such as "cuda:0". Unindexed "cuda" is rejected rather than resolved through process-global device state.

Returns

Immutable CUDA residency location.