FHElium

Appearance

fhelium.execution.buffer ​

Reusable fixed-address buffers for exact execution values.

Buffers provide exact fixed-storage data movement. Applications choose payload identity, movement timing, caching, and prefetch policy.

CopyHandle ​

class View source

python
CopyHandle(*, event: torch.cuda.Event | None, device: torch.device, source_tensors: tuple[torch.Tensor, ...], bytes_copied: int, target_token: object)

Future-like handle for one copy enqueued into a reusable buffer.

A handle owns the exact submitted tensor leaves until the CUDA event reports completion. This prevents a mutable source tree from releasing or replacing pinned host or device storage while an asynchronous copy may still read it. wait_on inserts a stream dependency without blocking the CPU; synchronize blocks the caller.

CopyHandle is intentionally not an asyncio Future and is not awaitable. A future extension may bridge CUDA events to an async scheduler, but the core object currently exposes CUDA stream/event ordering only.

Instances are returned by ReusableValueBuffer.copy_from; direct construction is not part of the public API. Internally, event tracks completion, device identifies the target, source_tensors keep the submitted storage alive, bytes_copied records payload size, and target_token binds the handle to its originating buffer.

event ​

property

python
event: torch.cuda.Event | None

CUDA completion event, or None for a synchronous CPU copy.

device ​

property

python
device: torch.device

Target device of the copy.

bytes_copied ​

property

python
bytes_copied: int

Logical tensor payload bytes submitted by this copy.

done ​

method

python
def done() -> bool: ...

Return whether the copy has completed without blocking.

wait_on ​

method

python
def wait_on(stream: torch.cuda.Stream | None=None) -> CopyHandle: ...

Make stream wait for this copy and return self.

The default is the current stream on the copy target device. This method enqueues an event wait and does not synchronize the CPU.

Parameters

  • stream: Consumer CUDA stream. Defaults to the current stream on this handle's target device.

Returns

This handle, allowing fluent ordering before consumption.

Raises

  • ValueError: If the supplied stream belongs to another device.

synchronize ​

method

python
def synchronize() -> None: ...

Block the CPU until the copy completes, then release the source.

ReusableValueBuffer ​

class View source

python
ReusableValueBuffer(*, node: _BufferNode, signature: ValueTreeSignature, device: torch.device, pinned: bool)

Bases: Generic[_ValueT]

Fixed-structure storage whose exact-value payload may be replaced.

Construct a buffer from a representative tensor/exact-value tree with like. The representative defines container structure, cryptographic metadata, and tensor topology. The buffer separately owns tensors on one target device. copy_from accepts a matching tree on CPU or CUDA, validates the complete source before copying anything, and preserves every target tensor address.

The buffer identifies payloads solely by exact structure. Applications or extensions associate it with model weights, user keys, request ciphertexts, and cache/prefetch/eviction policy.

The value property reconstructs ordinary tensors and exact FHElium values around the fixed storage, so an eager callable can consume it without a buffer-specific evaluator API. Do not retain a reconstructed tree as an ownership or readiness signal; use the buffer and the CopyHandle returned by copy_from for lifetime and stream ordering.

Examples

Reuse one CUDA allocation for changing plaintext tiles::

    buffer = ReusableValueBuffer.like(
        prototype_tile,
        device="cuda:0",
    )
    copied = buffer.copy_from(
        pinned_cpu_tile,
        stream=transfer_stream,
        non_blocking=True,
    )

    with torch.cuda.stream(compute_stream):
        copied.wait_on(compute_stream)
        output = evaluate_tile(ciphertext, buffer.value)

Double buffering is built by creating two independent buffers and
copying into one while an eager program reads the other. Choosing the
next tile and recording the prior consumer event remain application
policy.

Construct buffers with like; the direct constructor accepts internal allocation-tree state and is not part of the public API.

like ​

method

python
def like(example: _ValueT, *, device: torch.device | str | None=None, pin_memory: bool=False) -> ReusableValueBuffer[_ValueT]: ...

Allocate independent storage with the structure of example.

Parameters

  • example: Tensor/exact-value tree defining structure, metadata, and initial payload. Supported containers are lists, tuples, and dictionaries.
  • device: Target residency. If omitted, all example tensors must already share one device, which becomes the target.
  • pin_memory: Allocate pinned host tensors. This requires a CPU target and is useful before non-blocking host-to-device copies.

Returns

A buffer initialized with a copy of example.

Raises

  • TypeError: If the tree contains an unsupported leaf.
  • ValueError: If there are no tensor leaves, the implicit target is ambiguous, pin_memory is requested for a non-CPU target, or representative leaves alias the same storage.

signature ​

property

python
signature: ValueTreeSignature

Device-independent structure and exact-value state of this buffer.

device ​

property

python
device: torch.device

Target residency of the owned storage.

is_pinned ​

property

python
is_pinned: bool

Whether all owned CPU tensors use pinned host memory.

nbytes ​

property

python
nbytes: int

Logical bytes in the fixed target tensors.

value ​

property

python
value: _ValueT

A fresh ordinary value tree around the fixed target tensors.

Container and exact-value wrapper objects may be newly reconstructed on each access; their tensor addresses remain fixed for the buffer lifetime. Payload readiness is governed by the relevant CopyHandle or caller stream ordering.

copy_from ​

method

python
def copy_from(source: _ValueT, *, stream: torch.cuda.Stream | None=None, non_blocking: bool=True, wait_for: torch.cuda.Event | Sequence[torch.cuda.Event] | None=None) -> CopyHandle: ...

Copy a matching source tree into the fixed target storage.

Full structural and exact-state validation happens before the first Tensor.copy_. Tensor source devices may differ from the target and from the representative used by like.

Parameters

  • source: Matching tensor/exact-value tree whose payload replaces the buffer contents.
  • stream: Caller-owned CUDA stream on which target writes are enqueued. Defaults to the current target-device stream. CPU buffers do not accept a stream.
  • non_blocking: Forwarded to Tensor.copy_. Actual asynchronous host-to-device overlap requires pinned CPU source tensors.
  • wait_for: One event or sequence of events that the copy stream must wait for before overwriting this buffer, for example an event recorded after the prior eager consumer finished reading it.

Returns

A future-like CopyHandle. Keep or wait on this handle before consuming the new payload. The buffer also retains in-flight handles so dropping the returned Python object cannot prematurely release source storage.

Raises

  • ExecutionInputError: If source structure, metadata, or tensor topology differs from the buffer.
  • ValueError: If stream/event arguments are incompatible with a CPU target or the stream belongs to another CUDA device.
  • ExecutionError: If the buffer is closed.

Note

This method orders writes but cannot infer when an arbitrary eager consumer has finished reading the previous payload. Record a CUDA event after that consumer and pass it as wait_for before reusing the buffer.

wait_for ​

method

python
def wait_for(handle: CopyHandle, stream: torch.cuda.Stream | None=None) -> None: ...

Wait on a copy produced by this buffer without blocking the CPU.

Parameters

  • handle: Copy handle previously returned by this buffer.
  • stream: Consumer CUDA stream, or the current target stream when omitted.

Raises

  • ExecutionInputError: If handle belongs to another buffer.

close ​

method

python
def close() -> None: ...

Wait for outstanding copies and release target-storage references.

Closing is idempotent. The caller remains responsible for ensuring that arbitrary eager or captured consumers have finished reading the target tensors before closing; the buffer can track its own writes but cannot infer external read completion.

pin_value_tree ​

function View source

python
def pin_value_tree(value: _ValueT) -> _ValueT: ...

Clone a supported value tree into pinned CPU storage.

The returned ordinary tree owns pinned tensors and has the same device-independent signature as value. This helper chooses no cache or persistence policy; callers retain and release the returned tree normally.

Parameters

  • value: Supported tensor/exact-value tree to clone. Source leaves may be on CPU or CUDA.

Returns

Structurally equivalent ordinary value tree backed by pinned CPU tensors.

value_tree_nbytes ​

function View source

python
def value_tree_nbytes(value: object) -> int: ...

Return logical tensor bytes in one supported execution value tree.

Parameters

  • value: Tensor/exact-value tree whose tensor payload is counted.

Returns

Sum of numel * element_size over every tensor leaf.

Towards full-stack encrypted execution infrastructures

Released under the MIT License. Copyright VisualDust and FHElium contributors