IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /docs/manual/basics.md). For the complete Mojo documentation index, see llms.txt.
Skip to main content
Version: Nightly
For the complete Mojo documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /docs/manual/basics.md).

DevicePointerStorage

struct DevicePointerStorage[*, element_width: Int = Int(1)]

Implements TensorOps backed by a DevicePointer handle.

DevicePointerStorage is the device-pointer-backed analogue of PointerStorage, accepting the same element_width parameter. Its StorageType handle is a DevicePointer, which on the host carries the buffer's owning reference plus an element offset and size, and which substitutes to a bare device UnsafePointer at the kernel boundary (DevicePointer.device_type).

Because the handle conforms to DevicePassable, a host-side DevicePointer shrinks to a real device address when the enclosing TileTensor encodes its fields for a kernel launch. The address is written into the first bytes of the handle's slot, so the memory operations here reinterpret those bytes (_device_leaf_ptr) on device. They abort on host: device memory is not guaranteed to be host-dereferenceable. The operations that don't dereference storage — offset, distance, unsafe_cast — work on both host and device using DevicePointer arithmetic or pure reinterprets.

Parameters​

  • ​element_width (Int): Number of scalar elements per logical element. A value of 1 (the default) is a non-vectorized tensor; larger values describe a vectorized view whose logical elements are SIMD vectors. The DevicePointer handle is always scalar-typed and every operation works in scalar-element units, so element_width only sets element_size (and thus the tile's vectorized ElementType), exactly as for PointerStorage.

Implemented traits​

AnyType, ImplicitlyDeletable, TensorOps, TensorStorage

comptime members​

element_size​

comptime element_size = element_width

Number of scalar elements per logical element (alias of element_width).

OffsetResultType​

comptime OffsetResultType[offset_types: TypeList[values]] = DevicePointerStorage[element_width=element_width]

The storage type produced by offsetting with a given coordinate.

Offsetting never changes the storage policy, so this is Self.

Parameters​

  • ​offset_types (TypeList[values]): The coordinate element types of the applied offset.

StorageType​

comptime StorageType[mut: Bool, //, dtype: DType, origin: Origin[mut=mut], address_space: AddressSpace] = DevicePointer[dtype, origin]

A DevicePointer handle borrowing the storage.

The address_space is part of the TensorStorage interface but is unused: a DevicePointer always refers to GENERIC device memory.

Parameters​

  • ​mut (Bool): The mutability of the borrowed storage, inferred from origin.
  • ​dtype (DType): The element data type of the borrowed storage.
  • ​origin (Origin[mut=mut]): The origin tracking the lifetime of the borrowed storage.
  • ​address_space (AddressSpace): The address space the borrowed storage resides in.

Methods​

write_type_name_to​

static def write_type_name_to(mut writer: T)

Writes the storage type name representation to the writer.

Args:

  • ​writer (T): The Writer to output to.

unsafe_cast​

static def unsafe_cast[to_mut: Bool, //, to_dtype: DType, to_origin: Origin[mut=to_mut], to_address_space: AddressSpace](storage: DevicePointer) -> DevicePointer[to_dtype, to_origin]

Reinterprets a storage handle with new type parameters.

DevicePointer has an identical layout across dtype and origin, so this is a byte-for-byte reinterpret of the handle; no DeviceBuffer element conversion takes place. The caller is responsible for ensuring the new parameters are valid for the referenced storage.

Parameters:

  • ​to_mut (Bool): The mutability of the origin.
  • ​to_dtype (DType): The element data type to reinterpret the storage as.
  • ​to_origin (Origin[mut=to_mut]): The origin to reinterpret the storage as.
  • ​to_address_space (AddressSpace): The address space to reinterpret the storage as.

Args:

Returns:

DevicePointer[to_dtype, to_origin]: A handle referring to the same storage, viewed with the new type parameters.

load​

static def load[dtype: DType, //, width: SIMDSize, alignment: Int, invariant: Bool = False, non_temporal: Bool = False](storage: DevicePointer[dtype]) -> SIMD[dtype, width]

Loads a SIMD value from the storage.

Device-only: reinterprets the encoded device pointer, which aborts on host. Device memory is not guaranteed to be host-dereferenceable.

Parameters:

  • ​dtype (DType): The element data type of the storage.
  • ​width (SIMDSize): The number of elements to load.
  • ​alignment (Int): The alignment guarantee for the load.
  • ​invariant (Bool): If True, the compiler may assume the memory won't be modified during the kernel, enabling load hoisting and caching.
  • ​non_temporal (Bool): If True, indicates the data will not be reused soon, allowing the hardware to bypass caches (e.g., streaming loads).

Args:

Returns:

SIMD[dtype, width]: The loaded SIMD value.

static def load[dtype: DType, //, width: SIMDSize, alignment: Int, invariant: Bool = False, non_temporal: Bool = False](storage: DevicePointer[dtype], offset: T) -> SIMD[dtype, width]

Loads a SIMD value at a scalar-element offset from the storage.

Device-only: reinterprets the encoded device pointer, which aborts on host. Device memory is not guaranteed to be host-dereferenceable.

Parameters:

  • ​dtype (DType): The element data type of the storage.
  • ​width (SIMDSize): The number of elements to load.
  • ​alignment (Int): The alignment guarantee for the load.
  • ​invariant (Bool): If True, the compiler may assume the memory won't be modified during the kernel, enabling load hoisting and caching.
  • ​non_temporal (Bool): If True, indicates the data will not be reused soon, allowing the hardware to bypass caches (e.g., streaming loads).

Args:

  • ​storage (DevicePointer[dtype]): The storage to load from.
  • ​offset (T): The scalar-element offset to load at.

Returns:

SIMD[dtype, width]: The loaded SIMD value.

store​

static def store[dtype: DType, alignment: Int, *, non_temporal: Bool = False](storage: DevicePointer[dtype], value: SIMD[dtype])

Stores a SIMD value into the storage.

Device-only: reinterprets the encoded device pointer, which aborts on host. Device memory is not guaranteed to be host-dereferenceable.

Parameters:

  • ​dtype (DType): The element data type of the storage.
  • ​alignment (Int): The alignment guarantee for the store.
  • ​non_temporal (Bool): If True, indicates the data will not be reused soon, allowing the hardware to bypass caches (e.g., streaming stores).

Args:

static def store[dtype: DType, alignment: Int, *, non_temporal: Bool = False](storage: DevicePointer[dtype], offset: T, value: SIMD[dtype])

Stores a SIMD value at a scalar-element offset in the storage.

Device-only: reinterprets the encoded device pointer, which aborts on host. Device memory is not guaranteed to be host-dereferenceable.

Parameters:

  • ​dtype (DType): The element data type of the storage.
  • ​alignment (Int): The alignment guarantee for the store.
  • ​non_temporal (Bool): If True, indicates the data will not be reused soon, allowing the hardware to bypass caches (e.g., streaming stores).

Args:

  • ​storage (DevicePointer[dtype]): The storage to store into.
  • ​offset (T): The scalar-element offset to store at.
  • ​value (SIMD[dtype]): The SIMD value to store.

offset​

static def offset[offset_mut: Bool, offset_types: TypeList[offset_types.values], //, offset_dtype: DType, offset_origin: Origin[mut=offset_mut], offset_address_space: AddressSpace](var storage: DevicePointer[offset_dtype, offset_origin], var offset_coord: Coord[offset_types]) -> DevicePointer[offset_dtype, offset_origin]

Returns a storage handle offset by a number of scalar elements.

On host this advances the wrapped DevicePointer (bounds-checked against the owning DeviceBuffer). On device it advances the encoded device pointer held in the handle's first bytes, preserving the rest of the handle's (unused) bytes.

Parameters:

  • ​offset_mut (Bool): The mutability of the storage, inferred from offset_origin.
  • ​offset_types (TypeList[offset_types.values]): The coordinate element types of offset_coord.
  • ​offset_dtype (DType): The element data type of the storage.
  • ​offset_origin (Origin[mut=offset_mut]): The origin tracking the lifetime of the storage.
  • ​offset_address_space (AddressSpace): The address space the storage resides in.

Args:

Returns:

DevicePointer[offset_dtype, offset_origin]: A handle of the same type starting the given number of scalar elements into the referenced storage.

distance​

static def distance[dtype: DType, address_space: AddressSpace, //](storage: DevicePointer[dtype], other: DevicePointer[dtype]) -> Int

Returns the scalar-element distance from other to storage.

Parameters:

  • ​dtype (DType): The storages' DType.
  • ​address_space (AddressSpace): The storages' AddressSpace.

Args:

Returns:

Int: The number of scalar elements separating the two handles. The value is positive when storage is ahead of other and negative when it precedes other.

copy_from​

static def copy_from[SelfLayoutType: TensorLayout, self_origin: MutOrigin, self_address_space: AddressSpace, OtherLayoutType: TensorLayout, other_mut: Bool, other_origin: Origin[mut=other_mut], other_address_space: AddressSpace, //, dst_dtype: DType, src_dtype: DType, OtherStorage: TensorStorage](storage: Tuple[DevicePointer[dst_dtype, self_origin], SelfLayoutType], other: Tuple[OtherStorage.StorageType[other_mut, origin_of(other_origin), src_dtype, other_origin, other_address_space], OtherLayoutType])

Copies the elements of other into storage, in place.

Loads each logical element from other through its (possibly different) storage policy and stores it into storage through DevicePointerStorage, casting to the destination dtype. Delegates to the shared _copy_from loop. Device-only: the underlying loads and stores reinterpret the encoded device pointer and abort on host.

  • Both operands must have statically known shapes with matching total element count.
  • Both operands must have the same logical element size.
  • Source and destination dtypes may differ; each logical element is cast to the destination dtype.

Parameters:

  • ​SelfLayoutType (TensorLayout): The layout type of the destination storage.
  • ​self_origin (MutOrigin): The origin of the destination storage.
  • ​self_address_space (AddressSpace): The address space of the destination storage.
  • ​OtherLayoutType (TensorLayout): The layout type of the source storage.
  • ​other_mut (Bool): The mutability of the source storage.
  • ​other_origin (Origin[mut=other_mut]): The origin of the source storage.
  • ​other_address_space (AddressSpace): The address space of the source storage.
  • ​dst_dtype (DType): The element data type of the destination storage.
  • ​src_dtype (DType): The element data type of the source storage.
  • ​OtherStorage (TensorStorage): The storage policy of the source. May differ from Self as long as the two policies are copy-compatible (same logical element size).

Args:

add​

static def add[SelfLayoutType: TensorLayout, self_origin: MutOrigin, self_address_space: AddressSpace, OtherLayoutType: TensorLayout, other_mut: Bool, other_origin: Origin[mut=other_mut], other_address_space: AddressSpace, //, dtype: DType](storage: Tuple[DevicePointer[dtype, self_origin], SelfLayoutType], other: Tuple[DevicePointer[dtype, other_origin], OtherLayoutType])

Adds other into storage elementwise, in place.

Parameters:

  • ​SelfLayoutType (TensorLayout): The layout type of the destination storage.
  • ​self_origin (MutOrigin): The origin of the destination storage.
  • ​self_address_space (AddressSpace): The address space of the destination storage.
  • ​OtherLayoutType (TensorLayout): The layout type of the right-hand storage operand.
  • ​other_mut (Bool): The mutability of the right-hand storage operand.
  • ​other_origin (Origin[mut=other_mut]): The origin of the right-hand storage operand.
  • ​other_address_space (AddressSpace): The address space of the right-hand storage operand.
  • ​dtype (DType): The element data type of both storages.

Args:

mul​

static def mul[SelfLayoutType: TensorLayout, self_origin: MutOrigin, self_address_space: AddressSpace, OtherLayoutType: TensorLayout, other_mut: Bool, other_origin: Origin[mut=other_mut], other_address_space: AddressSpace, //, dtype: DType](storage: Tuple[DevicePointer[dtype, self_origin], SelfLayoutType], other: Tuple[DevicePointer[dtype, other_origin], OtherLayoutType])

Multiplies storage by other elementwise, in place.

Parameters:

  • ​SelfLayoutType (TensorLayout): The layout type of the destination storage.
  • ​self_origin (MutOrigin): The origin of the destination storage.
  • ​self_address_space (AddressSpace): The address space of the destination storage.
  • ​OtherLayoutType (TensorLayout): The layout type of the right-hand storage operand.
  • ​other_mut (Bool): The mutability of the right-hand storage operand.
  • ​other_origin (Origin[mut=other_mut]): The origin of the right-hand storage operand.
  • ​other_address_space (AddressSpace): The address space of the right-hand storage operand.
  • ​dtype (DType): The element data type of both storages.

Args:

sub​

static def sub[SelfLayoutType: TensorLayout, self_origin: MutOrigin, self_address_space: AddressSpace, OtherLayoutType: TensorLayout, other_mut: Bool, other_origin: Origin[mut=other_mut], other_address_space: AddressSpace, //, dtype: DType](storage: Tuple[DevicePointer[dtype, self_origin], SelfLayoutType], other: Tuple[DevicePointer[dtype, other_origin], OtherLayoutType])

Subtracts other from storage elementwise, in place.

Parameters:

  • ​SelfLayoutType (TensorLayout): The layout type of the destination storage.
  • ​self_origin (MutOrigin): The origin of the destination storage.
  • ​self_address_space (AddressSpace): The address space of the destination storage.
  • ​OtherLayoutType (TensorLayout): The layout type of the right-hand storage operand.
  • ​other_mut (Bool): The mutability of the right-hand storage operand.
  • ​other_origin (Origin[mut=other_mut]): The origin of the right-hand storage operand.
  • ​other_address_space (AddressSpace): The address space of the right-hand storage operand.
  • ​dtype (DType): The element data type of both storages.

Args:

floordiv​

static def floordiv[SelfLayoutType: TensorLayout, self_origin: MutOrigin, self_address_space: AddressSpace, OtherLayoutType: TensorLayout, other_mut: Bool, other_origin: Origin[mut=other_mut], other_address_space: AddressSpace, //, dtype: DType](storage: Tuple[DevicePointer[dtype, self_origin], SelfLayoutType], other: Tuple[DevicePointer[dtype, other_origin], OtherLayoutType])

Floor-divides storage by other elementwise, in place.

Parameters:

  • ​SelfLayoutType (TensorLayout): The layout type of the destination storage.
  • ​self_origin (MutOrigin): The origin of the destination storage.
  • ​self_address_space (AddressSpace): The address space of the destination storage.
  • ​OtherLayoutType (TensorLayout): The layout type of the right-hand storage operand.
  • ​other_mut (Bool): The mutability of the right-hand storage operand.
  • ​other_origin (Origin[mut=other_mut]): The origin of the right-hand storage operand.
  • ​other_address_space (AddressSpace): The address space of the right-hand storage operand.
  • ​dtype (DType): The element data type of both storages.

Args:

truediv​

static def truediv[SelfLayoutType: TensorLayout, self_origin: MutOrigin, self_address_space: AddressSpace, OtherLayoutType: TensorLayout, other_mut: Bool, other_origin: Origin[mut=other_mut], other_address_space: AddressSpace, //, dtype: DType](storage: Tuple[DevicePointer[dtype, self_origin], SelfLayoutType], other: Tuple[DevicePointer[dtype, other_origin], OtherLayoutType])

True-divides storage by other elementwise, in place.

Parameters:

  • ​SelfLayoutType (TensorLayout): The layout type of the destination storage.
  • ​self_origin (MutOrigin): The origin of the destination storage.
  • ​self_address_space (AddressSpace): The address space of the destination storage.
  • ​OtherLayoutType (TensorLayout): The layout type of the right-hand storage operand.
  • ​other_mut (Bool): The mutability of the right-hand storage operand.
  • ​other_origin (Origin[mut=other_mut]): The origin of the right-hand storage operand.
  • ​other_address_space (AddressSpace): The address space of the right-hand storage operand.
  • ​dtype (DType): The element data type of both storages.

Args:

min​

static def min[SelfLayoutType: TensorLayout, self_origin: MutOrigin, self_address_space: AddressSpace, OtherLayoutType: TensorLayout, other_mut: Bool, other_origin: Origin[mut=other_mut], other_address_space: AddressSpace, //, dtype: DType](storage: Tuple[DevicePointer[dtype, self_origin], SelfLayoutType], other: Tuple[DevicePointer[dtype, other_origin], OtherLayoutType])

Takes the elementwise minimum of storage and other, in place.

Parameters:

  • ​SelfLayoutType (TensorLayout): The layout type of the destination storage.
  • ​self_origin (MutOrigin): The origin of the destination storage.
  • ​self_address_space (AddressSpace): The address space of the destination storage.
  • ​OtherLayoutType (TensorLayout): The layout type of the right-hand storage operand.
  • ​other_mut (Bool): The mutability of the right-hand storage operand.
  • ​other_origin (Origin[mut=other_mut]): The origin of the right-hand storage operand.
  • ​other_address_space (AddressSpace): The address space of the right-hand storage operand.
  • ​dtype (DType): The element data type of both storages.

Args:

max​

static def max[SelfLayoutType: TensorLayout, self_origin: MutOrigin, self_address_space: AddressSpace, OtherLayoutType: TensorLayout, other_mut: Bool, other_origin: Origin[mut=other_mut], other_address_space: AddressSpace, //, dtype: DType](storage: Tuple[DevicePointer[dtype, self_origin], SelfLayoutType], other: Tuple[DevicePointer[dtype, other_origin], OtherLayoutType])

Takes the elementwise maximum of storage and other, in place.

Parameters:

  • ​SelfLayoutType (TensorLayout): The layout type of the destination storage.
  • ​self_origin (MutOrigin): The origin of the destination storage.
  • ​self_address_space (AddressSpace): The address space of the destination storage.
  • ​OtherLayoutType (TensorLayout): The layout type of the right-hand storage operand.
  • ​other_mut (Bool): The mutability of the right-hand storage operand.
  • ​other_origin (Origin[mut=other_mut]): The origin of the right-hand storage operand.
  • ​other_address_space (AddressSpace): The address space of the right-hand storage operand.
  • ​dtype (DType): The element data type of both storages.

Args:

abs​

static def abs[dtype: DType, //](storage: DevicePointer[dtype], layout: T)

Takes the elementwise absolute value of storage, in place.

For unsigned dtypes this is the identity. Device-only: the underlying loads and stores reinterpret the encoded device pointer and abort on host.

Parameters:

  • ​dtype (DType): The element data type of the storage.

Args:

  • ​storage (DevicePointer[dtype]): The storage to modify in place.
  • ​layout (T): The layout describing the storage's elements.

recip​

static def recip[dtype: DType, //](storage: DevicePointer[dtype], layout: T)

Replaces each element of storage with its reciprocal, in place.

Elements equal to zero produce infinity, following IEEE 754 division semantics. Device-only: the underlying loads and stores reinterpret the encoded device pointer and abort on host.

Parameters:

  • ​dtype (DType): The element data type of the storage. Must be a floating-point type.

Args:

  • ​storage (DevicePointer[dtype]): The storage to modify in place.
  • ​layout (T): The layout describing the storage's elements.

exp​

static def exp[dtype: DType, //, scale: Scalar[dtype]](storage: DevicePointer[dtype], layout: T)

Replaces each element x of storage with exp(scale * x), in place.

The scale factor is applied before exponentiation so that scaled exponentials (for example softmax logit scaling) fuse into a single pass over the elements. Pass a scale of 1 for a plain exponential. Device-only: the underlying loads and stores reinterpret the encoded device pointer and abort on host.

Parameters:

  • ​dtype (DType): The element data type of the storage. Must be a floating-point type.
  • ​scale (Scalar[dtype]): The compile-time factor each element is multiplied by before exponentiation.

Args:

  • ​storage (DevicePointer[dtype]): The storage to modify in place.
  • ​layout (T): The layout describing the storage's elements.
View source