Skip to main content
Version: Nightly

DeviceBuffer

struct DeviceBuffer[dtype: DType]

Represents a block of device-resident storage. For GPU devices, a device buffer is allocated in the device's global memory.

To allocate a DeviceBuffer, use one of the methods provided by DeviceContext, such as enqueue_create_buffer().

Parameters​

  • ​dtype (DType): Data dtype to be stored in the buffer.

Implemented traits​

AnyType, Copyable, DevicePassable, ImplicitlyCopyable, ImplicitlyDestructible, Movable, Sized, Writable

comptime members​

device_type​

comptime device_type = UnsafePointer[Scalar[dtype], MutAnyOrigin]

DeviceBuffer dtypes are remapped to UnsafePointer when passed to accelerator devices.

Methods​

__init__​

__init__(out self, *, copy: Self)

Creates a copy of an existing device buffer by incrementing its reference count.

This copy constructor creates a new reference to the same underlying device buffer by incrementing the reference count of the native buffer object. Both the original and the copy will refer to the same memory on the device.

Args:

  • ​copy (Self): The device buffer to copy.

__del__​

__del__(deinit self)

Releases resources associated with this device buffer.

This function schedules an owned buffer free using the stream in the device context. The actual deallocation may occur asynchronously after all operations using this buffer have completed.

get_type_name​

static get_type_name() -> String

Gets this dtype's name, for use in error messages when handing arguments to kernels. TODO: This will go away soon, when we get better error messages for kernel calls.

Returns:

String: This dtype's name.

__len__​

__len__(self) -> Int

Returns the number of elements in this buffer.

This method calculates the number of elements by dividing the total byte size of the buffer by the size of each element.

Returns:

Int: The number of elements in the buffer.

create_sub_buffer​

create_sub_buffer[view_type: DType](self, offset: Int, size: Int) -> DeviceBuffer[view_type]

Creates a sub-buffer view of this buffer with a different element dtype.

This method creates a new buffer that references a subset of the memory in this buffer, potentially with a different element dtype. The sub-buffer shares the underlying memory with the original buffer.

Parameters:

  • ​view_type (DType): The data type for elements in the new sub-buffer.

Args:

  • ​offset (Int): The starting offset in elements from the beginning of this buffer.
  • ​size (Int): The number of elements in the new sub-buffer.

Returns:

DeviceBuffer: A new DeviceBuffer referencing the specified region with the specified element dtype.

Raises:

If the operation fails.

enqueue_copy_to​

enqueue_copy_to(self, dst: Self)

Enqueues an asynchronous copy from this buffer to another device buffer.

This method schedules a memory copy operation from this buffer to the destination buffer. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

  • ​dst (Self): The destination device buffer to copy data to.

Raises:

If the operation fails.

enqueue_copy_to(self, dst: HostBuffer[dtype])

Enqueues an asynchronous copy from this buffer to a host buffer.

This method schedules a memory copy operation from this buffer to the destination buffer. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

  • ​dst (HostBuffer): The destination host buffer to copy data to.

Raises:

If the operation fails.

enqueue_copy_to(self, dst_ptr: Optional[UnsafePointer[Scalar[dtype], origin]])

Enqueues an asynchronous copy from this buffer to host memory.

This method schedules a memory copy operation from this device buffer to the specified host memory location. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

  • ​dst_ptr (Optional): Pointer to the destination host memory location.

Raises:

If the operation fails.

enqueue_copy_to(self, dst: Span[Scalar[dtype], dst.origin])

Enqueues an asynchronous copy from this buffer to a Span.

This method schedules a memory copy operation from this buffer to the destination span. The operation is asynchronous and will be executed in the stream associated with this buffer's context. The span must contain at least as many elements as this buffer; this invariant is checked via debug_assert.

Args:

  • ​dst (Span): The destination span to copy data to. Must have at least as many elements as this buffer.

Raises:

If the operation fails.

enqueue_copy_from​

enqueue_copy_from(self, src: Self)

Enqueues an asynchronous copy to this buffer from another device buffer.

This method schedules a memory copy operation to this buffer from the source buffer. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

  • ​src (Self): The source device buffer to copy data from.

Raises:

If the operation fails.

enqueue_copy_from(self, src: HostBuffer[dtype])

Enqueues an asynchronous copy to this buffer from a host buffer.

This method schedules a memory copy operation to this buffer from the source buffer. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

  • ​src (HostBuffer): The source host buffer to copy data from.

Raises:

If the operation fails.

enqueue_copy_from(self, src_ptr: Optional[UnsafePointer[Scalar[dtype], origin]])

Enqueues an asynchronous copy to this buffer from host memory.

This method schedules a memory copy operation to this device buffer from the specified host memory location. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

  • ​src_ptr (Optional): Pointer to the source host memory location.

Raises:

If the operation fails.

enqueue_copy_from(self, src: Span[Scalar[dtype], src.origin])

Enqueues an asynchronous copy to this buffer from a Span.

This method schedules a memory copy operation to this buffer from the source span. The operation is asynchronous and will be executed in the stream associated with this buffer's context. The span must contain at least as many elements as this buffer; this invariant is checked via debug_assert.

Args:

  • ​src (Span): The source span to copy data from. Must have at least as many elements as this buffer.

Raises:

If the operation fails.

enqueue_fill​

enqueue_fill(self, val: Scalar[dtype])

Enqueues an operation to fill this buffer with a specified value.

This method schedules a memory set operation that fills the entire buffer with the specified value. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

  • ​val (Scalar): The value to fill the buffer with.

Raises:

If the operation fails.

reassign_ownership_to​

reassign_ownership_to(self, ctx: DeviceContext)

Transfers ownership of this buffer to another device context.

This method changes the device context that owns this buffer. This can be useful when sharing buffers between different contexts or when migrating workloads between devices.

Args:

  • ​ctx (DeviceContext): The new device context to take ownership of this buffer.

Raises:

If the operation fails.

take_ptr​

take_ptr(var self) -> UnsafePointer[Scalar[dtype], MutAnyOrigin]

Takes ownership of the device pointer from this buffer.

This method releases the device pointer from the buffer's control and returns it to the caller. After this call, the buffer no longer owns the pointer, and the caller is responsible for managing its lifecycle.

Returns:

UnsafePointer: The raw device pointer that was owned by this buffer.

unsafe_ptr​

unsafe_ptr(self) -> UnsafePointer[Scalar[dtype], MutAnyOrigin]

Returns the raw device pointer without transferring ownership.

This method provides direct access to the underlying device pointer for advanced use cases. The buffer retains ownership of the pointer.

Returns:

UnsafePointer: The raw device pointer owned by this buffer.

context​

context(self) -> DeviceContext

Returns the device context associated with this buffer.

This method retrieves the device context that owns this buffer and is responsible for managing its lifecycle and operations.

Returns:

DeviceContext: The device context associated with this buffer.

Raises:

If the operation fails.

map_to_host​

map_to_host(self, out mapped_buffer: _HostMappedBuffer[dtype])

Maps this device buffer to host memory for CPU access.

This method creates a host-accessible view of the device buffer's contents. The mapping operation may involve copying data from device to host memory.

Notes:

Values modified inside the with statement are updated on the device when the with statement exits.

Example:

from std.gpu.host import DeviceContext

var ctx = DeviceContext()
var length = 1024
var in_dev = ctx.enqueue_create_buffer[DType.float32](length)
var out_dev = ctx.enqueue_create_buffer[DType.float32](length)

# Initialize the input and output with known values.
with in_dev.map_to_host() as in_host, out_dev.map_to_host() as out_host:
    for i in range(length):
        in_host[i] = i
        out_host[i] = 255

Returns:

_HostMappedBuffer: A host-mapped buffer that provides CPU access to the device buffer's contents inside a with-statement.

Raises:

If there's an error during buffer creation or data transfer.

write_to​

write_to(self, mut writer: T)

Writes a string representation of this buffer to the provided writer.

This method formats the buffer's contents as a string and writes it to the specified writer. For large buffers, a compact representation is used.

Args:

  • ​writer (T): The writer to output the formatted string to.
Edit this page