v24.3 (2024-05-02)
✨ Highlights
-
AnyPointerwas renamed toUnsafePointerand is now Mojo's preferred unsafe pointer type. It has several enhancements, including:-
The element type can now be any type: it doesn't require
Movable. -
Because of this, the
take_value(),emplace_value(), andmove_into()methods have been changed to top-level functions and renamed. The new functions are: -
A new
destroy_pointee()function runs the destructor on the pointee. -
UnsafePointercan be initialized directly from aReferencewithUnsafePointer(someRef)and can convert to a reference withyourPointer[]. Both infer element type and address space. Note that when you convert a pointer to a reference, there's no way for Mojo to track the lifetime of the original value. So the resulting reference is no safer than the original pointer.
-
-
All of the pointer types received some cleanup to make them more consistent, for example the
unsafe.bitcast()global function is now a consistentbitcast()method on the pointers, which can convert element type and address space. -
Improvements to variadic arguments support.
-
Heterogeneous variadic pack arguments now work reliably even with memory types, and have a more convenient API to use, as defined by the
VariadicPacktype. For example, a simplified version ofprintcan be implemented like this:fn print[T: Stringable, *Ts: Stringable](first: T, *rest: *Ts):
print_string(str(first))
@parameter
fn print_elt[T: Stringable](a: T):
print_string(" ")
print_string(a)
rest.each[print_elt]() -
Mojo now supports declaring functions that have both optional and variadic arguments, both positional and keyword-only. For example, this now works:
fn variadic_arg_after_default(
a: Int, b: Int = 3, *args: Int, c: Int, d: Int = 1, **kwargs: Int
): ...Positional variadic parameters also work in the presence of optional parameters. That is:
fn variadic_param_after_default[e: Int, f: Int = 2, *params: Int]():
passNote that variadic keyword parameters are not supported yet.
For more information, see Variadic arguments in the Mojo Manual.
-
-
The
mojo buildandmojo runcommands now support a-goption. This shorter alias is equivalent to writing--debug-level full. This option is also available in themojo debugcommand, but is already the default. -
Many new standard library APIs have been filled in, including many community contributions. Changes are listed in the standard library section.
-
The Mojo Manual has a new page on Types.
Language changes
-
Certain dunder methods that take indices (
__getitem__(),__setitem__(), and__refitem__()) or names (__getattr__()and__setattr__()) can now take the index or name as a parameter value instead of an argument value. This is enabled when you define one of these methods with no argument other thanself(for a getter) orselfand the set value (for a setter).This enables types that can only be subscripted into with parameters, as well as things like the following example, which passes the attribute name as a parameter so that attribute names can be checked at compile time.
struct RGB:
fn __getattr__[name: StringLiteral](self) -> Int:
@parameter
if name == "r": return ...
elif name == "g": return ...
else:
constrained[name == "b", "can only access with r, g, or b members"]()
return ...
var rgb = RGB()
print(rgb.b) # Works
print(rgb.q) # Compile error -
Mojo now allows users to capture the source location of code and call location of functions dynamically using the
__source_location()and__call_location()functions. For example:from builtin._location import __call_location
@always_inline
fn my_assert(cond: Bool, msg: String):
if not cond:
var call_loc = __call_location()
print("In", call_loc.file_name, "on line", str(call_loc.line) + ":", msg)
fn main():
my_assert(False, "always fails") # some_file.mojo, line 193This prints "
In /path/to/some_file.mojo on line 193: always fails". Note that__call_location()only works in@always_inlineor@always_inline("nodebug")functions. It gives incorrect results if placed in an@always_inlinefunction that's called from an@always_inline("nodebug")function.This feature is still evolving and for the time being you need to explicitly import these APIs, as shown above. In the future, these will probably be built-in functions and not require an import statement.
Neither
__source_location()nor__call_location()work when called in a parameter context. For example:from builtin._location import __call_location
@always_inline
fn mystery_location() -> String:
var loc = __call_location()
return str(loc.file_name)
def main():
alias doesnt_work = mystery_location() # <unknown location in parameter context>
Standard library changes
⭐️ New
-
Listhas several new methods:-
pop(index)for removing an element at a particular index. By default,List.pop()removes the last element in the list. (@LJ-9801, fixes #2017) -
resize(new_size)for resizing the list without the need to specify an additional value. (@mikowals, fixes #2133) -
insert(index, value)for inserting a value at a specified index into theList. (@whym1here, fixes #2134) -
A new constructor
List(ptr, size, capacity)to avoid needing to do a deep copy of an existing contiguous memory allocation when constructing a newList. (@StandinKP, fixes #2170)
-
-
Dictnow has aupdate()method to update keys/values from anotherDict. (@gabrieldemarmiesse) -
Setnow has named methods for set operations:difference()mapping to-difference_update()mapping to-=intersection_update()mapping to&=update()mapping to|=
-
Dict,List, andSetall conform to theBoolabletrait. The collections evaluate toTrueif they contain any elements,Falseotherwise:def list_names(names: List[String]):
if names:
for name in names:
print(name[])
else:
print("No names to list.") -
Added
reversed()function for creating reversed iterators. Several range types,List, andDictnow support iterating in reverse.var numbers = List(1, 2, 3, 4, 5)
for number in reversed(numbers):
print(number)(@helehex and @jayzhan211, contributes towards #2325)
-
Optionalnow implements__is__and__isnot__methods so that you can compare anOptionalwithNone. For example:var opt = Optional(1)
if opt is not None:
print(opt.value()[]) -
Tuplenow works with memory-only element types likeStringand allows you to directly index into it with a parameter expression. This means you can now simply usex = tup[1]like Python instead ofx = tup.get[1, Int](). You can also assign into tuple elements now as well withtup[1] = x.var tuple = ("Green", 9.3)
var name = tuple[0]
var value = tuple[1]Note that because the subscript must be a parameter expression, you can't iterate through a
Tupleusing an ordinaryforloop. -
The
Referencetype has several changes, including:-
It has moved to the
memory.referencemodule instead ofmemory.unsafe. -
Referencenow has anunsafe_bitcast()method, similar to the pointer types. -
Several unsafe methods were removed, including
offset(),destroy_element_unsafe()andemplace_ref_unsafe(). This is becauseReferenceis a safe type—useUnsafePointerto do unsafe operations.
-
-
Boolcan now be implicitly converted from any type conforming to theBoolabletrait. This means that you no longer need to write code like this:@value
struct MyBoolable:
fn __bool__(self) -> Bool: ...
fn takes_boolable[T: Boolable](cond: T): ...
takes_boolable(MyBoolable())Instead, you can simply write:
fn takes_bool(cond: Bool): ...
takes_bool(MyBoolable())Note that calls to
takes_bool()will perform the implicit conversion, so in some cases is it still better to explicitly declare a type parameter, e.g.:fn takes_two_boolables[T: Boolable](a: T, b: T):
# Short circuit means `b.__bool__()` might not be evaluated.
if a.__bool__() and b.__bool__():
... -
PythonObjectnow conforms to theKeyElementtrait, meaning that it can be used as key type forDict. This allows you to easily build and interact with Python dictionaries in Mojo:def main():
d = PythonObject(Dict[PythonObject, PythonObject]())
d["foo"] = 12
d[7] = "bar"
d["foo"] = [1, 2, "something else"]
print(d) # prints `{'foo': [1, 2, 'something else'], 7: 'bar'}` -
FileHandle.seek()now has awhenceargument that defaults toos.SEEK_SETto seek from the beginning of the file. You can now set toos.SEEK_CURto offset by the currentFileHandleseek position:var f = open("/tmp/example.txt")
# Skip 32 bytes
f.seek(os.SEEK_CUR, 32)Or
os.SEEK_ENDto offset from the end of file:# Start from 32 bytes before the end of the file
f.seek(os.SEEK_END, -32) -
FileHandle.read()can now read straight into aDTypePointer:var file = open("/tmp/example.txt", "r")
# Allocate and load 8 elements
var ptr = DTypePointer[DType.float32].alloc(8)
var bytes = file.read(ptr, 8)
print("bytes read", bytes)
print(ptr.load[width=8]()) -
The
sysmodule now contains anexit()function that would exit a Mojo program with the specified error code.from sys import exit
exit(0) -
The constructors for
Tensorhave been changed to be more consistent. As a result, constructors take the shape as the first argument (instead of the second) when constructing a tensor with pointer data.If you pass a single scalar value to the
Tensorconstructor, it now broadcasts the value to all elements in the tensor. For example,Tensor[DType.float32](TensorShape(2,2), 0)constructs a2x2tensor initialized with all zeros. This provides an easy way to fill in the data of a tensor. -
Stringnow hasremoveprefix()andremovesuffix()methods. (@gabrieldemarmiesse) -
The
ordandchrfunctions have been improved to accept any Unicode character. (@mzaks, contributes towards #1616) -
atol()now handles whitespace. Theatol()function is used internally byString.__int__(), soint(String( " 10 "))now returns10instead of raising an error. (@artemiogr97) -
SIMDnow implements the__rmod__()method. (@bgreni, fixes #1482) -
bool(None)is now implemented. (@zhoujingya) -
The
DTypePointertype now implementsgather()for gathering aSIMDvector from offsets of a current pointer. Similarly, support forscatter()was added to scatter aSIMDvector into offsets of the current pointer. (@leandrolcampos) -
The
len()function now handles arange()specified with a negative end value, so that things likelen(range(-1))work correctly. (@soraros) -
debug_assert()now prints its location (filename, line, and column where it was called) in its error message. Similarly, theasserthelpers in thetestingmodule now include location information in their messages. -
The
testing.assert_equal[SIMD]()function now raises if any of the elements mismatch in the twoSIMDarguments being compared. (@gabrieldemarmiesse) -
The
testing.assert_almost_equal()andmath.isclose()functions now have anequal_nanflag. When set toTrue, then NaNs are considered equal. -
The
objecttype now supports the division, modulo, and left and right shift operators, including the in-place and reverse variants. (@LJ-9801, fixes #2224) -
Added checked arithmetic operations for
SIMDintegers.SIMDinteger types (including the sized integer scalars likeInt64) can now perform checked additions, subtractions, and multiplications using the following new methods:add_with_overflow()sub_with_overflow()mul_with_overflow()
Checked arithmetic allows the caller to determine if an operation exceeded the numeric limits of the type. For example:
var simd = SIMD[DType.int8, 4](7, 11, 13, 17)
var product: SIMD[DType.int8, 4]
var overflow: SIMD[DType.bool, 4]
(product, overflow) = simd.mul_with_overflow(simd)
for i in range(len(product)):
if overflow[i]:
print("<overflow>")
else:
print(product[i])(@lsh)
-
Added
os.remove()andos.unlink()for deleting files. (@artemiogr97, fixes #2306)
🦋 Changed
-
The
parallel_memcpy()function has moved from thebufferpackage to thealgorithmpackage. Please update your imports accordingly. -
Optional.value()now returns a reference instead of a copy of the contained value.To perform a copy manually, dereference the result:
var result = Optional(123)
var value = result.value()[] -
Per the accepted community proposal, Standardize the representation of byte sequence as a sequence of unsigned 8-bit integers, began transition to using
UInt8by changing the data pointer ofErrortoDTypePointer[DType.uint8]. (@gabrieldemarmiesse, contributes towards #2317) -
Continued transition to
UnsafePointerfrom the legacyPointertype in various standard library APIs and internals. (@gabrieldemarmiesse)
Tooling changes
-
The behavior of
mojo buildwhen invoked without an output-oargument has changed slightly:mojo build ./test-dir/program.mojonow outputs an executable to the path./program, whereas before it would output to the path./test-dir/program. -
The
mojo packagecommand no longer supports the-Dflag. All compilation environment flags should be provided at the point of package use (e.g.mojo runormojo build). -
The REPL no longer allows type level variable declarations to be uninitialized, e.g. it will reject
var s: String. This is because it does not do proper lifetime tracking (yet!) across cells, and so such code would lead to a crash. You can work around this by initializing to a dummy value and overwriting later. This limitation only applies to top level variables, variables in functions work as they always have.
Other changes
Low-level language changes
-
A low-level
__get_mvalue_as_litref(x)builtin was added to give access to the underlying memory representation as a!lit.refvalue without checking initialization status of the underlying value. This is useful in very low-level logic but isn't designed for general usability and will likely change in the future. -
Properties can now be specified on inline MLIR ops:
_ = __mlir_op.`kgen.source_loc`[
_type = (
__mlir_type.index, __mlir_type.index, __mlir_type.`!kgen.string`
),
_properties = __mlir_attr.`{inlineCount = 1 : i64}`,
]()As the example shows above, the protected
_propertiesattribute can be passed during op construction, with an MLIRDictionaryAttrvalue.
❌ Removed
-
Support for "register only" variadic packs has been removed. Instead of
AnyRegType, please upgrade your code toAnyTypein examples like this:fn your_function[*Types: AnyRegType](*args: *Ts): ...This move gives you access to a nicer API and has the benefit of being memory safe and correct for non-trivial types. If you need specific APIs on the types, please use the correct trait instead of
AnyType. -
List.pop_back()has been removed. UseList.pop()instead which defaults to popping the last element in the list. -
SIMD.to_int(value)has been removed. Useint(value)instead. -
The
__get_lvalue_as_address(x)magic function has been removed. To get a reference to a value useReference(x)and if you need an unsafe pointer, you can useUnsafePointer.address_of(x).
🛠️ Fixed
-
#516 and #1817 and many others, e.g. "Can't create a function that returns two strings."
-
#1178 (os/kern) failure (5).
-
#1609 alias with
DynamicVector[Tuple[Int]]fails. -
#1987 Defining
mainin a Mojo package is an error, for now. This is not intended to work yet, erroring for now will help to prevent accidental undefined behavior. -
#1215 and #1949 The Mojo LSP server no longer cuts off hover previews for functions with functional arguments, parameters, or results.
-
#1901 Fixed Mojo LSP and documentation generation handling of inout arguments.
-
#1913 -
0__no longer crashes the Mojo parser. -
#1924 JIT debugging on Mac has been fixed.
-
#1941 Mojo variadic arguments don't work with non-trivial register-only types.
-
#1963
a!=0is now parsed and formatted correctly bymojo format. -
#1676 Fix a crash related to
@valuedecorator and structs with empty body. -
#1917 Fix a crash after syntax error during tuple creation.
-
#2006 The Mojo LSP now properly supports signature types with named arguments and parameters.
-
#2007 and #1997 The Mojo LSP no longer crashes on certain types of closures.
-
#1675 Ensure
@valuedecorator fails gracefully after duplicate field error.