Data structures
The amaranth.lib.data
module provides a way to describe the bitwise layout of values and a proxy class for accessing fields of values using the attribute access and indexing syntax.
Introduction
Overview
This module provides four related facilities:
Low-level bitwise layout description via
Field
andLayout
. These classes are rarely used directly, but are the foundation on which all other functionality is built. They are also useful for introspection.High-level bitwise layout description via
StructLayout
,UnionLayout
,ArrayLayout
, andFlexibleLayout
. These classes are the ones most often used directly, in particularStructLayout
andArrayLayout
.Data views via
View
or its user-defined subclasses. This class is used to apply a layout description to a plainValue
, enabling structured access to its bits.Data classes
Struct
andUnion
. These classes are data views with a layout that is defined using Python variable annotations (also known as type annotations).
Motivation
The fundamental Amaranth type is a Value
: a sequence of bits that can also be used as a number. Manipulating values directly is sufficient for simple applications, but in more complex ones, values are often more than just a sequence of bits; they have well-defined internal structure.
For example, consider a module that processes pixels, converting them from RGB to grayscale. The color pixel format is RGB565:
This module could be implemented (using a fast but very approximate method) as follows:
i_color = Signal(16)
o_gray = Signal(8)
m.d.comb += o_gray.eq((i_color[0:5] + i_color[5:11] + i_color[11:16]) << 1)
While this implementation works, it is repetitive, error-prone, hard to read, and laborous to change; all because the color components are referenced using bit offsets. To improve it, the structure can be described with a Layout
so that the components can be referenced by name:
from amaranth.lib import data, enum
rgb565_layout = data.StructLayout({
"red": 5,
"green": 6,
"blue": 5
})
i_color = data.View(rgb565_layout)
o_gray = Signal(8)
m.d.comb += o_gray.eq((i_color.red + i_color.green + i_color.blue) << 1)
The View
is value-castable and can be used anywhere a plain value can be used. For example, it can be assigned to in the usual way:
m.d.comb += i_color.eq(0) # everything is black
Composing layouts
Layouts are composable: a Layout
is a shape and can be used as a part of another layout. In this case, an attribute access through a view returns a view as well.
For example, consider a module that processes RGB pixels in groups of up to four at a time, provided by another module, and accumulates their average intensity:
input_layout = data.StructLayout({
"pixels": data.ArrayLayout(rgb565_layout, 4),
"valid": 4
})
i_stream = data.View(input_layout)
r_accum = Signal(32)
m.d.sync += r_accum.eq(
r_accum + sum((i_stream.pixels[n].red +
i_stream.pixels[n].green +
i_stream.pixels[n].blue)
* i_stream.valid[n]
for n in range(len(i_stream.valid))))
Note how the width of i_stream
is never defined explicitly; it is instead inferred from the shapes of its fields.
In the previous section, the precise bitwise layout was important, since RGB565 is an interchange format. In this section however the exact bit positions do not matter, since the layout is only used internally to communicate between two modules in the same design. It is sufficient that both of them use the same layout.
Defining layouts
Data layouts can be defined in a few different ways depending on the use case.
In case the data format is defined using a family of layouts instead of a single specific one, a function can be used:
def rgb_layout(r_bits, g_bits, b_bits):
return data.StructLayout({
"red": unsigned(r_bits),
"green": unsigned(g_bits),
"blue": unsigned(b_bits)
})
rgb565_layout = rgb_layout(5, 6, 5)
rgb24_layout = rgb_layout(8, 8, 8)
In case the data has related operations or transformations, View
can be subclassed to define methods implementing them:
class RGBLayout(data.View):
def __init__(self, target=None, *, r_bits, g_bits, b_bits, **kwargs):
super().__init__(layout=data.StructLayout({
"red": unsigned(r_bits),
"green": unsigned(g_bits),
"blue": unsigned(b_bits)
}, target=target, **kwargs))
def brightness(self):
return (self.red + self.green + self.blue)[-8:]
Here, the RGBLayout
class itself is shape-castable and can be used anywhere a shape is accepted.
In case the data format is static, Struct
(or Union
) can be subclassed instead of View
, to reduce the amount of boilerplate needed:
class IEEE754Single(data.Struct):
fraction: 23
exponent: 8 = 0x7f
sign: 1
def is_subnormal(self):
return self.exponent == 0
Discriminated unions
This module provides a UnionLayout
, which is rarely needed by itself, but is very useful in combination with a discriminant: a enumeration indicating which field of the union contains valid data.
For example, consider a module that can direct another module to perform one of a few operations, each of which requires its own parameters. The two modules could communicate through a channel with a layout like this:
class Command(data.Struct):
class Kind(enum.Enum):
SET_ADDR = 0
SEND_DATA = 1
valid : 1
kind : Kind
params : data.UnionLayout({
"set_addr": data.StructLayout({
"addr": unsigned(32)
}),
"send_data": data.StructLayout({
"byte": unsigned(8)
})
})
Here, the shape of the Command
is inferred, being large enough to accommodate the biggest of all defined parameter structures, and it is not necessary to manage it manually.
One module could submit a command with:
cmd = Command()
m.d.comb += [
cmd.valid.eq(1),
cmd.kind.eq(Command.Kind.SET_ADDR),
cmd.params.set_addr.addr.eq(0x00001234)
]
The other would react to commands as follows:
addr = Signal(32)
with m.If(cmd.valid):
with m.Switch(cmd.kind):
with m.Case(Command.Kind.SET_ADDR):
m.d.sync += addr.eq(cmd.params.set_addr.addr)
with m.Case(Command.Kind.SEND_DATA):
...
Modeling structured data
- class amaranth.lib.data.Field(shape, offset)
Description of a data field.
The
Field
class specifies the signedness and bit positions of a field in an Amaranth value.Field
objects are immutable.- Variables:
shape (shape-castable) – Shape of the field. When initialized or assigned, the object is stored as-is.
offset (
int
, >=0) – Index of the least significant bit of the field.
- property width
Width of the field.
This property should be used over
self.shape.width
becauseself.shape
can be an arbitrary shape-castable object, which may not have awidth
property.- Returns:
Shape.cast(self.shape).width
- Return type:
- __eq__(other)
Compare fields.
Two fields are equal if they have the same shape and offset.
- class amaranth.lib.data.Layout
Description of a data layout.
The shape-castable
Layout
interface associates keys (string names or integer indexes) with fields, giving identifiers to spans of bits in an Amaranth value.It is an abstract base class;
StructLayout
,UnionLayout
,ArrayLayout
, andFlexibleLayout
implement concrete layout rules. New layout rules can be defined by inheriting from this class.- static cast(obj)
Cast a shape-castable object to a layout.
This method performs a subset of the operations done by
Shape.cast()
; it will recursively call.as_shape()
, but only until a layout is returned.- Raises:
TypeError – If
obj
cannot be converted to aLayout
instance.RecursionError – If
obj.as_shape()
returnsobj
.
- static of(obj)
Extract the layout that was used to create a view.
- abstract __iter__()
Iterate fields in the layout.
- abstract __getitem__(key)
Retrieve a field from the layout.
- abstract property size
Size of the layout.
- Returns:
The amount of bits required to store every field in the layout.
- Return type:
- as_shape()
Shape of the layout.
- Returns:
unsigned(self.size)
- Return type:
Shape
- __eq__(other)
Compare layouts.
Two layouts are equal if they have the same size and the same fields under the same names. The order of the fields is not considered.
Common data layouts
- class amaranth.lib.data.StructLayout(members)
Description of a structure layout.
The fields of a structure layout follow one another without any gaps, and the size of a structure layout is the sum of the sizes of its members.
For example, the following layout of a 16-bit value:
can be described with:
data.StructLayout({ "first": 3, "second": 7, "third": 6 })
Note
Structures that have padding can be described with a
FlexibleLayout
. Alternately, padding can be added to the layout as fields called_1
,_2
, and so on. These fields won’t be accessible as attributes or by using indexing.- Variables:
members (mapping of
str
to shape-castable) – Dictionary of structure members.
- class amaranth.lib.data.UnionLayout(members)
Description of a union layout.
The fields of a union layout all start from bit 0, and the size of a union layout is the size of the largest of its members.
For example, the following layout of a 7-bit value:
can be described with:
data.UnionLayout({ "first": 3, "second": 7, "third": 6 })
- Variables:
members (mapping of
str
to shape-castable) – Dictionary of union members.
- class amaranth.lib.data.ArrayLayout(elem_shape, length)
Description of an array layout.
The fields of an array layout follow one another without any gaps, and the size of an array layout is the size of its element multiplied by its length.
For example, the following layout of a 16-bit value:
can be described with:
.. testcode::
data.ArrayLayout(unsigned(4), 4)
Note
Arrays that have padding can be described with a
FlexibleLayout
.- Variables:
elem_shape (shape-castable) – Shape of an individual element.
length (int) – Amount of elements.
- class amaranth.lib.data.FlexibleLayout(size, fields)
Description of a flexible layout.
The fields of a flexible layout can be located arbitrarily, and its size is explicitly defined.
For example, the following layout of a 16-bit value:
can be described with:
data.FlexibleLayout(16, { "first": data.Field(unsigned(3), 1), "second": data.Field(unsigned(7), 0), "third": data.Field(unsigned(6), 10), 0: data.Field(unsigned(1), 14) })
Both strings and integers can be used as names of flexible layout fields, so flexible layouts can be used to describe structures and arrays with arbitrary padding.
Data views
- class amaranth.lib.data.View(layout, target=None, *, name=None, reset=None, reset_less=None, attrs=None, decoder=None, src_loc_at=0)
A value viewed through the lens of a layout.
The value-castable class
View
provides access to the fields of an underlying Amaranth value via the names or indexes defined in the provided layout.Creating a view
When creating a view, either only the
target
argument, or any of thename
,reset
,reset_less
,attrs
, ordecoder
arguments may be provided. If a target is provided, it is used as the underlying value. Otherwise, a newSignal
is created, and the rest of the arguments are passed to its constructor.Accessing a view
Slicing a view or accessing its attributes returns a part of the underlying value corresponding to the field with that index or name, which is always an Amaranth value, but it could also be a
View
if the shape of the field is aLayout
, or an instance of the data class if the shape of the field is a class deriving fromStruct
orUnion
.Slicing a view whose layout is an
ArrayLayout
can be done with an index that is an Amaranth value instead of a constant integer. The returned element is chosen dynamically in that case.Custom view classes
The
View
class can be inherited from to define additional properties or methods on a view. The only two names that are reserved on instances ofView
areas_value()
andeq()
, leaving the rest to the developer. TheStruct
andUnion
classes provided in this module are subclasses ofView
that also provide a concise way to define a layout.- as_value()
Get underlying value.
- Returns:
The
target
provided when constructing the view, or theSignal
that was created.- Return type:
Value
- eq(other)
Assign to the underlying value.
- Returns:
self.as_value().eq(other)
- Return type:
Assign
- __getitem__(key)
Slice the underlying value.
A field corresponding to
key
is looked up in the layout. If the field’s shape is aLayout
, returns aView
. If it is a subclass ofStruct
orUnion
, returns an instance of that class. Otherwise, returns an unspecified Amaranth expression with the right shape.- Parameters:
key (
str
orint
orValueCastable
) – Name or index of a field.- Returns:
A slice of the underlying value defined by the field.
- Return type:
Value
, inout- Raises:
KeyError – If the layout does not define a field corresponding to
key
.TypeError – If
key
is a value-castable object, but the layout of the view is not aArrayLayout
.
- __getattr__(name)
Access a field of the underlying value.
Returns
self[name]
.- Raises:
AttributeError – If the layout does not define a field called
name
, or ifname
starts with an underscore.
Data classes
- class amaranth.lib.data.Struct(target=None, *, name=None, reset=None, reset_less=None, attrs=None, decoder=None, src_loc_at=0)
Structures defined with annotations.
The
Struct
base class is a subclass ofView
that provides a concise way to describe the structure layout and reset values for the fields using Python variable annotations.Any annotations containing shape-castable objects are used, in the order in which they appear in the source code, to construct a
StructLayout
. The values assigned to such annotations are used to populate the reset value of the signal created by the view. Any other annotations are kept as-is.As an example, a structure for IEEE 754 single-precision floating-point format can be defined as:
class IEEE754Single(Struct): fraction: 23 exponent: 8 = 0x7f sign: 1 def is_subnormal(self): return self.exponent == 0
The
IEEE754Single
class itself can be used where a shape is expected:>>> IEEE754Single.as_shape() StructLayout({'fraction': 23, 'exponent': 8, 'sign': 1}) >>> Signal(IEEE754Single).width 32
Instances of this class can be used where values are expected:
>>> flt = IEEE754Single() >>> Signal(32).eq(flt) (eq (sig $signal) (sig flt))
Accessing shape-castable properties returns slices of the underlying value:
>>> flt.fraction (slice (sig flt) 0:23) >>> flt.is_subnormal() (== (slice (sig flt) 23:31) (const 1'd0))
The reset values for individual fields can be overridden during instantiation:
>>> hex(IEEE754Single().as_value().reset) '0x3f800000' >>> hex(IEEE754Single(reset={'sign': 1}).as_value().reset) '0xbf800000' >>> hex(IEEE754Single(reset={'exponent': 0}).as_value().reset) '0x0'
Classes inheriting from
Struct
can be used as base classes. The only restrictions are that:Classes that do not define a layout cannot be instantiated or converted to a shape;
A layout can be defined exactly once in the inheritance hierarchy.
Behavior can be shared through inheritance:
class HasChecksum(Struct): def checksum(self): bits = Value.cast(self) return sum(bits[n:n+8] for n in range(0, len(bits), 8)) class BareHeader(HasChecksum): address: 16 length: 8 class HeaderWithParam(HasChecksum): address: 16 length: 8 param: 8
>>> HasChecksum.as_shape() Traceback (most recent call last): ... TypeError: Aggregate class 'HasChecksum' does not have a defined shape >>> bare = BareHeader(); bare.checksum() (+ (+ (+ (const 1'd0) (slice (sig bare) 0:8)) (slice (sig bare) 8:16)) (slice (sig bare) 16:24)) >>> param = HeaderWithParam(); param.checksum() (+ (+ (+ (+ (const 1'd0) (slice (sig param) 0:8)) (slice (sig param) 8:16)) (slice (sig param) 16:24)) (slice (sig param) 24:32))
- class amaranth.lib.data.Union(target=None, *, name=None, reset=None, reset_less=None, attrs=None, decoder=None, src_loc_at=0)
Unions defined with annotations.
The
Union
base class is a subclass ofView
that provides a concise way to describe the union layout using Python variable annotations. It is very similar to theStruct
class, except that its layout is aUnionLayout
.A
Union
can have only one field with a specified reset value. If a reset value is explicitly provided during instantiation, it overrides the reset value specified with an annotation:class VarInt(Union): int8: 8 int16: 16 = 0x100
>>> VarInt().as_value().reset 256 >>> VarInt(reset={'int8': 10}).as_value().reset 10