Data Layouts#
Introduction#
The Wormhole SDK uses the layout package to define, serialize, and deserialize data structures efficiently. This modular system ensures consistent data formatting and cross-environment compatibility, benefiting projects that require robust handling of structured data.
By understanding the layout mechanism, you’ll be able to:
- Define data structures (numbers, arrays, and custom types)
- Efficiently serialize and deserialize data using the SDK’s utilities
- Handle protocol-specific layouts with ease
This guide is beneficial for developers looking to integrate Wormhole into their applications or protocols, especially those dealing with complex payloads or cross-chain communication.
Key Concepts#
Layout Items#
A layout defines how data structures should be serialized (converted into binary format) and deserialized (converted back into their original structure). This ensures consistent data formatting when transmitting information across different blockchain environments.
Layouts are composed of layout items, which describe individual fields or sets of fields in your data. Each layout item specifies:
name
- name of the fieldbinary
- type of data (e.g.,uint
,bytes
)size
- byte length for fixed-size fields within uint and bytes items only
Layout items can represent:
- Primitive types - basic data types like unsigned integers (
uint
) or byte arrays (bytes
) - Composite types - more complex structures, such as arrays or nested objects
Below is an example of a layout that might be used to serialize a message across the Wormhole protocol:
const exampleLayout = [
{ name: 'sourceChain', binary: 'uint', size: 2 },
{ name: 'orderSender', binary: 'bytes', size: 32 },
{ name: 'redeemer', binary: 'bytes', size: 32 },
{ name: 'redeemerMessage', binary: 'bytes', lengthSize: 4 },
] as const;
In this example:
sourceChain
is a 2-byte unsigned integer (uint
) identifying the source blockchainorderSender
is a fixed-length 32-byte array representing the sender's addressredeemer
is another 32-byte array used for the redeemer’s addressredeemerMessage
is a variable-length byte sequence, with its length specified by a 4-byte integer
This layout definition ensures that all necessary data fields are consistently encoded and can be correctly interpreted when they are deserialized.
Serialization and Deserialization#
Serialization converts structured data into binary format; deserialization reverses this, reconstructing the original objects.
You can serialize data using the serializeLayout
function:
To deserialize the binary data back into a structured object, use the deserializeLayout
function:
Custom Conversions#
Layouts also allow for custom conversions, which help map complex or custom types (like chain IDs or universal addresses) into a more usable format. This is useful when serializing or deserializing data that doesn’t fit neatly into simple types like integers or byte arrays.
For example, consider a custom conversion for a chain ID:
const chainCustomConversion = {
to: (chainId: number) => toChain(chainId),
from: (chain: Chain) => chainToChainId(chain),
} satisfies CustomConversion<number, Chain>;
This setup allows Wormhole to convert between human-readable formats and binary-encoded data used in payloads.
Error Handling#
The layout system performs error checks during serialization and deserialization. An error is thrown if data is incorrectly sized or in the wrong format. Refer to the below example:
try {
deserializeLayout(fillLayout, corruptedData);
} catch (error) {
console.error('Error during deserialization:', error.message);
}
Application of Layouts#
This section will focus on applying the concepts explained earlier through examples. These will help developers better understand how to define layouts, serialize and deserialize data, and use custom conversions where needed.
Defining Layouts#
To get started with layouts in Wormhole, you need to define your structure. A layout is simply a list of fields (layout items) describing how each data piece will be serialized.
Consider the following layout for a payload:
const exampleLayout = [
{ name: 'sourceChain', binary: 'uint', size: 2 },
{ name: 'orderSender', binary: 'bytes', size: 32 },
{ name: 'redeemer', binary: 'bytes', size: 32 },
{ name: 'redeemerMessage', binary: 'bytes', lengthSize: 4 },
] as const;
In this example:
sourceChain
is an unsigned integer (uint) of 2 bytesorderSender
is a 32-byte fixed-length byte arrayredeemer
is another 32-byte byte arrayredeemerMessage
is a length-prefixed byte array, with the length specified by a 4-byte integer
Serialize Data#
Once a layout is defined, the next step is to serialize data according to that structure. You can accomplish this using the serializeLayout
function from the Wormhole SDK.
const examplePayload = {
sourceChain: 6,
orderSender: new Uint8Array(32),
redeemer: new Uint8Array(32),
redeemerMessage: new Uint8Array([0x01, 0x02, 0x03]),
};
const serializedData = serializeLayout(exampleLayout, examplePayload);
This takes the data structure (examplePayload
) and serializes it according to the rules defined in the layout (exampleLayout
). The result is a Uint8Array
representing the serialized binary data.
Deserialize Data#
Deserialization is the reverse of serialization. Given a serialized Uint8Array
, we can convert it back into its original structure using the deserializeLayout
function.
This will output the structured object, making it easy to work with data transmitted or received from another chain.
Handling Variable-Length Fields#
One relevant aspect of Wormhole SDK's layout system is the ability to handle variable-length fields, such as arrays and length-prefixed byte sequences.
For instance, if you want to serialize or deserialize a message where the length of the content isn't known beforehand, you can define a layout item with a lengthSize
field.
This tells the SDK to read or write the message's length (in 4 bytes) and then handle the content.
Nested Layouts and Strong Typing#
The Wormhole SDK simplifies handling complex structures with nested layouts and strong typing. Nested layouts clearly represent hierarchical data, while strong typing ensures data consistency and catches errors during development.
Nested Layout#
In complex protocols, layouts can contain nested structures. Nested layouts become relevant here, allowing you to represent hierarchical data (such as transactions or multi-part messages) in a structured format.
Refer to the following nested layout where a message contains nested fields:
const nestedLayout = [
{
name: 'source',
binary: 'bytes',
layout: [
{ name: 'chainId', binary: 'uint', size: 2 },
{ name: 'sender', binary: 'bytes', size: 32 },
],
},
{
name: 'redeemer',
binary: 'bytes',
layout: [
{ name: 'address', binary: 'bytes', size: 32 },
{ name: 'message', binary: 'bytes', lengthSize: 4 },
],
},
] as const satisfies Layout;
In this layout:
source
is an object with two fields:chainId
andsender
redeemer
is another object with two fields:address
and a length-prefixedmessage
Strong Typing#
One of the benefits of using the Wormhole SDK in TypeScript is its support for strong typing. This ensures that serialized and deserialized data conform to expected structures, reducing errors during development by enforcing type checks at compile time.
Using TypeScript, the LayoutToType
utility provided by the SDK automatically generates a strongly typed structure based on the layout:
This ensures that when you serialize or deserialize data, it matches the expected structure.
const message: NestedMessage = {
source: {
chainId: 6,
sender: new Uint8Array(32),
},
redeemer: {
address: new Uint8Array(32),
message: new Uint8Array([0x01, 0x02, 0x03]),
},
};
Attempting to assign data of incorrect types will result in a compile-time error. The Wormhole SDK's layout system enforces strong types, reducing runtime errors and improving code reliability.
Serialization and Deserialization with Nested Layouts#
You can serialize and deserialize nested structures in the same way as simpler layouts:
const serializedNested = serializeLayout(nestedLayout, message);
const deserializedNested = deserializeLayout(nestedLayout, serializedNested);
Strong typing in TypeScript ensures that the message object conforms to the nested layout structure. This reduces the risk of data inconsistency during cross-chain communication.
Commonly Used Layouts#
The Wormhole SDK includes predefined layouts frequently used in cross-chain messaging. These layouts are optimized for standard fields such as chain IDs, addresses, and signatures. You can explore the complete set of predefined layouts in the layout-items
directory of the Wormhole SDK.
Chain ID Layouts#
Chain ID layouts in the Wormhole SDK derive from a common foundation: chainItemBase
. This structure defines the binary representation of a chain ID as a 2-byte unsigned integer, ensuring consistency across serialization and deserialization processes.
Base Structure#
This simple structure is the blueprint for more specific layouts by standardizing the binary format and size.
Dynamic Chain ID Layout#
The dynamic chain ID layout, chainItem
, extends chainItemBase
by adding flexible custom conversion logic. It enables runtime validation of chain IDs, supports optional null values, and restricts chain IDs to a predefined set when needed.
export const chainItem = <
const C extends readonly Chain[] = typeof chains,
const N extends boolean = false,
>(opts?: {
allowedChains?: C;
allowNull?: N;
}) =>
({
...chainItemBase, // Builds on the base structure
custom: {
to: (val: number): AllowNull<C[number], N> => { ... },
from: (val: AllowNull<C[number], N>): number => { ... },
},
});
This layout is versatile. It allows the serialization of human-readable chain names (e.g., Ethereum
) to numeric IDs (e.g., 1
) and vice versa. This is particularly useful when working with dynamic configurations or protocols supporting multiple chains.
Fixed Chain ID Layout#
The fixed chain ID layout, fixedChainItem
, is more rigid. It also extends chainItemBase
, but the custom field is hardcoded for a single chain. This eliminates runtime validation and enforces strict adherence to a specific chain.
export const fixedChainItem = <const C extends Chain>(chain: C) => ({
...chainItemBase, // Builds on the base structure
custom: {
to: chain,
from: chainToChainId(chain),
},
});
This layout allows developers to efficiently serialize and deserialize messages involving a single, fixed chain ID.
Address Layout#
The Wormhole SDK uses a Universal Address Layout to serialize and deserialize blockchain addresses into a standardized format. This layout ensures that addresses are always represented as fixed 32-byte binary values, enabling seamless cross-chain communication.
Base Structure#
The universalAddressItem
defines the layout for addresses. It uses the binary type bytes and enforces a fixed size of 32 bytes for consistency.
export const universalAddressItem = {
binary: 'bytes',
size: 32,
custom: {
to: (val: Uint8Array): UniversalAddress => new UniversalAddress(val),
from: (val: UniversalAddress): Uint8Array => val.toUint8Array(),
} satisfies CustomConversion<Uint8Array, UniversalAddress>,
} as const satisfies LayoutItem;
This layout ensures consistent address handling by defining the following:
- Serialization - converts a high-level
UniversalAddress
object into raw binary (32 bytes) for efficient storage or transmission - Deserialization - converts raw binary back into a
UniversalAddress
object, enabling further interaction in a human-readable or programmatic format
Signature Layout#
In the Wormhole SDK, the Signature Layout defines how to serialize and deserialize cryptographic signatures. These signatures verify message authenticity and ensure data integrity, particularly in Guardian-signed VAAs.
Base Structure#
The signatureLayout
specifies the binary structure of a secp256k1 signature. It divides the signature into three components:
const signatureLayout = [
{ name: 'r', binary: 'uint', size: 32 },
{ name: 's', binary: 'uint', size: 32 },
{ name: 'v', binary: 'uint', size: 1 },
] as const satisfies Layout;
This layout provides a clear binary format for the secp256k1 signature, making it efficient to process within the Wormhole protocol.
Layout with Custom Conversion#
The signatureItem
builds upon the signatureLayout
by adding custom conversion logic. This conversion transforms raw binary data into a high-level Signature
object and vice versa.
export const signatureItem = {
binary: 'bytes',
layout: signatureLayout,
custom: {
to: (val: LayoutToType<typeof signatureLayout>) =>
new Signature(val.r, val.s, val.v),
from: (val: Signature) => ({ r: val.r, s: val.s, v: val.v }),
} satisfies CustomConversion<LayoutToType<typeof signatureLayout>, Signature>,
} as const satisfies BytesLayoutItem;
The custom
field ensures seamless integration of raw binary data with the Signature
class, encapsulating signature-specific logic.
Advanced Use Cases#
The Wormhole SDK’s layout system is designed to handle various data structures and serialization needs. This section will explore more advanced use cases, such as handling conditional data structures, fixed conversions, and optimizing serialization performance.
Switch Statements for Conditional Layouts
In some cases, the structure of serialized data might change based on a specific field, such as a payload ID. The switch layout type conditionally defines layouts based on a value.
For example, different message types can be identified using a payload ID, and the layout for each message can be determined at runtime:
const switchLayout = {
binary: 'switch',
idSize: 1, // size of the payload ID
idTag: 'messageType', // tag to identify the type of message
layouts: [
[[1, 'messageType1'], fillLayout], // layout for type 1
[[2, 'messageType2'], fastFillLayout], // layout for type 2
],
} as const satisfies Layout;
The switch statement helps developers parse multiple payload types using the same structure, depending on a control field like an ID.
Fixed Conversions and Omitted Fields
Fixed conversions and omitted fields allow developers to handle known, static data without including it in every serialization or deserialization operation. For instance, when specific fields in a layout always hold a constant value, they can be omitted from the deserialized object.
Example: Fixed Conversion
In some cases, a field may always contain a predefined value. The layout system supports fixed conversions, allowing developers to “hard-code” these values:
const fixedConversionLayout = {
binary: 'uint',
size: 2,
custom: {
to: 'Ethereum',
from: chainToChainId('Ethereum'),
},
} as const satisfies Layout;
Example: Omitted Fields
Omitted fields are useful for handling padding or reserved fields that do not carry meaningful information and can safely be excluded from the deserialized output:
const omittedFieldLayout = [
{ name: 'reserved', binary: 'uint', size: 2, omit: true },
] as const satisfies Layout;
In this example, reserved
is a padding field with a fixed, non-dynamic value that serves no functional purpose. It is omitted from the deserialized result but still considered during serialization to maintain the correct binary format.
Only fields with a fixed, known value, such as padding or reserved fields, should be marked as omit: true
. Fields with meaningful or dynamic information, such as sourceChain
or version
, must remain in the deserialized structure to ensure data integrity and allow seamless round-trip conversions between serialized and deserialized representations.
Integration with Wormhole Protocol#
The layout system facilitates seamless interaction with the Wormhole protocol, mainly when dealing with VAAs. These cross-chain messages must be serialized and deserialized to ensure they can be transmitted and processed accurately across different chains.
VAAs and Layouts#
VAAs are the backbone of Wormhole’s cross-chain communication. Each VAA is a signed message encapsulating important information such as the originating chain, the emitter address, a sequence number, and Guardian signatures. The Wormhole SDK leverages its layout system to define, serialize, and deserialize VAAs, ensuring data integrity and chain compatibility.
Base VAA Structure#
The Wormhole SDK organizes the VAA structure into three key components:
- Header - contains metadata such as the Guardian set index and an array of Guardian signatures
- Envelope - includes chain-specific details such as the emitter chain, address, sequence, and consistency level
- Payload - provides application-specific data, such as the actual message or operation being performed
Header layout:
const guardianSignatureLayout = [
{ name: 'guardianIndex', binary: 'uint', size: 1 },
{ name: 'signature', ...signatureItem },
] as const satisfies Layout;
export const headerLayout = [
{ name: 'version', binary: 'uint', size: 1, custom: 1, omit: true },
{ name: 'guardianSet', ...guardianSetItem },
{
name: 'signatures',
binary: 'array',
lengthSize: 1,
layout: guardianSignatureLayout,
},
] as const satisfies Layout;
The header defines metadata for validating and processing the VAA, such as the Guardian set index and signatures. Each signature is represented using the signatureItem
layout, ensuring consistency and compatibility across different platforms.
Signature Standard Compliance
The signature field uses the signatureItem
layout, which is explicitly defined as 65 bytes. This layout is aligned with widely used standards such as EIP-2612 and Uniswap's Permit2, ensuring compatibility with cryptographic protocols and applications.
Envelope layout:
export const envelopeLayout = [
{ name: 'timestamp', binary: 'uint', size: 4 },
{ name: 'nonce', binary: 'uint', size: 4 },
{ name: 'emitterChain', ...chainItem() },
{ name: 'emitterAddress', ...universalAddressItem },
{ name: 'sequence', ...sequenceItem },
{ name: 'consistencyLevel', binary: 'uint', size: 1 },
] as const satisfies Layout;
The envelope encapsulates the VAA's core message data, including chain-specific information like the emitter address and sequence number. This structured layout ensures that the VAA can be securely transmitted across chains.
Payload Layout:
The Payload contains the user-defined data specific to the application or protocol, such as a token transfer message, governance action, or other cross-chain operation. The layout of the payload is dynamic and depends on the payload type, identified by the payloadLiteral
field.
const examplePayloadLayout = [
{ name: 'type', binary: 'uint', size: 1 },
{ name: 'data', binary: 'bytes', lengthSize: 2 },
] as const satisfies Layout;
This example demonstrates a payload containing:
- A type field specifying the operation type (e.g., transfer or governance action)
- A data field that is length-prefixed and can store operation-specific information
Dynamic payload layouts are selected at runtime using the payloadLiteral
field, which maps to a predefined layout in the Wormhole SDK.
Combined Base Layout:
The base VAA layout combines the header, envelope, and dynamically selected payload layout:
At runtime, the payload layout is appended to the baseLayout
to form the complete structure.
Serializing VAA Data#
The Wormhole SDK provides the serialize
function to serialize a VAA message. This function combines the base layout (header and envelope) with the appropriate payload layout, ensuring the message’s format is correct for transmission across chains.
import { serialize } from '@wormhole-foundation/sdk-core/vaa/functions';
const vaaData = {
guardianSet: 1,
signatures: [{ guardianIndex: 0, signature: new Uint8Array(65).fill(0) }],
timestamp: 1633000000,
nonce: 42,
emitterChain: 2, // Ethereum
emitterAddress: new Uint8Array(32).fill(0),
sequence: BigInt(1),
consistencyLevel: 1,
payloadLiteral: 'SomePayloadType',
payload: { key: 'value' },
};
const serializedVAA = serialize(vaaData);
How does it work?
Internally, the serialize function dynamically combines the baseLayout
(header and envelope) with the payload layout defined by the payloadLiteral
. The complete layout is then passed to the serializeLayout
function, which converts the data into binary format.
Deserializing VAA Data#
The Wormhole SDK provides the deserialize
function to parse a VAA from its binary format back into a structured object. This function uses the baseLayout
and payload discriminator logic to ensure the VAA is correctly interpreted.
import { deserialize } from '@wormhole-foundation/sdk-core/vaa/functions';
const serializedVAA = new Uint8Array([
/* Serialized VAA binary data */
]);
const vaaPayloadType = 'SomePayloadType'; // The payload type expected for this VAA
const deserializedVAA = deserialize(vaaPayloadType, serializedVAA);
How does it work?
Internally, the deserialize
function uses the baseLayout
(header and envelope) to parse the main VAA structure. It then identifies the appropriate payload layout using the provided payload type or discriminator.
const [header, envelopeOffset] = deserializeLayout(headerLayout, data, {
consumeAll: false,
});
const [envelope, payloadOffset] = deserializeLayout(envelopeLayout, data, {
offset: envelopeOffset,
consumeAll: false,
});
const [payloadLiteral, payload] =
typeof payloadDet === 'string'
? [
payloadDet as PayloadLiteral,
deserializePayload(payloadDet as PayloadLiteral, data, payloadOffset),
]
: deserializePayload(
payloadDet as PayloadDiscriminator,
data,
payloadOffset
);
return {
...header,
...envelope,
payloadLiteral,
payload,
} satisfies VAA;
Registering Custom Payloads#
In the Wormhole SDK, payloads rely on layouts to define their binary structure, ensuring consistency and type safety across protocols. Custom payloads extend this functionality, allowing developers to handle protocol-specific features or unique use cases.
To learn how to define and register payloads using layouts, refer to the Building Protocols and Payloads page for a detailed guide.
Common Pitfalls & Best Practices#
When working with the Wormhole SDK layout system, it's important to be aware of a few common issues that can arise. Below are some pitfalls to avoid and best practices to ensure smooth integration.
Pitfalls to Avoid#
Defining Sizes for Data Types#
When defining sizes for each data type, make sure to match the actual data length to the specified size to prevent serialization and deserialization errors:
uint
andint
- the specified size must be large enough to accommodate the data value. For instance, storing a value greater than 255 in a single byte (uint8
) will fail since it exceeds the byte’s capacity. Similarly, an undersized integer (e.g., specifying 2 bytes for a 4-byte integer) can lead to data loss or deserialization failurebytes
- the data must match the specified byte length in the layout. For example, defining a field as 32 bytes (size: 32
) requires the provided data to be exactly 32 bytes long; otherwise, serialization will fail
// Pitfall: Mismatch between the size of data and the defined size in the layout
{ name: 'orderSender', binary: 'bytes', size: 32 }
// If the provided data is not exactly 32 bytes, this will fail
Incorrectly Defined Arrays#
Arrays can be fixed-length or length-prefixed, so it’s important to define them correctly. Fixed-length arrays must match the specified length, while length-prefixed arrays need a lengthSize
field.
// Pitfall: Array length does not match the expected size
{ name: 'redeemerMessage', binary: 'bytes', lengthSize: 4 }
Best Practices#
These best practices and common pitfalls can help prevent bugs and improve the reliability of your implementation when working with layouts in the Wormhole SDK.
Reuse Predefined Layout Items#
Rather than defining sizes or types manually, reuse the predefined layout items provided by the Wormhole SDK. These items ensure consistent formatting and enforce strong typing.
For instance, use the chainItem
layout for chain IDs or universalAddressItem
for blockchain addresses:
import {
chainItem,
universalAddressItem,
} from '@wormhole-foundation/sdk-core/layout-items';
const exampleLayout = [
{ name: 'sourceChain', ...chainItem() }, // Use predefined chain ID layout
{ name: 'senderAddress', ...universalAddressItem }, // Use universal address layout
] as const;
By leveraging predefined layout items, you reduce redundancy, maintain consistency, and ensure compatibility with Wormhole’s standards.
Use Class Instances#
Whenever possible, convert deserialized data into higher-level class instances. This makes it easier to validate, manipulate, and interact with structured data. For example, the UniversalAddress
class ensures consistent address handling:
import { UniversalAddress } from '@wormhole-foundation/sdk-core';
const deserializedAddress = new UniversalAddress(someBinaryData);
Focusing on reusing predefined layout items and converting deserialized data into higher-level abstractions can ensure a more robust and maintainable implementation.
Consistent Error Handling#
Always handle errors during both serialization and deserialization. Catching exceptions allows you to log or resolve issues gracefully when working with potentially corrupted or invalid data.
try {
const deserialized = deserializeLayout(fillLayout, data);
} catch (error) {
console.error('Deserialization failed:', error);
}
Leverage Reusable Layouts#
Creating reusable layouts for commonly repeated structures improves code maintainability and reduces duplication. These layouts can represent fields or combinations of fields frequently encountered in cross-chain communication, such as chain IDs, addresses, and signatures.
For example, define a reusable layout for chain IDs and addresses:
const commonLayout = [
{ name: 'chainId', binary: 'uint', size: 2 },
{ name: 'address', binary: 'bytes', size: 32 },
] as const satisfies Layout;
// Reuse the common layout in different contexts
const exampleLayout = [
...commonLayout,
{ name: 'sequence', binary: 'uint', size: 8 },
];
By abstracting common elements into a single layout, you ensure consistency across different parts of your application and simplify future updates.
Performance Considerations#
Efficient serialization and deserialization are crucial when handling large amounts of cross-chain data. Below are some strategies and best practices to ensure optimal performance when using Wormhole SDK layouts.
Lazy Instantiation#
Building a discriminator can be resource-intensive for complex or large datasets. The layout structures do not incur significant upfront costs, but deferring the creation of discriminators until needed can improve efficiency.
This approach ensures that discriminators are only built when required, helping to optimize performance, especially for complex or conditional layouts.
Resources#
For further learning and practical experience, explore the following resources:
-
Wormhole TypeScript SDK - the Wormhole SDK repository contains the core implementation of layouts, including predefined layout items and utilities like
serializeLayout
anddeserializeLayout
-
Layout tests repository - for hands-on experimentation, check out this layout package repository, which provides examples and unit tests to help you better understand serialization, deserialization, and the strong typing mechanism. Running these tests locally is a great way to deepen your understanding of how layouts function in real-world scenarios