This document describes the current messaging protocol between domains in a trusted code environment. This protocol describes messaging between the consensus chain and any domain and between two domains.
Chain
A chain is a blockchain within the Subspace Network. A chain is identified as the Consensus chain or a Domain with a DomainID
pub enum ChainId {
Consensus,
Domain(DomainId),
}
Domain
A Domain is a blockchain with some application modules. These applications act as senders and receivers of the messages using messaging protocol. A unique identifier identifies each application.
Domain operators execute transactions bundled by other operators of the same domain when a new block is available.
Trusted third party
A trusted third party is the consensus chain, the farmer network for block production. Domain uses consensus chain to submit transaction bundles, verify the Message proofs of domain_a on domain_b and submit fraud proofs.
Channel
A Channel is a bi-directional connection between two domains. Channel connection is established when the src_chain_id initiates the channel connection open message, and dst_chain_id responds with either approval or rejection. Once a connection is open, sending messages back and forth is possible. A Channel would be open until a maximum number of messages are sent. This could be configured or defaulted to the maximum possible value of the Nonce type.
There is a deposit to open a channel between domains. Deposit should be high enough to discourage and make it economically inefficient to DDOS channel initiation connections between domains.
A channel can be closed on either end of the domain by the root user. Once closed, the channel will stop sending and receiving any further messages. The Relayer will communicate to the other domain to close the channel and clean up.
A channel could be in one of the following states (State):
Initiated: When a channel is initiated but has not received acknowledgment from the other domain.Open: When a bi-directional channel is open to relay messages between domains.Closed: When the channel is closed between the domains and stops receiving and sending new messages to the other domain.A Channel is defined as follows
type Channel {
// Unique channel identifier within DomainID namespace
channel_id: ChannelID
// State of the channel
state: State
// Next valid Inbox nonce
next_inbox_nonce: Nonce
// Next valid Outbox nonce
next_outbox_nonce: Nonce
// Latest outbox message nonce for which response was received from dst_chain.
latest_response_received_message_nonce: Nonce
// Fee Model for this channel
fee: FeeModel
// Max number of messages to be in outbox at a given time on both domains.
max_outgoing_messages: u32
}
Channel Inbox
All the incoming messages to the domain are validated and added to a pool before processing. If specific message arrived earlier than a previous message, it is stored until the previous message(s) is processed in the order of Nonce.
Channel Outbox
All messages originated from src_chain_id to a dst_chain_id will be added to this queue in the runtime state.
Messages stay in the outbox of src_chain_id until the domain block of src_chain_id containing the originating extrinsic is out of the challenge period or reached archiving depth (if src_chain_id is consensus chain).
There is also a notion of back pressure by limiting maximum number of messages queued max_outgoing_messages to outbox of src_chain_id. So when dst_chain_id doesn’t send any message responses, this should throttle the outbox until normal operation. Message is removed from the outbox once the message response is received from the dst_chain_id.
Channel response queue
All the message responses to the messages in the outbox are validated and added to this queue. The message responses are passed to the application units within the domain. If a response for message arrived earlier that previous message responses, then this response is stored until the previous message responses are delivered.
Channel Nonce
Channel nonce is used to order messages with in the channel and to avoid replay attacks.
A domain maintains 2 nonces for each domain and channel:
Incoming nonce: This nonce is used to order the incoming messages to the domain through the channel from other domain. The nonce starts at 0 and is incremented after each received message.Outgoing nonce: This nonce is used to order the outgoing messages from this domain to the other domains. The nonce starts at 0 and is incremented after every sent message.Message Proof
Message proof that can verify the validity of the message from the point-of-view of the consensus chain. Proof combines the storage proofs to validate messages.
The proof consists of
pub struct MMRProof {
// consensus block number below archiving depth at which this MMR proof was generated
consensus_block_number
// Leaf data that contains consensus storage root
// storage root is used to verify the `ConfirmedDomainBlocks` storage
leaf_data
// merkle proof for this MMR
proof
}
pub struct Proof<BlockNumber, BlockHash, StateRoot> {
// MMR proof, which provides the state root of consensus hash
MMRProof
/// Storage proof that src chain state_root is confirmed on Consensus chain.
/// This is optional when the src_chain is Consensus.
pub domain_confirmed_proof: Option<StorageProof>,
// Storage proof that message is processed on src_chain.
pub message_proof: StorageProof,
}
Message
Message encompasses the actual message being sent and metadata about the message itself. MessageID is a unique tuple of (ChannelID , Nonce). There are two types of Message payloads:
Protocol Payload
This payload is used by the protocol to open or close, acknowledge channel connection with other domain.
Endpoint Payload
This payload is used by the protocol to pass messages between endpoint on the src and dst domains.
pub struct Message<Balance> {
/// Chain (consensus or domain id) that initiated this message.
pub src_chain_id: ChainId,
/// Chain (consensus or domain id) this message is intended for.
pub dst_chain_id: ChainId,
/// ChannelId the message was sent through.
pub channel_id: ChannelId,
/// Message nonce within the channel.
pub nonce: Nonce,
/// Payload of the message
pub payload: VersionedPayload<Balance>,
/// Last delivered message response nonce on src_chain.
pub last_delivered_message_response_nonce: Option<Nonce>,
}
The response message follows the same structure except the payload contains either Protocol(Response) or Endpoint(Response)
Message Lifecycle
Conceptually, a message could be in one of the following states during its lifecycle:
From the POV of src_chain_id:
dst_chain_id. It stays in the outbox until the message-response is received.From the POV of dst_chain_id:
src_chain_id containing the originating extrinsic is out of the challenge period or reached archiving depth (if src_chain_id is consensus chain).Relayer Component
A relayer component relays message from src_chain_id to dst_chain_id. Domain operators have builtin relayer to relay messages from the domain to other domains and the consensus chain.
Operators on domain_a relay messages originating in domain_a to the consensus network and listen for messages destined to domain_a from any other domain. Messages are sent through the consensus network where all operators of all domains are present.
The payload for the extrinsic could be a message-request or a message-response.
Fees
Fees are collected from the sender of the message on src_chain_id to pay for relay and execution of their message on both src_chain_id and dst_chain_id respectively.
Compute fees are computed based on weights of the exact calls performed on both src_chain_id and dst_chain_id in total. Collected compute fees for the portion of execution happening on src_chain_id is paid to operators of src_chain_id and the compute fees for the portion of execution happening dst_chain_id. The portion of fees that is to be distributed on dst_chain_id is burned on src_chain_id when message is added to outbox.
The burnt fees are subtracted from src_chain_id bookkeeping balance (if it’s a domain).
The relay fee is split equally among operators on src_chain_id and dst_chain_id who have submitted the ER that includes the message).
On the source chain, this reward is distributed when the message gets the response from the dst_chain_id. On the dst_chain_id, when it receives the next message, it will collect all the messages that are marked delivered on src_chain_id, mints the funds, and, distributes the rewards to the relayer pool on dst_chain_id for each message.
The minted fees are added to dst_chain_id bookkeeping balance (if it’s a domain).
Outbox Message Fees
sender sends a message from src_chain_idsrc_chain_id collects fees to be paid by sender as follows:
dst_chain_id. This amount is burnt on src_chain_id and minted on dst_chain_id later.dst_chain_id. This amount is burnt on src_chain_id and minted on dst_chain_id later.src_chain_id.src_chain_id for relaying the response.dst_chain_id.dst_chain_id, src_chain_id distributes the rewards from sender to operators.dst_chain_id as last_delivered_message_response_nonce as an acknowledgement so that it can rewards its operators.Inbox Message Fees
dst_chain_id receives a message from src_chain_iddst_chain_id mints received fees after message validationsrc_chain_id, the dst_chain_id distributes the fees from sender equally to operators who have submitted the ER containing the message extrinsicThe following describes the generic message from one domain to another. This message could be a protocol message to initiate or close channel connection or an endpoint specific message through an established Channel. In either case, the base message passing remains same:
nonce is added to the outbox of src_chain_id with a runtime event issued.