Introduction

Welcome to the Cardano Blueprint, a project that creates the knowledge foundation about how the Cardano protocol works. Blueprints are implementation independent assets like explanations, diagrams, interface specifications, test data, etc. that will enable a wide developer audience to understand the protocol and build Cardano components.

Goal

Our goal is to make the Cardano protocol documentation and specifications implementation-independent and accessible to a wider audience of builders in the Cardano community.

Ultimately, sharing knowledge will support node diversity and is the only way to maintain security of the Cardano network as its software stack becomes more and more decentralized.

Why it’s needed

The cardano-node was developed over the last 8+ years at Input Output Group (IO) to become the reference implementation of the Ouroboros network and consensus protocols, the extended UTxO (eUTxO) ledger model and Plutus smart contract language.

Most of these things are rigorously researched, specified and documented, but the documentation is spread across multiple repositories, in different formats, some in very dense formal methods syntax, and some mixed with details of the Haskell implementation.

This project aims to produce a set of blueprints for Cardano in a grassroots initiative to make existing documentation:

understandable by a wide audience
owned by the Cardano community
useful to multiple implementations

Where the audience includes primarily developers of cardano nodes, current and future implementations, but also builders of applications and integrations, or anyone wanting to understand Cardano at a deeper technical level.

What is a blueprint

We understand that not one format, style or type of specification will work for everyone. But no matter how exactly an artifact for a given protocol aspect turns out, blueprints should capture the following values:

Accessible - understandable language, human readable formats, maintainable diagrams
Open - easy to contribute to by people from different backgrounds, common tools
Minimal - describes required functionality and behaviour, not implementation details
Lightweight - easy to use, reference, and test against
Evidence-based - contains test scenarios, test data, simulations, models or similar to rely on

Given these values, we believe an explanation of key concepts like network protocols, consensus algorithms, block and transaction formats or how transactions in Cardano are validated using Markdown, rendered into a website which can be searched and linked to, ideally with lots of updatable diagrams, e.g. using mermaid, is already a great starting point!

If those documents are then also providing an introduction and home to various test data sets, json schemas, cddl definitions, or test suites in the spirit of ethereum/tests, then this will be a great asset to the Cardano community.

Finally, hosting this on Github means that it can become a community effort with familiar processes of contribution to an Open Source software project like pull requests, issues, discussions etc. Note that this would only present a thin layer of our ways of working and should we want to move or work on blueprints differently, for example in a more decentralized radicle way. Even changing the ways of making knowledge available - e.g. producing a book or learn Cardano concepts with an LLM agent - are possible, as long as we capture the essence of what makes Cardano.

What about Cardano Improvement Proposals (CIPs)?

The Cardano Improvement Proposal (CIP) process is the standard way that new features are proposed, discussed and ratified for the Cardano network, and it does this job well. In fact, CIPs to capture a lot of our values already and tick many boxes.

We did consider whether these blueprints could be CIPs themselves, but were concerned that the sheer volume of information could overwhelm the technical and personal capacity of the CIP process, particularly in the initial bootstrap phase. Also, the single-layer, procedural nature of the CIP documents could be restrictive for the highly-connected and aggregated form of documentation we envision.

That being said, this project will of course tightly integrate with the CIP process:

CIPs will of course remain the place for new features and discussion
The Cardano blueprint itself may be presented and ratified as a CIP, given the Cardano community final say over its status
CIPs could include changes to the cardano-blueprint in their “Path to Active”

graph TB
    COM((Community))
    BP(Blueprints)
    N1[Node Implementation 1]
    N2[Node Implementation 2]
    C[Component 3]

    COM <-..-> BP
    BP --> N1
    BP --> N2
    BP --> C

Network

The network layer of the Cardano protocol handles two aspects of communication:

Node-to-node (N2N) - transmission of data between network nodes
Node-to-client (N2C) - integration of application clients to a single node

graph LR
    A(Node)
    B(Node)
    C(Node)
    X[Client]
    Y[Client]
    A <-->|N2N| C
    A <-->|N2N| B
    B <-->|N2N| C
    X <-->|N2C| A
    C <-->|N2C| Y

The network protocols consist of a multiplexing layer which carries one or more mini-protocols, according to the type of connection - for example:

graph TB
    HS(Handshake)
    CS(ChainSync)
    BF(BlockFetch)
    Mux[Multiplexing]
    Con([Raw Connection])

    HS <--> Mux
    CS <--> Mux
    BF <--> Mux
    Mux <--> Con

Shared mini-protocols

These protocols are used in both N2N and N2C modes:

Handshake - for connection and version negotiation
Chain Synchronization - for synchronization of changes to the Cardano chain¹

Node-to-node mini-protocols

These protocols are only used for node-to-node communication:

Block Fetch - for transferring chain blocks between nodes
Transaction Submission - for propagating transactions between nodes
Keep Alive - for maintaining and measuring timing of the connection
Peer Sharing - for exchanging peer information to create the peer-to-peer (P2P) network

Node-to-client mini-protocols

These protocols are only used for node-to-client communication:

Local State Query - for querying ledger state
Local Tx Submission - for submitting transactions locally
Local Tx Monitor - for monitoring transactions

Dummy mini-protocols

These protocols are only used for testing and experimentation:

Ping-Pong - a simple presence test
Request-Response - a generic mechanism for exchanging data

ChainSync is shared between N2N and N2C, but shares full blocks in N2C as opposed to just headers in N2N

Network: Multiplexing

The multiplexing layer is a simple binary protocol which runs on top of the raw connection (TCP or local socket) and provides:

Multiplexing of multiple mini-protocols¹ over a single connection
Framing and segmentation of messages within a stream connection
Timing information for latency measurement

This shows the arrangement for a typical node-to-node (N2N) connection:

graph LR
  subgraph n1 [Node 1]
    direction LR
    HS1(Handshake)
    CS1(ChainSync)
    BF1(BlockFetch)
    Mux1[Multiplexer]

    HS1 <--> Mux1
    CS1 <--> Mux1
    BF1 <--> Mux1
  end

  subgraph n2 [Node 2]
    direction LR
    Mux2[Multiplexer]
    HS2(Handshake)
    CS2(ChainSync)
    BF2(BlockFetch)

    Mux2 <--> HS2
    Mux2 <--> CS2
    Mux2 <--> BF2
  end

  Mux1 <==> Mux2

Packet format

A multiplexer packet consists of an 8-byte header followed by up to 65535 bytes of payload. Multiple payload segments can be combined to form a full message.

packet-beta
    0-31: "Transmission time"
    32: "M"
    33-47: "Mini-protocol ID"
    48-63: "Payload length N"
    64-95: "Payload (variable length N)"

Field	Size	Meaning
Transmission time	32	Monotonic time stamp (µsec, lowest 32 bits)
M	1	Mode: 0 from initiator, 1 = from responder
Mini-protocol ID	15	Mini-protocol ID (see below)
Payload length	16	Segment payload length (N) in bytes
Payload	N	Raw payload data

All fields are network/big-endian byte order.

warning

How are multi-segment messages delimited? - there is no ‘start of message’ flag or ‘N of M’ counter

Although the multiplexer is only used with mini-protocols in Cardano, it’s actually completely agnostic as to data format.

Network: Mini-protocols

The Cardano mini-protocols are a set of protocols that each provides a particular aspect of the communication between nodes (node-to-node or N2N) or between a node and an application client (node-to-client or N2C). They run over a multiplexer which allows multiple mini-protocols to share the same underlying TCP or local socket connection.

Each mini-protocol is represented by a state machine and a set of messages that can be passed between the parties.

State machines

The progress of the communication is defined by a state machine, which is replicated at each end. The transitions of the state machine are messages being sent/received. As well as defining which messages are valid to send and receive in each state, the state machine also defines which side has agency - that is, should be the one to send the next message.

The initiator of a connection is the one that requested the connection be opened - the client in a simple client/server model.

The responder or is the one that responds to the connection request - the server, in other words.

In every case it is the initiator (client) which has agency first. In many cases the initiator and responder take turns to have agency (send messages), but in some cases where one party must wait for a response, the other will keep agency and send a follow-up message later.

We can draw this state machine in the standard way, with circles and arrows, but with the addition of an indicator of which side has agency. This one is for the minimal example mini-protocol, Ping Pong:

stateDiagram

    [*] --> StIdle
    StIdle --> StBusy: MsgPing
    StBusy --> StIdle: MsgPong
    StIdle --> [*]: MsgDone

    direction LR

    classDef initiator color:#080
    classDef responder color:#008, text-decoration: underline
    class StIdle initiator
    class StBusy responder

It has been the convention to mark states where the initiator has agency in green and the responder in blue, as here, but we also underline it for responder agency in case colours aren’t clear.

As a double check, we can show the agency for each state as a table as well:

State	Agency
StIdle	Initiator
StBusy	Responder

By convention state names have an St prefix, while messages have Msg, to avoid confusion.

We can also show the transitions of the state machine as a table, and indicate what data is passed with each message, although Ping Pong doesn’t carry any:

From state	Message	Parameters	to state
StIdle	MsgPing	-	StBusy
StBusy	MsgPong	-	StIdle
StIdle	MsgDone	-	End

Message formats

The messages of the mini-protocols are encoded in CBOR, a compact binary encoding of JSON. The syntax of valid messages is expressed in CDDL (Concise Data Definition Language).

Here’s the CDDL for the node-to-node handshake protocol:

;
; NodeToNode Handshake, v7 to v10
;

handshakeMessage
    = msgProposeVersions
    / msgAcceptVersion
    / msgRefuse

msgProposeVersions = [0, versionTable]
msgAcceptVersion   = [1, versionNumber, nodeToNodeVersionData]
msgRefuse          = [2, refuseReason]

versionTable = { * versionNumber => nodeToNodeVersionData }

versionNumber = 7 / 8 / 9 / 10

nodeToNodeVersionData = [ networkMagic, initiatorOnlyDiffusionMode ]

; range between 0 and 0xffffffff
networkMagic = 0..4294967295
initiatorOnlyDiffusionMode = bool

refuseReason
    = refuseReasonVersionMismatch
    / refuseReasonHandshakeDecodeError
    / refuseReasonRefused

refuseReasonVersionMismatch      = [0, [ *versionNumber ] ]
refuseReasonHandshakeDecodeError = [1, versionNumber, tstr]
refuseReasonRefused              = [2, versionNumber, tstr]

Network: Handshake mini-protocol

Mini-protocol number: 0

The Handshake mini-protocol is used to establish a connection and negotiate protocol versions and parameters between the initiator (client) and responder (server). There are two versions, one for node-to-node (N2N) and one for node-to-client (N2C), which differ only in their protocol parameters.

State machine

stateDiagram

    [*] --> StPropose
    StPropose --> StConfirm: MsgProposeVersions
    StConfirm --> [*]: MsgAcceptVersion
    StConfirm --> [*]: MsgReplyVersion
    StConfirm --> [*]: MsgRefuse

    direction LR

    classDef initiator color:#080
    classDef responder color:#008, text-decoration: underline
    class StPropose initiator
    class StConfirm responder

State agencies

State	Agency
StPropose	Initiator
StConfirm	Responder

State transitions

From state	Message	Parameters	to state
StPropose	MsgProposeVersions	`versionTable`	StConfirm
StConfirm	MsgReplyVersion	`versionTable`	End
StConfirm	MsgAcceptVersion	`(versionNumber, versionData)`	End
StConfirm	MsgRefuse	`reason`	End

TCP simultaneous open

In the rare case when both sides try to connect to each other at the same time, it’s possible to get a “TCP simultaneous open” where you end up with a single socket, not two. In this case, both sides will think they are the initiator so will send a MsgProposeVersions, and this protocol handles this by treating the received one in StConfirm state as a MsgReplyVersion, which has the same CBOR encoding.

warning

Why does the message need to change name? The state machine would be valid with an StConfirm -- MsgProposeVersions --> End arc.

note

Also, is the negotiation always deemed successful in this case? What if one side can’t accept the other’s version? (there is talk of resetting the connection)

warning

MsgReplyVersion is no longer mentioned in the CDDL - is this therefore out of date?

Messages

The MsgProposeVersions message is sent by the initiator to propose a set of possible versions and protocol parameters. versionTable is a map of version numbers to associated parameters - bear in mind that different versions may have different sets of parameters. The version number keys must be unique and in ascending order.

note

This seems an arbitrary constraint which could easily be avoided by implementations, although deterministic CBOR encoding would enforce it.

The MsgAcceptVersion message is returned by the responder to confirm a mutually acceptable version and set of parameters.

The MsgRefuse message is returned by the responder to indicate there is no acceptable version match, or another reason. If it is a version mismatch it returns a set of version numbers that it could have accepted.

warning

The content of MsgRefuse is inconsistent between the paper and CDDL - check the above.

Message size limits

Because the Handshake protocol operates before the multiplexer is fully set up, the messages must not be split into segments, and this imposes a size limit of 5760 bytes.

warning

This seems like a protocol level mix, and since the negotiated parameters don’t seem to affect the mux config (and could be changed dynamically even if they did), it’s not clear why this constraint is needed.

note

Why 5076 when the mux protocol can handle 65535? Implementation detail?

Timeouts

The maximum time to wait for a message in StPropose (for the responder) or StConfirm (for the initiator) is 10 seconds. After this the connection should be torn down.

CDDL

Here’s the CDDL for the latest node-to-node handshake protocol:

;
; NodeToNode Handshake (>=v13)
;
handshakeMessage
    = msgProposeVersions
    / msgAcceptVersion
    / msgRefuse
    / msgQueryReply

msgProposeVersions = [0, versionTable]
msgAcceptVersion   = [1, versionNumber, nodeToNodeVersionData]
msgRefuse          = [2, refuseReason]
msgQueryReply      = [3, versionTable]

versionTable = { * versionNumber => nodeToNodeVersionData }

versionNumber = 13 / 14

nodeToNodeVersionData = [ networkMagic, initiatorOnlyDiffusionMode, peerSharing, query ]

; range between 0 and 0xffffffff
networkMagic = 0..4294967295
initiatorOnlyDiffusionMode = bool
; range between 0 and 1
peerSharing = 0..1
query = bool

refuseReason
    = refuseReasonVersionMismatch
    / refuseReasonHandshakeDecodeError
    / refuseReasonRefused

refuseReasonVersionMismatch      = [0, [ *versionNumber ] ]
refuseReasonHandshakeDecodeError = [1, versionNumber, tstr]
refuseReasonRefused              = [2, versionNumber, tstr]

And the node-to-client version:

;
; NodeToClient Handshake
;

handshakeMessage
    = msgProposeVersions
    / msgAcceptVersion
    / msgRefuse
    / msgQueryReply

msgProposeVersions = [0, versionTable]
msgAcceptVersion   = [1, versionNumber, nodeToClientVersionData]
msgRefuse          = [2, refuseReason]
msgQueryReply      = [3, versionTable]

; Entries must be sorted by version number. For testing, this is handled in `handshakeFix`.
versionTable = { * versionNumber => nodeToClientVersionData }


; as of version 2 (which is no longer supported) we set 15th bit to 1
;               16    / 17    / 18    / 19
versionNumber = 32784 / 32785 / 32786 / 32787

; As of version 15 and higher
nodeToClientVersionData = [networkMagic, query]

networkMagic = uint
query        = bool

refuseReason
    = refuseReasonVersionMismatch
    / refuseReasonHandshakeDecodeError
    / refuseReasonRefused

refuseReasonVersionMismatch      = [0, [ *versionNumber ] ]
refuseReasonHandshakeDecodeError = [1, versionNumber, tstr]
refuseReasonRefused              = [2, versionNumber, tstr]

The Consensus Layer

This document describes the components of the Consensus layer of a Cardano node, serving as a reference for Cardano developers who want to implement a node or interact with the Consensus layer of an existing implementation. We strive to provide implementation-agnostic requirements and responsibilities for the Consensus layer.

warning

This document is a work in progress.

The Consensus Layer runs the Consensus Protocol and invokes the Ledger layer to validate chains produced according to the protocol. The chain is then persisted in the Storage layer. Such chains are diffused using the Networking layer. The contents of new blocks are provided by the Mempool.

flowchart TD
    NN1("Node") --> NTN1
    NN2("Node") --> NTN1
    NN3("Node") --> NTN1

    NTN2 --> N6("Node")
    NTN2 --> N4("Node")
    NTN2 --> N5("Node")

    NTC <--> CL("Client (e.g. wallet)")

    subgraph "Node";
      subgraph NTN1 ["Network NTN (upstream)"];
          M1("ChainSync")
          M2("BlockFetch")
          M3("TxSubmission2")
      end
      M1 --> |block headers| A("Consensus")
      M2 --> |block bodies| A("Consensus")
      subgraph NTN2 ["Network NTN (downstream)"];
          N1("ChainSync")
          N2("BlockFetch")
          N3("TxSubmission2")
      end

      M3 --> |transactions| G
      D --> |block headers| N1
      D --> |block bodies| N2
      A("Consensus") <--> |blocks| D("Storage")
      G <--> |apply transactions| C
      A("Consensus") <--> |apply blocks| C("Ledger")
      A("Consensus") <-->|transactions snapshot| G("Mempool")
      G --> |transactions| N3
      G <--> |transactions| O2
      A("Consensus") <-->|state queries| O1
      subgraph NTC [Network NTC];
          O1("LocalStateQuery")
          O2("TxSubmission2")
      end

    end

The Consensus Protocol in Cardano

The consensus protocol has three main responsibilities:

Chain validity check: the validity of a chain of blocks is defined by the Consensus protocol, whether the values in the block match are as expected. This involves things like singature checking, checking the previous hash, ensuring the header is consistent, etc.
Chain selection: Competing chains arise when two or more nodes extend the chain with different blocks. This can happen when nodes are not aware of each other’s blocks due to temporarily network delays or partitioning, but depending on the particular choice of consensus algorithm it can also happen in the normal course of events. When it happens, it is the responsibility of the consensus protocol to choose between these competing chains.
Leadership check and block forging: In proof-of-work blockchains any node can produce a block at any time, provided that they have sufficient hashing power. By contrast, in proof-of-stake time is divided into slots, and each slot has a number of designated slot leaders who can produce blocks in that slot. It is the responsibility of the consensus protocol to decide on this mapping from slots to slot leaders.

The Ledger layer, upstream from the Consensus layer, has traditionally divided development in several eras. Eras are names that designate major versions of the network. Each era uses a different set of rules, mostly extending the rules from the previous era with new constructs and rules or implementing completely new features. The general interface of the Ledger layer is common to all eras, and that is all that Consensus interacts with. See this table for the list of eras and their specifics.

note

Era transitions are enacted on-chain through specialized transactions.

Depending on the Ledger era in effect, the Consensus protocol (which governs both chain selection and block production) is different:

Era	Protocol	Link
Byron	Ouroboros Classic	Paper
Byron (reimplementation, block forging)	Ouroboros BFT	Paper
Byron (reimplementation, block processing)	Ouroboros Permissive BFT	Section 4 of the Byron spec
Shelley	Ouroboros Transitional Praos (TPraos)	Section 12 of the Shelley spec
Allegra	Ouroboros TPraos
Mary	Ouroboros TPraos
Alonzo	Ouroboros TPraos
Babbage	Ouroboros Praos	Paper
Conway	Ouroboros Praos

Each of these protocols defines how to fulfill the responsibilities above. Regarding validity of blocks, the Consensus layer can remain oblivious to the details on validation and rely on the Ledger layer to make such judgment, based on the particular era in effect.

Header|body split

tip

It is not mandatory that every implementation follows this split, however the protocols used in the Network will use this distinction of headers and bodies, so implementations can as well consider leveraging it.

An essential and uncontroversial design refinement in any blockchain implementation is to separate block headers and block bodies:

If blocks can be almost fully validated in constant time based on looking at only a small fixed size block header, then honest nodes can validate candidate chains with a small bounded amount of work.
It also enables a design where a node can see blocks available from many immediate peers but can choose to download each block body of interest just once (from a peer of its choosing from which it is available). This saves network bandwidth.

In the case of Ouroboros, all the cryptographic consensus evidence is packed into the block header, leaving the block body containing only the ledger data, and check that the block has been signed by a node that is the slot leader. If we validate this in the context of a chain of headers, then we can establish this is a plausible candidate chain, thus we eliminate several potential resource draining attacks.

So the design at this stage involves transmitting chains of headers rather than whole blocks, and using a secondary mechanism to download block bodies of interest. This gives the reason why ChainSync and BlockFetch are separate protocols. The Consensus chain selection can look only at chains of block headers, whereas the validity check of the block body can be performed by the Ledger rules, effectively separating concerns.

Mini-protocols

The mini-protocols mentioned in this chapter of the book are one of the possible mechanisms used for data difussion. The Networking design document has many more insights on why these protocols were implemented, and how they differ from other off-the-shelf mechanisms.

Although it is conceivable having other alternative mechanisms to exchange the data, these mini-protocols are for now the common language spoken by the nodes in the Cardano network, and as such it is expected that all node implementations use them, or at least are capable of using them to communicate with the rest of the network.

Note that mini-protocols are defined in the Networking layer, but it is the Consensus layer the one that provides the data for such protocols. Mini-protocols are therefore the interface between Network and Consensus.

Resilience of the Consensus layer

Consensus must not expose meaningful advantages for adversaries that could trigger a worst-case situation in which the amount of computation to be performed would block the node. This is generally achieved by trying to respect the following principle:

The cost of the worst case should be no greater than the cost of the best case.

We don’t want to optimize for the best case because it exposes the node to DoS attacks if the adversary is capable of tricking the node into the worst case.

Requirements imposed onto the Networking/Diffusion layer

To maximize the probability of the block being included in the definitive chain, the Consensus layer has to strive to mint new blocks on top of the best block that exists in the network. Therefore it necessitates of a fast diffusion layer for blocks to arrive on time to the next block minters.

Requirements imposed onto the Ledger layer

The role of the Ledger layer is to define what is stored inside the blocks of the blockchain. It is involved in mutating the Ledger State which is the result of applying blocks from the chain and can be used to validate further blocks or transactions. From the perspective of the consensus layer, the ledger layer has four primary responsibilities:

Applying blocks: The most obvious and most important responsibility of the ledger is to define how the ledger state changes in response to new blocks, validating blocks at it goes and rejecting invalid blocks.
Applying transactions: Similar to applying blocks, the ledger layer must also provide an interface for applying a single transaction to the ledger state. This is important, because the consensus layer does not just deal with previously constructed blocks, but also constructs new blocks.
Ticking time: Some parts of the ledger state change only due to the passage of time. For example, blocks might schedule some changes to be applied at a given slot, without the need for a block to be processed at that slot.
Forecasting: Some consensus protocols require limited information from the ledger. For instance, in Praos, a node’s probability of being elected a slot leader is proportional to its stake, but the stake distribution is something that the ledger keeps track of. This information is referred to as ledger view. We require not just that the ledger can provide a view of the current ledger state but also that it can predict what view will be for slots in the near future.

Chain validity

A chain of blocks has to be valid in order to be considered for adoption. In particular the mainnet chain is valid.

Validity is defined by induction on the length of the chain as:

The first block on the chain fragment is valid,
The tail of the chain fragment is valid.

For a block to be considered valid it has to be valid in three directions:

The envelope of the header must be valid, validity in this dimension is a Consensus responsibility.
The header must be valid, validity in this dimension is defined by the Consensus Protocol in effect.
The body must be valid, validity in this dimension is defined by the Ledger era in effect. This check belongs to the Ledger layer so we will omit its details in the rest of the document.

From the description below, we omit the Ouroboros Classic case as such protocol has been effectively retired once (P)BFT were implemented. On the original implementation of Ouroboros Classic, the concept of Epoch Boundary Blocks (EBBs) was introduced, blocks which were made redundant by Ouroboros BFT. Such blocks had the peculiarity of sharing the slot number with the parent block and the block number with the subsequent block.

warning

TODO: this whole page could probably benefit of formal specs.

The Envelope of a header

The Envelope of a header consists of the ledger-independent data of a block, such as block number, slot, hash, shape of the block, size, … These checks are independent of the actual Consensus Protocol in effect and are used as a first sanity check on received blocks.

An envelope is valid if:

The block number is greater or equal (if the previous one was an EBB) to the previous one,
The slot number is greater or equal (if it is an EBB) to the previous one,
The hash of the previous block is the expected one,
If the block is a known checkpoint, check that it matches the information of such known checkpoint,
The era of the header matches the era of the body,
In Byron, the block is not an EBB when none was expected,
In Shelley:
- The protocol version in the block is not greater than the maximum version understood by the node (note that this check depends on each node’s version, so this check might pass in up-to-date nodes and fail in out-of-date nodes for the same block).
- The header is no larger than the maximum header size allowed by the protocol parameters,
- The body is no larger than the maximum body size allowed by the protocol parameters.

Ouroboros BFT

In Ouroboros BFT, a block is a tuple $B_{i} = (h, d, s l, σ_{s l}, σ_{block})$ where

$h$ is the hash of the previous block,
$d$ is a set of transactions,
$s l$ is a (slot) time-stamp,
$σ_{s l}$ is a signature of the slot number, and
$σ_{block}$ is a signature of the entire block.

We can reorganize the contents of a block in header and body following the Header|body split. The body would contain the list of transactions $d$ , and the header would contain all the rest of the data.

A block is said to be valid if:

The header is valid:
- Signatures are correct, both for the slot and for the entire block,
  - The issuer of the block was delegated in the Genesis block,
  - The issuer has not signed more than the allowed number of blocks recently (TODO: where does this come from? the signature scheme? It doesn’t seem to appear in the paper)
- The slot of the block is greater than the last signed slot in the chain,
- $h$ is indeed the hash of the previous block,
The body, $d$ , is a valid sequence of transactions to be applied on top of the Ledger State resulted from applying all previous blocks since the Genesis block. Notice the validity of these transactions is defined in the Ledger layer.

See Figure 1 in the paper.

Ouroboros PBFT

Ouroboros PBFT has two separate rules for validity of blocks, depending on whether they are Epoch Boundary Blocks or Regular Blocks:

An Epoch Boundary Block is valid if its header is valid.
A Regular Block is valid if it is valid for Ouroboros BFT.

Ouroboros Praos

TODO

Ouroboros TPraos

TODO

Skipping the validation checks on trusted data

Blocks can be applied much more quickly if they are known to have been previously validated, as the outcome will be exact same, since they can only validly extend a single chain.

This can be leveraged to skip such checks when replaying a chain that has already been validated, for example when restarting a node and having to replay the chain from scratch.

Chain selection

Chain selection is the mechanism, specified in the Protocol associated to the current era in effect in the network, of identifying and and adopting the best chain in the network. A node might receive distinct chains from its peers. This situation is expected during normal operation of the Praos protocol and its derivatives.

It is important to note that all nodes participating in the network are in charge of this responsibility, as opposed to producing blocks, which only some nodes will be involved in doing (namely those with forging credentials, the cryptographic keys for signing new blocks on behalf of a stake pool). This implies that even non-block-forging nodes contribute to the security of the network as a whole.

Because each node performs chain selection, the Cardano network acts as a forward-filtering diffusion network.

The block headers come from ChainSync and the block bodies come from BlockFetch.

note

Chain selection is performed based on data from the previous headers, so there is some state (specific to each protocol) that influences the selection, which we will call chain state, as opposed to the state of the ledger or on-chain state, which we will call ledger state.

It is the decisions from running chain selection the ones that mutate the data in the Storage layer. Also the newly selected chain will influence the Mempool as transactions will be revalidated against this new chain.

Forecast range

TODO: Explain that the ledger can predict the view inside the forecast range. How long is it? What bounds it?

The validity of the headers of a candidate can be checked using the Chain State and Ledger State at the intersection point, as long as the distance in slots between the intersection and the candidate header is no more than the forecast range. For example, in this chains, the headers of the blocks in the candidate that are no further than the forecast range in slots from block 1 could be validated. In Praos, the forecast range is set to 3k/f = 129,600 slots.

---
config:
  gitGraph:
    mainBranchName: "selected chain"
---
gitGraph
    commit
    commit
    branch candidate
    checkout candidate
    commit
    commit
    checkout "selected chain"
    commit
    checkout candidate
    commit
    commit
    checkout "selected chain"
    commit

This is possible because the parts of the ledger state necessary to validate a header were completely determined some positive number of slots ago on that header’s chain.

The `k` security parameter

Both Ouroboros Classic and Ouroboros Praos are based on a chain selection rule that imposes a maximum rollback condition: alternative chains to a node’s current chain that fork off more than a certain number of blocks ago are never considered for adoption. This limit is known as the security parameter, and is usually denoted by k, which on mainnet is currently set to 2160 blocks. Analysis from research has shown this parameter ensures nodes will not diverge more than k blocks under honest circumstances with sufficiently high probability.

Ouroboros BFT does not impose a maximum rollback, but adding such a requirement does not change the protocol in any fundamental way.

The Consensus layer must evaluate the validity of any candidate chain. To evaluate the validity of a particular header, a Chain State at the predecessor header is required. To evaluate the validity of a particular block, a Ledger State at the predecessor block is required.

tip

Along the selected chain, the combined Ledger and Chain states should be kept in memory for all the k volatile blocks (and for the immutable tip, so k+1 states). This way, evaluation of candidates can be performed in a timely manner.

Making use of this parameter, the chain can be subdivided into the Immutable and Volatile parts of the chain:

flowchart RL
    subgraph "Volatile chain";
      subgraph "current selection";
        Tip -->|...| Km1["(k-1)-th Block"]
        Km1 --> K["k-th Block"]
      end

      O1["Block"] --> O2["Block"]
      O3["Block"] --> K

      O4["Block"]
    end

    subgraph "Immutable chain";
     Kp1["(k+1)-th Block"] --> Kp2["(k+2)-th Block"]
     Kp2["(k+2)-th Block"] --> |...| A[Genesis]
    end
    K --> Kp1

The Immutable part can be persisted in the storage as Ouroboros guarantees it will not change, whereas the selection in the Volatile part might change as new blocks arrive, creating new candidates.

The Chain Selection Rule

warning

TODO: this section could probably benefit of formal specs.

The specific rule for determining the best chain is dictated by the active era’s protocol.

Ouroboros Classic

In Ouroboros Classic, the protocol defines the function $maxvalid_{S} (C, C)$ for choosing the best chain considering the current selected chain $C$ and the set of valid chains $C$ available in the network. The rule states that the chosen candidate must be the longest one, breaking ties in favor of $C$ .

See the definition of $maxvalid$ in section 4.1 in the paper.

Ouroboros BFT and PBFT

In Ouroboros BFT, the protocol declares that a node should replace the local chain $B_{0} B_{1} \dots B_{l}$ with a different chain $B_{0} B^{'}_{1} \dots B^{'}_{s}$ if $s > l$ , therefore if the candidate is longer than the current selection.

See Figure 1 in the paper.

Ouroboros PBFT does not change the chain selection rules of BFT, just focuses on validity of blocks, making parts of the block superfluous. It therefore also uses length to choose among candidates.

Ouroboros TPraos and Praos

Ouroboros Praos circles back to defining the $maxvalid_{S} (C, C)$ function, with identical behaviour to that in Ouroboros Classic. This means that longer candidates are preferrable to shorter ones, and that ties are broken in favour of the already selected candidate.

See Figure 4 and the definition of $maxvalid$ above it in the paper.

Ouroboros TPraos refines how blocks are produced, but doesn’t modify the $maxvalid$ function in any way.

Tie breakers in Praos and older protocols

The rules above do specify that in case of a tie with the already selected chain, the node should not alter its selection however this leads to some detrimental consequences for the network: chain diffusion times influence which chain is adopted first and therefore would win chain selection against any other equally long chain.

Some stake pool operators then would have the unfair incentive of gathering in very close data centers or even in the same one, to make their chains arrive as fast as possible to other peers that would adopt those chains and then reject alternative chains.

This is bad for Cardano as a whole as it goes against geographical decentralization of the network.

To prevent this incentive from disrupting the chain, some tie breaks were put in place, effectively refining the Praos chain selection rule:

Chains are first compared by length, longer chains are preferred.
If the tip of both chains is issued by the same stake pool, the one with the higher ocert is preferred. Note that for a block to be valid, the ocert can increase at most by 1.
If the tips are issued by different stake pools, the block with the lower VRF value is preferred. Up to and including Babbage this comparison was unconditional, but starting on Conway this comparison is only performed if blocks do not differ in more than $n$ slots. This ensures that blocks forged much later cannot win against already selected chains, as that would incentivize stake pools to ignore blocks from other pools if theirs can win a VRF comparison.

Ouroboros Genesis

When a node is syncing with the network, there is little benefit in having it participate in producing blocks or validating transactions as it doesn’t have a ledger state close enough to the tip of the chain. This in particularly means that all the responsibilities of a node are discharged from such syncing node except the duty to select the best chain it can.

On BFT/Classic, as only seven authorized Core nodes were able to make new blocks, there can be no mistake on the chosen chain (assuming Core nodes keys aren’t compromised and used to create an old adversarial chain in the Byron era).

warning

TODO: double-check ^

However, in Praos, the rule for selecting the best chain between two competing chains is fundamentally based on chain length. The protocol assumes instantaneous transmission of entire chains, meaning that the perceived length of a candidate would be its actual length. Due to fundamental real-world limitations, sending such a large amount of data instantaneously is impossible; instead, data is streamed to the syncing node. Consequently, nodes only know the length of the received prefix of the candidate chain. This creates the risk of adversaries tricking nodes into committing to an adversarial chain during synchronization.

Ouroboros Genesis is a refinement of Praos used by nodes only during network synchronization. The key point of Genesis is precisely a refinement of the Praos chain selection rule: instead of choosing based (primarily) on length of the candidate chains, it chooses based on the density of blocks at the intersection of the candidates, leveraging the property of Ouroboros that the honest chain will have more blocks than any other adversarial chain within a specific window of slots from the intersection point. More information can be found here.

This means that in order to make an correct decision about the density of candidate chains, the node must know all the blocks (or lack thereof) within a genesis window from the intersection point of the candidates. The genesis window is defined as 3k/f slots.

warning

TODO ^ is that correct?

Chain selection is stalled while the necessary information is gathered. To prevent servers from indefinitely blocking clients, two practical refinements were implemented on top of the Genesis specification:

Limit on Eagerness: TODO
Limit on Patience: TODO

warning

TODO: do we want to specify the genesis state machine? the dynamo and all that?

Once the node finishes syncing with the network, this rule gracefully converges into the usual length-based comparison used in Praos, so the node can safely switch to running only Ouroboros Praos.

Forging new blocks

Forging new blocks is the process of packing data that has not yet been included in the blockchain (pending transactions) into a fresh block. Blocks are signed by stake pool credentials, so only the nodes which are configured with such credentials would be able to forge blocks and therefore only those nodes are strictly need to implement the forging mechanism.

Forging

Every slot, the stake pool (whose keys the node was configured with) might be entitled to produce a block, depending on the rules of the particular Protocol that is running at the tip of the current chain. For Praos on the current chain on mainnet, slots have a duration of one second. The probability of a node being elected in a slot is controlled via the active slot coefficient parameter (commonly referred to as f) which on mainnet is 0.05. This means that a block is expected once every 20 slots on average.

Forging a block would look something similar to the following sequence of actions:

Determine whether my stake pool is entitled to create a block on this slot. Below we describe what this means in Ouroboros Praos. We omit the details for other protocols as they do not produce blocks anymore on Cardano mainnet.
Acquire a data to constitute the body of a block. Consensus is theoretically unaware of what data makes up the block body, and it will be the Mempool the one that specifies which data is valid and provides such data.
Pack the data into a block body, produce a block header and emit a signature.

Note that it is in the interest of the stake pool (so that its block is included in the chain) and of the network as a whole (so that the chain grows) that the process of creating a block is as fast as possible while still producing a fully valid block.

tip

To ensure the new block is actually valid, it could be fed into the Chain Selection logic, which is expected to select it and then diffuse it via the usual mechanisms. This sanity check is not a hard requirement but it is advised to be implemented. If not even the node that created the block can adopt it, sending such block to the network would be useless.

Multi-leader slots

There exists the possibility of multiple stake pools being elected in the same slot. If the nodes of both pools produce blocks, a momentary fork will exist in the chain, which we call a slot battle. Ouroboros Praos guarantees that only one of those blocks will end up in the honest chain, and that such a fork will not survive more than the time it will take for either of the candidates to grow to k blocks.

Two stake pools might be elected in very close slots, which implies that if the block of the first stake pool doesn’t arrive at the node of the second stake pool on time, it will not be considered when the second node forges its block, creating another short-lived fork. This situation is called a height battle. To avoid this situation, it is of utmost importance that the forging of a block and its diffusion through the network (first its header via ChainSync, then its body via BlockFetch) is as fast as possible.

Leadership check in Ouroboros Praos

In Ouroboros Praos, the stake distribution is what dictates the elegibility of an stake pool for being elected. The probability of a stake pool $U_{i}$ with relative stake $α_{i}$ being elected as slot leader comes from this formula:

$p_{i} = ϕ_{f} (α_{i}) = 1 - (1 - f)^{α_{i}}$

Some important properties are ensured by Praos:

There can be multiple slot leaders as the events “ $U_{i}$ is a leader for slot sl” are independant,
There can be slots with no leader,
Only a slot leader is aware that it is indeed a leader for a given slot,
The probability of being a slot leader is independent of whether the stake pool acts as a single party or splits it stake among several “virtual” parties.

warning

TODO should probably discuss the leadership schedule, how can one know it in advance, the stability periods, the parts of the epoch, which distribution is used (2 epochs ago?)

Ledger queries

warning

TODO: explain ledger queries. Mention them in the top level of consensus

Multi-era considerations

With the blockchain network evolving, the block format and ledger rules are bound to change. In Cardano, every significant change starts a new “era”. There are several ways to deal with multiple eras in the node software, associated here with some of the DnD alignments:

Chaotic Evil: the node software only ever implements one era. When the protocol needs to be updated, all participants must update the software or risk being ejected from the network. Most importantly, the decision to transition to the new era needs to happen off-chain.
- Pros:
  - the simplest possible node software.
- Cons:
  - on-chain governance of the hard-fork is impossible, as the software has no way of knowing where the era boundary is and does not even have such a concept.
  - Additionally, special care is needed to process history blocks: chain replay is likely to be handled by special code, separate from the active era’s logic.
Chaotic Good: the node software supports the current era and the next era. Once the next era is adopted, a grace period is allowed for the participants to upgrade. The decision to upgrade may happen on chain.
- Pros:
  - allows for on-chain governance of the hard fork.
- Cons:
  - supporting two eras is more difficult than one: higher chances of bugs that will cause the network to fork in an unintended way.
  - Like in the previous case, special care is needed to process historic blocks.
True Neutral: software is structured in such a way that is supports all eras.
- Pros:
  - enables massive code reuse and forces the code to be structured in the way that allows for abstract manipulation of blocks of different eras.
  - The on-chain governance of hard-forks is reflected in the code, and ideally in the types as well, making it more likely that unintended scenarios are either impossible or easily discover able through type checking and testing.
- Cons:
  - abstraction is a double-edged sword and may be difficult to encode in some programming languages.
  - Engineers require extended onboarding to be productive.

We argue that Cardano has been the True Neutral so far, which allowed to maintain the stability of the network while allowing it to change and evolve.

Having multiple eras comes with some subtleties on era boundaries that implementors need to take into account:

Time

Forecast range

The Storage Layer

The Storage layer is responsible of storing the blocks on behalf of the Consensus layer. It is also involved in serving the data for Chain diffusion.

Some of this data is volatile and relates to the candidate chains and other data is immutable and relates to the historical chain, following the principle described in the k security parameter section.

flowchart TD
    A("Storage") -- Sequential Access --> B("Immutable Chain")
    A("Storage") -- Random Access --> C("Volatile Chain")
    A("Storage") -- Random Access --> D("Recent Leder States")

    subgraph noteA ["Hot - Efficient Rollback"]
        C
        D
    end
    subgraph noteB ["Cold - Efficient Storage"]
        B
    end

Any storage system designed for Cardano must meet certain requirements for the miniprotocols and the Consensus layer to function properly:

Fast sequential access to immutable blocks: syncing peers request historical chain blocks sequentially,
Fast sequential access to current chain-selection blocks in the volatile part of the chain: peers request this information during syncing and when caught-up,
Fast switch to an alternative chain in the volatile part of the chain,
Fast identification of chains of blocks in the volatile part of the chain: even if blocks arrive in arbitrary order, the Consensus Chain Selection should be invoked to select a better candidate chain once assembled in the volatile part of the chain,
Fast node restart after shutdown without full chain replay, while supporting k-deep forks.

It is interesting to note that the storage layer does not need to provide the Durability in the ACID acronym: upstream peers will always be available to replace any blocks a node loses.

Semantics of storage mini-protocols

The mathematical model of the Ouroboros Consensus Protocols assumes instantaneous transmission (and validation) of chains, becoming instantly available even for newly joined peers. This is, even if just because of physical real-world limitations, infeasible in practice. For this reason, transmission of chains is done in a block-by-block basis instead. This allows chains to be incrementally sent to peers but comes with its own risks for newly joined peers (see Ouroboros Genesis).

Furthermore, blocks (which in the end is the data transmitted over the network) are subdivided in block headers and block bodies, as described in the Header|body split section.

The diffusion of data in the Cardano network is in fact a responsibility of the Networking layer. The diffusion of chains involves accessing the Storage layer and serving its contents to peers, however it is ultimately the Consensus layer the one that decides how the data in the Storage layer is mutated, as an outcome of Chain Selection.

Chain diffusion is a joint effort of the Consensus, Network and Storage layers.

Diffusion of chains is achieved by means of ChainSync and BlockFetch mini-protocols.

In a sense, the Storage layer has the data to provide the meaning of the messages in the mini-protocol whereas the networking layer describes the kinds of messages a protocol is composed by. It is possible to run an interaction of the protocol exchanging data that does not follow the intended semantics (for example an evil node sending all the chains it knows about instead of only the best selection). That’s why we describe here when each message should be emitted and what information it should carry.

ChainSync

ChainSync is the miniprotocol used to transmit chains of headers. It is a pull-based miniprotocol: data is transmitted only upon explicit request from the client.

The purpose of ChainSync is to enable the client to acquire and validate the headers of the server’s selected chain, and if it is better than the client’s current selection, direct BlockFetch to download the corresponding blocks.

tip

There usually is one ChainSync client per-peer connected to the node, such that the chain state of each peer is tracked independently.

The connection is abruptly terminated if the peer misbehaves. In particular, actions considered as misbehaviour are (not exclusively):

The peer violates the state machine of the protocol,
The server sends an invalid header,
The server announces a fork that is more than k blocks deep from the client’s current selection.

warning

TODO: Make this list exhaustive

The specification of the state machine of ChainSync is described in the Network documentation (design and spec).

ChainSync pipelining or pipelined diffusion

Not to be confused with protocol pipelining. The original design of ChainSync was extended with pipelining capabilities: a server can transmit a tentative header on top of the selected chain, and the invalidity of such header (or the associated body) will not cause the connection to terminate. If the client wants (by considering such header as the best known chain) it can request the body of the block via BlockFetch as usually done for any block.

This optimization is used to shorten the time it takes to diffuse chains on the network, as otherwise nodes would only announce blocks after they validated it, causing each hop through the network to be bottle-necked by validation times.

warning

TODO: expand pipelining explanation, possibly with diagrams

There are some important considerations to take into account regarding pipelined diffusion:

Nodes can pipeline only one header which must be on top of its current selection,
If the server then validates the pipelined block and finds out it was invalid, it is encouraged to announce it promptly to its clients.

warning

TODO: Are these hard requirements?

More information can be found here.

Access pattern of ChainSync

ChainSync involves potentially serving the whole chain, both the immutable part and the volatile part (the current node’s selection). As the current selection is bound to be rolled back, the ChainSync protocol has capabilities for announcing such rollbacks to clients and following rollbacks of servers.

For the immutable part of the chain: ChainSync accesses blocks in a sequential manner, a simple iterator over such chain would suffice.
For the volatile part of the chain: ChainSync accesses the current selection in a sequential manner but such selection is bound to change if a new chain is selected. The abstraction used to implement the access to the selection must be able to follow such rollbacks.
Blocks that become immutable usually are written to the disk as they are not used for following the current chain once the node is caught up, following the description in the k security parameter section. The implementation of ChainSync should be able to identify this situation, as blocks might be gone from the volatile part of the chain as the selection advances. This does not need to be made explicit for clients but it has to be taken into account on the implementation of the server.

Codecs

The headers sent through ChainSync on the Cardano network are tagged with the index of the era they belong to. The serialization of the header proper is its CBOR-in-CBOR representation.

serialise header = <era tag><cbor-in-cbor of header>

BlockFetch

BlockFetch is the mini-protocol in charge of diffusing block bodies. It is a pull-based mini-protocol: data is transmitted only upon explicit request from the client.

tip

There is usually one BlockFetch client per peer which is the one in charge of exchanging the messages, but BlockFetch is orchestrated by a central decision component in order to minimise network usage by fetching blocks on-demand from a single peer and avoid duplicated requests.

Received blocks are then given to the chain selection logic to determine their validity and, depending on the chain selection outcome, may be incorporated into the currently selected chain.

If the peer misbehaves, the connection will be abruptly terminated. Actions that are considered misbehaving are (not exclusively):

The peer violates the state machine of the protocol,
The server provided blocks that the client did not request,
The server sends a block that does not match the header it was supposed to match,
The server sends a block with a valid header but an invalid body.

warning

TODO: Make this list exhaustive

The specification of the state machine of BlockFetch is described in the Network documentation (design and spec).

Access pattern of `BlockFetch`

The requests for blocks involve sequential portions of the chain, whether in the immutable part or the volatile part of the chain.

The only special case being when a block has become immutable due to the current chain selection growing in length. In such case the abstraction used to iterate over the blocks has to be able to find the block which now would live in the immutable storage.

Codecs

The blocks sent through BlockFetch on the Cardano network are tagged with the index of the era they belong to. The serialization of the block proper is its CBOR-in-CBOR representation.

serialise block = <era tag><cbor-in-cbor of block>

The ChainDB format

In the Haskell reference implementation, the Storage layer manages a ChainDB, whose storage components are the Immutable Database, Volatile Database and Ledger State snapshots:

Component	Responsibility
Immutable Database	Store definitive blocks and headers sequentially
Volatile Database	Store a bag of non-definitive blocks and headers. In particular it contains the blocks which, when linked sequentially, form the current selected chain
Ledger State snapshots	Periodically store the ledger state at the tip of the ImmutableDB

Although this implementation represents just one of many possible data storage solutions, it is the one used by the reference cardano-node implementation and has become the de-facto standard for distributing the Cardano chain when the node is not involved. Consequently, services like Mithril sign this directory structure and its format.

It consists of 3 separate directories in which different data is stored:

db
├── immutable
│   ├── 00000.chunk
│   ├── 00000.primary
│   ├── 00000.secondary
│   └── ...
├── ledger
│   └── 164021355
└── volatile
    ├── blocks-0.dat
    └── ...

This diagram depicts where the blocks are distributed in such directories:

flowchart RL
    subgraph volatile;
      subgraph "current selection";
        Tip -->|...| Km1["(k-1)-th Block"]
        Km1 --> K["k-th Block"]
      end

      O1["Block"] --> O2["Block"]
      O3["Block"] --> K

      O4["Block"]
    end

    subgraph immutable;
     subgraph "Chunk 0";
        C0[" "]
        C1[" "] --> C0
        C2[" "] -->|...| C1
     end
     subgraph "Chunk 1";
        D0[" "]
        D1[" "] --> D0
        D2[" "] -->|...| D1
     end

     D0 --> C2
     subgraph "Chunk n";
     Kp1["(k+1)-th Block"] --> Kp2["(k+2)-th Block"]
     Kp2["(k+2)-th Block"] --> |...| E0[" "]
     end

     E0 -->|...| D2
    end
    K --> Kp1
    C0 --> Genesis
    subgraph ledger;
     S["Persisted snapshot"] --> Kp1
    end

`immutable`

Contains the historical chain of blocks.

TODO explain NestedCtxt, Header, Block, the Hard Fork Block, chunks, primary and secondary indices.

`volatile`

Contains blocks that form the current selection of the node (i.e. the best chain it knows about) and other blocks (both connected or disconnected from the selected chain) but whose slot number is greater than the k-th block in the current selection (so they might belong to a fork less than k blocks deep).

TODO describe the format in which the blocks are stored

`ledger`

Contains ledger state snapshots at immutable blocks. Ideally, this is the most recent immutable block, but since snapshots are taken periodically (due to their cost), it may be an older block. Snapshots are named after the slot number of the block when they were taken.

The snapshot is a file containing a CBOR-encoded Extended Ledger State. The Extended Ledger State is a combination of the Chain State (used by the protocol used in such era) and the Ledger State.

Coming soon: UTxO-HD

Mempool

In Cardano, for blocks to have useful data, they have to contain transactions, which are codifications of operations on the Ledger state that only some authorized actors can enact. Notice that such transactions are what makes up the contents of the block. In one way, one could see the ledger as being able to validate transactions, and by implication full blocks as those are mainly collections of transactions.

The Mempool is the abstract component of the node that stores transactions which are valid on top of the latest known Ledger State (the one at the tip of the chain selected by Chain Selection in the Consensus layer). The validity of transactions is defined by the Ledger layer. Notice a change in the selected chain should trigger a prompt revalidation of the pending transactions to discard those that became invalid on the new selection. Such collection of valid transactions will be requested by the Consensus layer to forge new blocks.

Once a transaction has been included in some block, it cannot be a pending transaction anymore, in fact from that point onwards it will be an invalid transaction, as in Cardano every transaction consumes at least one UTxO, in this case consumed by the instance of the transaction that was included in a block.

note

Notice that a transaction might be considered valid in one peer, but invalid in another peer which has selected a different chain. Even in the case both peers have the same selection, the existence in the mempool (or lack thereof) of some other specific transaction that creates (or consumes) the inputs for this one could lead to different veredicts on each peer.

While only nodes that forge new blocks will use these transactions to create blocks, non-block-producing nodes are still expected to diffuse transactions to their peers. All participating nodes must be able to receive, validate, and distribute transactions.

tip

When a node is shut down, the implementation can choose what to do with the pending transactions. The reference Haskell implementation discards them. However, it is conceivable for a node to store on disk the pending transactions from its mempool.

The TxSubmission2 miniprotocol is the one used to diffuse the pending transactions through the network.

To be able to diffuse transactions through TxSubmission2 and to fulfill the requirements of the Consensus layer, any mempool implementation has the following requirements:

Acquiring a snapshot of valid transactions for a newly forged block should be as fast as possible, as it will delay all other steps in the diffusion of such a block. This is required by the Consensus layer for forging blocks.
Cursor-like access to pending transactions. This is required by the TxSubmission2 protocol.
Re-validation on a new selected chain: it is useless to keep transactions that are no longer valid in the mempool. This is an inherent requirement of the mempool itself.

Notice that what a transaction actually is defined by the Ledger layer and it depends on the current era at the tip of the chain. The Ledger layer provides mechanisms to translate transactions from older eras to more recent ones. Once a transaction is translated to a more recent era, it is such translation the one that is distributed to the network afterwards.

note

In order to prevent overusing resources, the Ledger places a limit on the number of pending transactions it accepts on a single block, which should be enforced by the mempool. Once this limit is reached, requests to add new transactions will be rejected. This back-pressure mechanism is critically important in periods of intense traffic to preserve the overall throughput of the network.

This limit is not defined as a raw number of transactions but as a cap on various metrics, currently transaction size in bytes, execution units in CPU and Memory units.

Fairness

warning

TODO: describe fairness and what should mempools do in this regard.

`TxSubmission2`

warning

TxSubmission2 has traditionally lived in the Network layer, but it is intimately related to the Mempool. As such, we describe it here briefly, but the Networking documentation should have more details.

TxSubmission2 is the mini protocol in charge of diffusing pending transactions through the network. It is a pull-based miniprotocol: data is transmitted only upon explicit request from the client.

The goal of TxSubmission2 is to let other peers know about the transactions that the local node considers valid (with respects to the chain that the local node has selected in Chain Selection in the consensus layer), and transmit such transactions if requested.

Honest nodes will try to validate every transaction they come to know about.

There are some situations in which this miniprotocol would terminate abruptly, closing all the connections to the remote peer. Actions that are considered as misbehaviour are (not exclusively):

Violation of the miniprotocol state machine,
Too many or not enough transactions sent or acknowledged via the client or server,
Requesting zero transactions,
The client requesting a transaction that was not announced by the server.

warning

TODO: Make this list exhaustive

The state machine specification of TxSubmission2 and general mechanics are described in the Networking spec.

Codecs

The transactions sent through TxSubmission2 on the Cardano network are tagged with the index of the era they belong to. The serialization of the transaction proper is its CBOR representation.

serialize tx = <era tag><cbor of tx>

For transaction IDs, the same mechanism is used:

serialize txid = <era tag><cbor of txid>

Ledger

Cardano uses the Extended Unspent Transaction Output (EUTxO) ledger model…

data S

Ledger: Block Validation

Block validation is the process of applying a set of ledger rules to a candidate block before adding it to the blockchain and updating the state of the ledger. Each era has it’s own set of rules for block validation.

note

TODO: Write a full introduction here with relevant terminology and concepts defined.

While different node implementations may implement these rules in different ways, it’s vital that they all agree on the outcome of the validation process to prevent forks in the blockchain.

Conway Block Validation

In this section, we will walk through the cardano-ledger implementation of Conway era block validation. We will break up the validation process into smaller sections to make it easier to visualize and understand. All diagrams should be read from left to right and top to bottom in terms of order of execution.

The cardano-ledger has the concept of an EraRule, which is a set of validations that are applied to a block in a specific era. Often, a newer era may call a previous era’s EraRule instead of reimplementing the same logic.

EraRule BBODY

This is the “entrypoint” for block validation, responsible for validating the body of a block.

flowchart LR
    EBBC[EraRule BBODY Conway]
        EBBC --> CBBT[conwayBbodyTransition]
            CBBT --> totalScriptRefSize(totalScriptRefSize <= maxRefScriptSizePerBlock)
            CBBT --> S[(state)]

        EBBC --> ABBT[alonzoBbodyTransition]
            ABBT --> ELC[EraRule LEDGERS Conway]
            ABBT --> txTotalExUnits(txTotal <= ppMax ExUnits)
            ABBT --> BBodyState[(BbodyState @era ls')]

EraRule LEDGERS

This EraRule is responsible for validating and updating the ledger state, namely UTxO state, governance state, and certificate state.

flowchart LR
    ELC[EraRule LEDGERS Conway]
                    ELC --> ELS[EraRule LEDGERS Shelley]
                    ELS --> ledgersTransition
                        ledgersTransition --> |repeat| ledgerTransition
                            ledgerTransition --> |when mempool| EMC[EraRule Mempool Conway]
                                EMC --> mempoolTransition
                                    mempoolTransition --> unelectedCommitteeMembers(failOnNonEmpty unelectedCommitteeMembers)
                            ledgerTransition --> isValid{isValid}
                                isValid --> |True| ltDoBlock[do]
                                    ltDoBlock --> |currentTreasuryValueTxBodyL| submittedTreasuryValue(submittedTreasuryValue == actualTreasuryValue)
                                    ltDoBlock --> totalRefScriptSize(totalRefScriptSize <= maxRefScriptSizePerTx)
                                    ltDoBlock  --> nonExistentDelegations(failOnNonEmpty nonExistentDelegations)

                                    ltDoBlock --> ECSC[EraRule CERTS Conway]
                                    ltDoBlock --> EGC[EraRule GOV Conway]
                                    ltDoBlock --> utxoState[(utxoState', certStateAfterCerts)]
                                isValid --> |False| utxoStateCertState[(utxoState, certState)]
                            ledgerTransition --> EUC[EraRule UTXOW Conway]

EraRule CERTS

This EraRule is responsible for validating and updating the certificate state.

flowchart LR
    ECSC --> conwayCertsTransition
        conwayCertsTransition --> certificates{isEmpty certificates}
        certificates --> |True| cctDoBlock[do]
            cctDoBlock --> validateZeroRewards(validateZeroRewards)
            cctDoBlock --> certStateWithDrepExiryUpdated[(certStateWithDrepExiryUpdated)]

        certificates --> |False| sizeCheck{size > 1}
            sizeCheck --> |True| conwayCertsTransition
            sizeCheck --> |False| ECC[EraRule CERT Conway]
                ECC --> certTransition
                certTransition --> |ConwayTxCertDeleg| EDC[EraRule DELEG Conway]
                    EDC --> conwayDelegTransition
                        conwayDelegTransition --> |ConwayRegCert| crcDoBlock[do]
                            crcDoBlock --> crcCheckDepositAgaintPParams(checkDespoitAgainstPParams)
                            crcDoBlock --> crcCheckStakeKeyNotRegistered(checkStakeKeyNotRegistered)
                        conwayDelegTransition --> |ConwayUnregCert| cucDoBlock[do]
                            cucDoBlock --> checkInvalidRefund(checkInvalidRefund)
                            cucDoBlock --> mUMElem(isJust mUMElem)
                            cucDoBlock --> cucCheckStakeKeyHasZeroRewardBalance(checkStakeKeyHasZeroRewardBalance)
                        conwayDelegTransition --> |ConwayDelegCert| cdcDoBlock[do]
                            cdcDoBlock --> checkStakeKeyIsRegistered(checkStakeKeyIsRegistered)
                            cdcDoBlock --> checkStakeDelegateeRegistered(checkStakeDelegateeRegistered)
                        conwayDelegTransition --> |ConwayRegDelegCert| crdcDoBlock[do]
                            crdcDoBlock --> checkDepositAgainstPParams(checkDepositAgainstPParams)
                            crdcDoBlock --> checkStakeKeyNotRegistered(checkStakeKeyNotRegistered)
                            crdcDoBlock --> checkStakeKeyZeroRewardBalance(checkStakeKeyHasZeroRewardBalance)
                certTransition --> EPC[EraRule POOL Conway]
                    EPC --> EPS[EraRule POOL Shelley]
                    EPS --> poolDelegationTransition
                        poolDelegationTransition --> |regPool| rpDoBlock[do]
                            rpDoBlock --> actualNetId(actualNetId == suppliedNetId)
                            rpDoBlock --> pmHash(length pmHash <= sizeHash)
                            rpDoBlock --> ppCost(ppCost >= minPoolCost)
                            rpDoBlock --> ppId{ppId ∉ dom psStakePoolParams}
                                ppId --> |True| payPoolDeposit --> psDeposits[(psDeposits)]
                                ppId --> |False| psFutureStakePoolParams[(psFutureStakePoolParams, psRetiring)]
                        poolDelegationTransition --> |RetirePool| retirePoolDoBlock[do]
                            retirePoolDoBlock --> hk(hk ∈ dom psStakePoolParams)
                            retirePoolDoBlock --> cEpoch(cEpoch < e && e <= limitEpoch)
                            retirePoolDoBlock --> psRetiring[(psRetiring)]
                certTransition --> EGOVERTC[EraRule GOVERT Conway]
                    EGOVERTC --> conwayGovCertTransition
                    conwayGovCertTransition --> |ConwayRegDRep| crdrDoBlock[do]
                        crdrDoBlock --> notMemberCredVsDReps(Map.notMember cred vsDReps)
                        crdrDoBlock --> deposit(deposit == ppDRepDeposit)
                        crdrDoBlock --> crdrDRepState[(dRepState)]
                    conwayGovCertTransition --> |ConwayUnregDRep| curdrDoBlock[do]
                        curdrDoBlock --> mDRepState(isJust mDRepState)
                        curdrDoBlock --> drepRefundMismatch(failOnJust drepRefundMismatch)
                        curdrDoBlock --> curdrDRepState[(dRepState)]
                    conwayGovCertTransition -->|ConwayUpdateDRep| cudrDoBlock[do]
                        cudrDoBlock --> memberCredVsDreps(Map.member cred vsDReps)
                        cudrDoBlock --> cudrDRepState[(vsDReps)]
                    conwayGovCertTransition --> |ConwayResignCommitteeColdKey| crcckDoBlock[do]
                    conwayGovCertTransition --> |ConwayAuthCommitteeHotKey| cachkDoBlock[do]
                        crcckDoBlock --> checkAndOverwriteCommitteMemberState
                        cachkDoBlock --> checkAndOverwriteCommitteMemberState
                            checkAndOverwriteCommitteMemberState --> coldCredResigned(failOnJust coldCredResigned)
                            checkAndOverwriteCommitteMemberState --> isCurrentMember(isCurrentMember OR isPotentialFutureMember)
                            checkAndOverwriteCommitteMemberState --> vsCommitteeState[(vsCommitteeState)]

EraRule GOV

This EraRule is responsible for validating and updating the governance state.

flowchart LR
    EGC[EraRule GOV Conway]
    EGC --> govTransition
        govTransition --> badHardFork(failOnJust badHardFork)
        govTransition --> actionWellFormed(actionWellFormed)
        govTransition --> refundAddress(refundAddress)
        govTransition --> nonRegisteredAccounts(nonRegisteredAccounts)
        govTransition --> pProcDepost(pProcDeposit == expectedDeposit)
        govTransition --> pProcReturnAddr(pProcReturnAddr == expectedNetworkId)
        govTransition --> govAction{case pProcGovAction}
            govAction --> |TreasuryWithdrawals| twDoBlock[do]
                twDoBlock --> mismatchedAccounts(mismatchedAccounts)
                twDoBlock --> twCheckPolicy(checkPolicy)
            govAction --> |UpdateCommittee| ucDoBlock[do]
                ucDoBlock --> setNull(Set.null conflicting)
                ucDoBlock --> mapNull(Map.null invalidMembers)
            govAction --> |ParameterChange| pcDoBlock[do]
                pcDoBlock --> checkPolicy(checkPolicy)
        govTransition --> ancestryCheck(ancestryCheck)
        govTransition --> unknownVoters(failOnNonEmpty unknownVoters)
        govTransition --> unknwonGovActionIds(failOnNonEmpty unknownGovActionIds)
        govTransition --> checkBootstrapVotes(checkBootstrapVotes)
        govTransition --> checkVotesAreNotForExpiredActions(checkVotesAreNotForExpiredActions)
        govTransition --> checkVotersAreValid(checkVotersAreValid)
        govTransition --> updatedProposalStates[(updatedProposalStates)]

EraRule UTXOW

This EraRule is responsible for validating and updating the UTxO state.

flowchart LR
EUC[EraRule UTXOW Conway]
    EUC --> babbageUtxowTransition
        babbageUtxowTransition --> validateFailedBabbageScripts(validateFailedBabbageScripts)
        babbageUtxowTransition --> babbageMissingScripts(babbageMissingScripts)
        babbageUtxowTransition --> missingRequiredDatums(missingRequiredDatums)
        babbageUtxowTransition --> hasExactSetOfRedeemers(hasExactSetOfRedeemers)
        babbageUtxowTransition --> validateVerifiedWits(Shelley.validateVerifiedWits)
        babbageUtxowTransition --> validateNeededWitnesses(validateNeededWitnesses)
        babbageUtxowTransition --> validateMetdata(Shelley.validateMetadata)
        babbageUtxowTransition --> validateScriptsWellFormed(validateScriptsWellFormed)
        babbageUtxowTransition --> ppViewHashesMatch(ppViewHashesMatch)
        babbageUtxowTransition --> EUTXOC[EraRule UTXO Conway]
            EUTXOC --> utxoTransition
                utxoTransition --> disjointRefInputs(disjointRefInputs)
                utxoTransition --> validateOutsideValidityIntervalUtxo(Allegra.validateOutsideValidityIntervalUtxo)
                utxoTransition --> validateOutsideForecast(Alonzo.validateOutsideForecast)
                utxoTransition --> validateInputSetEmptyUTxO(Shelley.validateInputSetEmptyUTxO)
                utxoTransition --> feesOk(feesOk)
                utxoTransition --> validateBadInputsUTxO(Shelley.validateBadInputsUTxO)
                utxoTransition --> validateValueNotConservedUTxO(Shelley.validateValueNotConservedUTxO)
                utxoTransition --> validateOutputTooSmallUTxO(validateOutputTooSmallUTxO)
                utxoTransition --> validateOutputTooBigUTxO(Alonzo.validateOutputTooBigUTxO)
                utxoTransition --> validateOutputBootAddrAttrsTooBig(Shelley.validateOuputBootAddrAttrsTooBig)
                utxoTransition --> validateWrongNetwork(Shelley.validateWrongNetwork)
                utxoTransition --> validateWrongNetworkWithdrawal(Shelley.validateWrongNetworkWithdrawal)
                utxoTransition --> validateWrongNetworkInTxBody(Alonzo.validateWrongNetworkInTxBody)
                utxoTransition --> validateMaxTxSizeUTxO(Shelley.vallidateMaxTxSizeUTxO)
                utxoTransition --> validateExUnitsTooBigUTxO(Alonzo.validateExUnitsTooBigUTxO)
                utxoTransition --> EUTXOSC[EraRule UTXOS Conway]
                    EUTXOSC --> utxosTransition
                utxosTransition --> isValidTxL{isValidTxL}
                    isValidTxL --> |True| conwayEvalScriptsTxValid
                        conwayEvalScriptsTxValid --> expectScriptsToPass(expactScriptsToPass)
                        conwayEvalScriptsTxValid --> conwayEvalScriptsTxValidUtxosPrime[(utxos')]
                    isValidTxL --> |False| babbageEvalScriptsTxInvalid
                        babbageEvalScriptsTxInvalid --> evalPlutusScripts(evalPlutusScripts FAIL)
                        babbageEvalScriptsTxInvalid --> babbageEvalScriptsTxInvalidUtxosPrime([utxos'])
    EUC --> LedgerState[(LedgerState utxoState'' certStateAfterCERTS)]

Full Diagram

Here is the full diagram, with all EraRules combined.

flowchart LR
    EBBC[EraRule BBODY Conway]
        EBBC --> CBBT[conwayBbodyTransition]
            CBBT --> totalScriptRefSize(totalScriptRefSize <= maxRefScriptSizePerBlock)
            CBBT --> S[(state)]

        EBBC --> ABBT[alonzoBbodyTransition]
            ABBT --> ELC[EraRule LEDGERS Conway]
                ELC --> ELS[EraRule LEDGERS Shelley]
                ELS --> ledgersTransition
                    ledgersTransition --> |repeat| ledgerTransition
                        ledgerTransition --> |when mempool| EMC[EraRule Mempool Conway]
                            EMC --> mempoolTransition
                                mempoolTransition --> unelectedCommitteeMembers(failOnNonEmpty unelectedCommitteeMembers)
                        ledgerTransition --> isValid{isValid}
                            isValid --> |True| ltDoBlock[do]
                                ltDoBlock --> |currentTreasuryValueTxBodyL| submittedTreasuryValue(submittedTreasuryValue == actualTreasuryValue)
                                ltDoBlock --> totalRefScriptSize(totalRefScriptSize <= maxRefScriptSizePerTx)
                                ltDoBlock  --> nonExistentDelegations(failOnNonEmpty nonExistentDelegations)

                                ltDoBlock --> ECSC[EraRule CERTS Conway]
                                ECSC --> conwayCertsTransition
                                    conwayCertsTransition --> certificates{isEmpty certificates}

                                    certificates --> |True| cctDoBlock[do]
                                        cctDoBlock --> validateZeroRewards(validateZeroRewards)
                                        cctDoBlock --> certStateWithDrepExiryUpdated[(certStateWithDrepExiryUpdated)]

                                    certificates --> |False| sizeCheck{size > 1}
                                        sizeCheck --> |True| conwayCertsTransition
                                        sizeCheck --> |False| ECC[EraRule CERT Conway]
                                            ECC --> certTransition
                                            certTransition --> |ConwayTxCertDeleg| EDC[EraRule DELEG Conway]
                                                EDC --> conwayDelegTransition
                                                    conwayDelegTransition --> |ConwayRegCert| crcDoBlock[do]
                                                        crcDoBlock --> crcCheckDepositAgaintPParams(checkDespoitAgainstPParams)
                                                        crcDoBlock --> crcCheckStakeKeyNotRegistered(checkStakeKeyNotRegistered)
                                                    conwayDelegTransition --> |ConwayUnregCert| cucDoBlock[do]
                                                        cucDoBlock --> checkInvalidRefund(checkInvalidRefund)
                                                        cucDoBlock --> mUMElem(isJust mUMElem)
                                                        cucDoBlock --> cucCheckStakeKeyHasZeroRewardBalance(checkStakeKeyHasZeroRewardBalance)
                                                    conwayDelegTransition --> |ConwayDelegCert| cdcDoBlock[do]
                                                        cdcDoBlock --> checkStakeKeyIsRegistered(checkStakeKeyIsRegistered)
                                                        cdcDoBlock --> checkStakeDelegateeRegistered(checkStakeDelegateeRegistered)
                                                    conwayDelegTransition --> |ConwayRegDelegCert| crdcDoBlock[do]
                                                        crdcDoBlock --> checkDepositAgainstPParams(checkDepositAgainstPParams)
                                                        crdcDoBlock --> checkStakeKeyNotRegistered(checkStakeKeyNotRegistered)
                                                        crdcDoBlock --> checkStakeKeyZeroRewardBalance(checkStakeKeyHasZeroRewardBalance)
                                            certTransition --> EPC[EraRule POOL Conway]
                                                EPC --> EPS[EraRule POOL Shelley]
                                                EPS --> poolDelegationTransition
                                                    poolDelegationTransition --> |regPool| rpDoBlock[do]
                                                        rpDoBlock --> actualNetId(actualNetId == suppliedNetId)
                                                        rpDoBlock --> pmHash(length pmHash <= sizeHash)
                                                        rpDoBlock --> ppCost(ppCost >= minPoolCost)
                                                        rpDoBlock --> ppId{ppId ∉ dom psStakePoolParams}

                                                        ppId --> |True| payPoolDeposit --> psDeposits[(psDeposits)]
                                                        ppId --> |False| psFutureStakePoolParams[(psFutureStakePoolParams, psRetiring)]

                                                    poolDelegationTransition --> |RetirePool| retirePoolDoBlock[do]
                                                        retirePoolDoBlock --> hk(hk ∈ dom psStakePoolParams)
                                                        retirePoolDoBlock --> cEpoch(cEpoch < e && e <= limitEpoch)
                                                        retirePoolDoBlock --> psRetiring[(psRetiring)]

                                            certTransition --> EGOVERTC[EraRule GOVERT Conway]
                                                EGOVERTC --> conwayGovCertTransition
                                                conwayGovCertTransition --> |ConwayRegDRep| crdrDoBlock[do]
                                                    crdrDoBlock --> notMemberCredVsDReps(Map.notMember cred vsDReps)
                                                    crdrDoBlock --> deposit(deposit == ppDRepDeposit)
                                                    crdrDoBlock --> crdrDRepState[(dRepState)]
                                                conwayGovCertTransition --> |ConwayUnregDRep| curdrDoBlock[do]
                                                    curdrDoBlock --> mDRepState(isJust mDRepState)
                                                    curdrDoBlock --> drepRefundMismatch(failOnJust drepRefundMismatch)
                                                    curdrDoBlock --> curdrDRepState[(dRepState)]
                                                conwayGovCertTransition -->|ConwayUpdateDRep| cudrDoBlock[do]
                                                    cudrDoBlock --> memberCredVsDreps(Map.member cred vsDReps)
                                                    cudrDoBlock --> cudrDRepState[(vsDReps)]
                                                conwayGovCertTransition --> |ConwayResignCommitteeColdKey| crcckDoBlock[do]
                                                conwayGovCertTransition --> |ConwayAuthCommitteeHotKey| cachkDoBlock[do]
                                                    crcckDoBlock --> checkAndOverwriteCommitteMemberState
                                                    cachkDoBlock --> checkAndOverwriteCommitteMemberState
                                                        checkAndOverwriteCommitteMemberState --> coldCredResigned(failOnJust coldCredResigned)
                                                        checkAndOverwriteCommitteMemberState --> isCurrentMember(isCurrentMember OR isPotentialFutureMember)
                                                        checkAndOverwriteCommitteMemberState --> vsCommitteeState[(vsCommitteeState)]
                                ltDoBlock --> EGC[EraRule GOV Conway]
                                    EGC --> govTransition
                                        govTransition --> badHardFork(failOnJust badHardFork)
                                        govTransition --> actionWellFormed(actionWellFormed)
                                        govTransition --> refundAddress(refundAddress)
                                        govTransition --> nonRegisteredAccounts(nonRegisteredAccounts)
                                        govTransition --> pProcDepost(pProcDeposit == expectedDeposit)
                                        govTransition --> pProcReturnAddr(pProcReturnAddr == expectedNetworkId)
                                        govTransition --> govAction{case pProcGovAction}
                                            govAction --> |TreasuryWithdrawals| twDoBlock[do]
                                                twDoBlock --> mismatchedAccounts(mismatchedAccounts)
                                                twDoBlock --> twCheckPolicy(checkPolicy)
                                            govAction --> |UpdateCommittee| ucDoBlock[do]
                                                ucDoBlock --> setNull(Set.null conflicting)
                                                ucDoBlock --> mapNull(Map.null invalidMembers)
                                            govAction --> |ParameterChange| pcDoBlock[do]
                                                pcDoBlock --> checkPolicy(checkPolicy)
                                        govTransition --> ancestryCheck(ancestryCheck)
                                        govTransition --> unknownVoters(failOnNonEmpty unknownVoters)
                                        govTransition --> unknwonGovActionIds(failOnNonEmpty unknownGovActionIds)
                                        govTransition --> checkBootstrapVotes(checkBootstrapVotes)
                                        govTransition --> checkVotesAreNotForExpiredActions(checkVotesAreNotForExpiredActions)
                                        govTransition --> checkVotersAreValid(checkVotersAreValid)
                                        govTransition --> updatedProposalStates[(updatedProposalStates)]
                                ltDoBlock --> utxoState[(utxoState', certStateAfterCerts)]
                            isValid --> |False| utxoStateCertState[(utxoState, certState)]
                        ledgerTransition --> EUC[EraRule UTXOW Conway]
                            EUC --> babbageUtxowTransition
                                babbageUtxowTransition --> validateFailedBabbageScripts(validateFailedBabbageScripts)
                                babbageUtxowTransition --> babbageMissingScripts(babbageMissingScripts)
                                babbageUtxowTransition --> missingRequiredDatums(missingRequiredDatums)
                                babbageUtxowTransition --> hasExactSetOfRedeemers(hasExactSetOfRedeemers)
                                babbageUtxowTransition --> validateVerifiedWits(Shelley.validateVerifiedWits)
                                babbageUtxowTransition --> validateNeededWitnesses(validateNeededWitnesses)
                                babbageUtxowTransition --> validateMetdata(Shelley.validateMetadata)
                                babbageUtxowTransition --> validateScriptsWellFormed(validateScriptsWellFormed)
                                babbageUtxowTransition --> ppViewHashesMatch(ppViewHashesMatch)
                                babbageUtxowTransition --> EUTXOC[EraRule UTXO Conway]
                                    EUTXOC --> utxoTransition
                                        utxoTransition --> disjointRefInputs(disjointRefInputs)
                                        utxoTransition --> validateOutsideValidityIntervalUtxo(Allegra.validateOutsideValidityIntervalUtxo)
                                        utxoTransition --> validateOutsideForecast(Alonzo.validateOutsideForecast)
                                        utxoTransition --> validateInputSetEmptyUTxO(Shelley.validateInputSetEmptyUTxO)
                                        utxoTransition --> feesOk(feesOk)
                                        utxoTransition --> validateBadInputsUTxO(Shelley.validateBadInputsUTxO)
                                        utxoTransition --> validateValueNotConservedUTxO(Shelley.validateValueNotConservedUTxO)
                                        utxoTransition --> validateOutputTooSmallUTxO(validateOutputTooSmallUTxO)
                                        utxoTransition --> validateOutputTooBigUTxO(Alonzo.validateOutputTooBigUTxO)
                                        utxoTransition --> validateOutputBootAddrAttrsTooBig(Shelley.validateOuputBootAddrAttrsTooBig)
                                        utxoTransition --> validateWrongNetwork(Shelley.validateWrongNetwork)
                                        utxoTransition --> validateWrongNetworkWithdrawal(Shelley.validateWrongNetworkWithdrawal)
                                        utxoTransition --> validateWrongNetworkInTxBody(Alonzo.validateWrongNetworkInTxBody)
                                        utxoTransition --> validateMaxTxSizeUTxO(Shelley.vallidateMaxTxSizeUTxO)
                                        utxoTransition --> validateExUnitsTooBigUTxO(Alonzo.validateExUnitsTooBigUTxO)
                                        utxoTransition --> EUTXOSC[EraRule UTXOS Conway]
                                            EUTXOSC --> utxosTransition
                                                utxosTransition --> isValidTxL{isValidTxL}
                                                    isValidTxL --> |True| conwayEvalScriptsTxValid
                                                        conwayEvalScriptsTxValid --> expectScriptsToPass(expactScriptsToPass)
                                                        conwayEvalScriptsTxValid --> conwayEvalScriptsTxValidUtxosPrime[(utxos')]
                                                    isValidTxL --> |False| babbageEvalScriptsTxInvalid
                                                        babbageEvalScriptsTxInvalid --> evalPlutusScripts(evalPlutusScripts FAIL)
                                                        babbageEvalScriptsTxInvalid --> babbageEvalScriptsTxInvalidUtxosPrime([utxos'])
                            EUC --> LedgerState[(LedgerState utxoState'' certStateAfterCERTS)]
            ABBT --> txTotalExUnits(txTotal <= ppMax ExUnits)
            ABBT --> BBodyState[(BbodyState @era ls')]

Styleguide

note

Expand this as needed with examples, tips, glossary and general language used. Try not to replicate upstream docs, e.g. for mermaid or math markup.

Overall style

The blueprints should be well written. It might seem this should go without saying, but it’s put here to emphasise its importance. Poorly written documentation can confuse readers rather than clarify, and the more of it exists, the more damage it can cause.

Recognize that our goal is to make Cardano the most well-documented blockchain, rather than the most heavily-documented blockchain.

When writing you should always be keeping the reader in mind, thinking what is the best way to pass on the knowledge you have given the current state of knowledge they have. This is a hard thing to do, but it is worth aiming for!

People take in information in different ways. Some like literate prose, many like visual diagrams and pseudo-code, and a few like mathematical formulae. If you can do all of these, so much the better, but good diagrams are often the place to start.

As a general rule, approach it with the same level of care and thoughtfulness as you would when writing a peer-reviewed paper for a top-tier conference or journal that you are eager to publish. More specifically, here are a few general tips.

Do not reference a non-obvious term without defining it

Avoid explaining what X is by saying “X allows us to do Y” without defining Y. This only adds to the confusion: not only will the reader not understand X, but now Y as well. Also avoid circular definitions.

Instead of scattering definitions throughout the documentation, consider keeping them in one place by creating a glossary page.

Use consistent terminology

Sometimes people use different names for the same thing. Examples include “constitution script” vs. “guardrail script”, “Plutus version” vs. “Plutus ledger language”, “minting script” vs. “minting policy”.

Using consistent terminology is important for clarity. In some cases, using multiple terms interchangeably can be beneficial, but this should be clearly stated, such as in the glossary page. To further enhance consistency, consider creating a single, unified glossary page for the entire Cardano blueprint.

Many documents focus on describing how things are done in a particular system, but they often leave the readers wondering why certain design decisions were made over other design choices.

Just like a research paper, related work and alternative approaches should be discussed, especially in the following cases:

there is a comparable system that does things differently
a reasonable alternative approach exists, but it wasn’t chosen due to certain disadvantages
we previously adopted an alternative approach but later switched to the current one for specific reasons

In the last case, it is also important to document the time and/or the package version that switched to the new approach.

Machine checking for code snippets is strongly recommended

The benefits of doing so are obvious. It helps ensure the code is up to date, and allows users to access the full working code if they want to explore further.

Make individual documents self-contained, and avoid excessive linking

Excessive links in documentation can be irritating and distracting. Before linking to another page, consider whether the concept can be briefly explained in one or two sentences, so that the reader does not need to go to and read that page.

Ideally, a reader should be able to go through a page of documentation from beginning to end without interruptions. While links can sometimes be beneficial or even necessary, they can be placed in a “further reading” section at the end, rather than scattered throughout the text.

Back up your claims

When saying “this function boosts performance significantly”, you should generally provide some data to back it up. Again, treat it like a peer-reviewed paper - you know what would happen if you make such claims without evidence!

Use plenty of examples, but avoid trivial toy examples, and avoid many different examples

Examples are very important, and more is almost always better than fewer.

However, avoid trivial toy examples in general, as they are often not effective in explaining core concepts. Instead, use simple yet realistic examples. For instance, when introducing smart contracts, an auction example is far more illustrative than a trivial “hello world” example.

Whenever possible, use running examples (that is, examples that are used in many places, if not throughout the entire document). They are more illustrative and less distracting compared to introducing a new example in each instance. For example, using an auction example to explain on-chain code, a vesting example to explain off-chain code, and an escrow example to explain transaction balancing is less effective than using any of these examples consistently.

It can be challenging to think of good running examples, but it’s worth it. This is something you’d do if you are writing a peer reviewed paper you really want to publish, and you should strive to achieve the same standard. It is worth coordinating on examples across teams, and strive for consistent examples in documentation written by different teams.

Maintain changelog and versioning

It is sometimes useful to see how certain things were done in the past, and how they have evolved over time. Thus documentation should be versioned and released like software packages, with old versions accessible and changelogs available.

Avoid multiple sources of truth

Having multiple sources of truth makes documentation difficult to maintain. To mitigate this, generate documentation from the source of truth whenever possible - for example, deriving compiler flag documentation from code instead of writing it manually.

Add a summary or TLDR for long pages

This can save readers time and lets them decide whether to dive into the details, or if the summary/TLDR is sufficient.

Other tips

Create one file per major topic
Divide text up into logical, hierarchical sections
Break up long paragraphs into bulleted lists
Use code blocks to give short examples
Use backquote to set off identifiers - e.g. message or state names

Language

Text should be written in American English, at the reading level of a competent software developer - which is often very high, but bear in mind that English may not be their first language. Use technical language by all means, but there is no need to be egregiously erudite in your elucidation.

It may not be necessary to talk about humans in much of this documentation, but if you do, please use gender-neutral pronouns - ‘they’ and ‘their’:

A user must keep the keys to their wallet safe.

If you find this difficult, cast it into the plural:

Users must keep the keys to their wallets safe.

Normative vs declarative style

Some standards documentation is very SHOUTY about MUST, SHALL and so on. Even if it doesn’t SHOUT it can still be rather clumsy to read. Instead, we want to use a declarative style. We can express what an implementation has to do conform to the specification as a simple descriptive fact - e.g.

The tester discards widgets with broken flibbits.

rather than the ‘normative’

The tester SHALL discard widgets with broken flibbits.

So in this style there is an implicit ‘must’. If something is optional, this can be said explicitly:

The package may use extra sproggles if required.

It’s still OK to use ‘must’ to emphasise cases which are absolutely critical, but still, please don’t SHOUT.

Diagrams & Maths

We can use mermaid diagrams (live editor):

graph LR;
    A-->B-->D;
    A-->C;

and maths using katex:

$U = (Φ \times O)^{*}$

Alerts

warning

We can use the github flavored callouts, documented here

note

A friendly note in github. How about code blocks?

cargo install mdbook-alerts

Footnotes

Additional information that would complicate the read-flow can be put into footnotes ¹.

Example footnote

Other stuff

The footnote should appear below. If not, we need to contribute this to mdbook.

Contributing to Cardano Blueprint

Thank you for considering contributing and sharing your knowledge about the Cardano protocol in one capacity or another.

There are basically two ways to contribute:

Validate and provide feedback about existing blueprints
Create or update blueprints

Validate blueprints

As you might have seen, the goal of cardano-blueprint is to create accessible assets that are understandable and useful to wide variety of builders from the Cardano community.

Ask questions

A great way to make sure we cover relevant areas of the Cardano protocol first, is to understand what you are trying to build or understand right now.

We use Q&A discussions to gather immediate questions and use them to identify gaps for where blueprints can help most.

While discussions and providing answers in the Q&A forum are obviously welcome, ideally we would be writing blueprints and be able to answer by pointing future readers there.

If you happen to find the information provided through blueprints useful to your current project, or if you are confused about the why of blueprints, the general category is the right place for that.

Or maybe you have an idea on how we could reach our goals by changing one or the other process? Ideas are collected in the ideas category.

We would love to hear about any feedback! 💙

Report issues

Submit an issue if you spot an errors, or should you encounter a problem with using or contributing to the blueprints themselves.

Write blueprints

Creating or updating blueprints is the bread & butter of this project and anyone is invited to do just that!

The bandwidth of possible contributions is very broad and can range from fixing typos via the Suggest an edit button on the rendered website, over contributing a whole set of documents about how consensus works in Cardano, to including a conformance test suite.

In fact, it’s not even set in stone what a blueprint is and we are gladly exploring any asset that you think covers our values and contributes to reaching the goal of the Cardano Blueprint initiative.

Creating a pull request

Thank you for contributing your changes by opening a pull request!

We do appreciate it if your pull request meets the following criteria:

Description of the changes: providing a clear summary of the changes is beneficial
Quality of changes: ensure that new or updated automated tests validate your changes
Scope: we prefer multiple pull requests that are well-scoped rather than a single large one
Correctness: the pull request should pass the continuous integration (CI) pipeline.

The project is currently in an expand-and-gather phase with no requirement of approvals, but consider requesting for reviews on Github-suggested reviewers.

Anyone is free to fork the repository and we consider pull requests from forks.

The project is currently situated in the cardano-scaling organization, to which we happily invite you and give write permissions onto this repository - so you can pull request from branches directly.

Review contributions

The Cardano Blueprint aims to be a community effort and we would love to have you become a regular visitor and maybe even reviewer of new contributions! 🤝

Publishing

Any changes to the main branch are automatically published to the live version of the blueprints hosted via Github Pages. If you have made changes and they are not propagated to the live version, please check the build status on GitHub Actions, or make sure you fully reload the webpage (Ctrl+Shift+R on firefox).

Versioning

The Cardano Blueprint is currently not versioned, tagged or released.

We are prepared to put such process in place as we see demand from implementations relying on a specific version of test data, interface descriptions or similar contained in cardano-blueprint.

Communication

Besides the many ways to engage through issues and discussions, we also use text and video channels on the Cardano Scaling Discord server for informal chats.

There is a weekly office hours event to which anyone is welcome and we are looking forward to see you there!

Last but not least and probably the best way to contribute: Share the love for blueprints! 📘📐💙