Why do we need to ‘ZKify’ the consensus layer?

Abstract

Since the competition among numerous public chains, to Danksharding in the Ethereum roadmap, and to layer 2 solutions such as op/zk, we have been continuously discussing the scalability of blockchain – what to do with a large number of users and funds coming in? Through a series of articles to follow, I would like to show you a future scenario consisting of three parts: trustless data access, off-chain computation, and on-chain verification.

Trustless Data Access + Off-chain Computation + On-chain Verification

“Proof of Consensus” is an important part of this blueprint. This article discusses the significance of using zero-knowledge proof of consensus on top of Ethereum’s PoS, including:

• Understanding how different parties view the Grayscale victory Predicting the SEC’s follow-up actions
• What are the current BTC spot ETF applications? What are the approval timelines and deadlines?
• zkSync earned over 20 million US dollars in half a year. How does Layer2 actually make a profit?

1. The importance of decentralization for EVM.

2. The importance of decentralized data access for web3 scalability.

Proving full consensus on the Ethereum mainnet is a complex task, but if we can achieve zkization of the consensus layer, it will help Ethereum scale while ensuring security and trust, enhance the robustness of the entire Ethereum ecosystem, reduce participation costs, and allow more people to participate.

I. Why does proving consensus matter?

  1. Perspective from Ethereum

  2. Perspective of Protocol Stacks on Ethereum 

II. Where is Blockchain Data? Trust Assumptions for Different Data Sources

III. The LianGuaith to Prove Consensus Using ZK 

  1. Key Steps in Consensus Formation in Ethereum 2.0 

  2. Tech Stacks to Prove Consensus

  3. The End Game: Diversified Level 1 zkEVM 

IV. What is the Future?

V. Reference

I. Why does proving consensus matter?

Using zk to verify the consensus layer of Ethereum L1 makes sense in two main directions. Firstly, it can compensate for the diversity of current nodes, enhancing the decentralization and security of Ethereum itself. Secondly, it provides the basis for availability and security for various layers of protocols in the Ethereum ecosystem, including cross-chain security, trustless data access, decentralized oracles, and scalability for more users.

1. Perspective from Ethereum

For Ethereum to achieve its decentralization and robustness, it needs an environment of client diversity. This means more people participating, especially ordinary users, running clients based on different code environments. However, requiring every user to run a full node is unrealistic, as it requires a large amount of resources that few people can afford, such as at least 16 GB+ RAM and Fast SSD with 2+TB, and these requirements are still growing.

The current goal is to achieve a light node that can provide the same level of trust as a full node (minimizing trust) while having lower costs in terms of memory, storage, and bandwidth requirements. However, currently, light nodes do not participate in the consensus process, or in other words, they are only protected by a partial consensus mechanism (Sync Committee).

This goal is referred to as “The Verge” in Ethereum’s roadmap.

“The Verge” aims to bridge the client gap, with a key step being how to achieve a trustless light node with the same level of security as today’s full nodes, filling the client gap and allowing more people to actively participate in the network’s decentralization and robustness.

https://www.ethernodes.org/network-types

https://clientdiversity.org/

2. Perspective of Protocol Stacks on Ethereum

From first principles, we need to solve the combination problem of on-chain data access and off-chain computation verification.

Currently, the use of on-chain data is relatively primitive and insufficient. In many cases, the data required for protocol adjustments is too complex to be computed on-chain, and the cost of obtaining the data in a trustless manner is too high, requiring extensive historical data access and frequent digital computations.

For individual users and projects, our ideal scenario is to achieve decentralized, end-to-end, trustless data transmission and read/write. Based on this, we should aim to minimize computation costs while considering security, usability, and cost-effectiveness for future users.

Specifically, the following aspects are included:

1. Decentralized and trustless oracles: Currently, centralized oracles are used in protocols to avoid direct access to a large amount of historical data on-chain, which increases unnecessary trust costs and reduces composability.

2. Data read/write for data-sensitive protocols and assets: For example, DeFi protocols require some dynamic parameter adjustments during operation. However, whether it is possible to access historical data and perform more complex computations without trust, such as adjusting AMM fees based on recent market fluctuations, designing on-chain derivative trading price models and dynamic fluctuations, introducing machine learning methods for asset management, adjusting lending interest rates based on market conditions, etc.

3. Cross-chain security: Currently, light node solutions based on zk technology are superior in terms of security, capital efficiency, statefulness, and diversity of information transmission. The current Succinct TeleLianGuaithy cross-chain solution and the Polehedra cross-chain solution on LayerZero are both based on Sync Committee for light node block header zk verification. However, the Sync Committee is not the Ethereum PoS consensus layer itself and has certain trust assumptions, leaving room for further improvements in the future.

Currently, due to considerations such as economic costs, technical limitations, and user experience, developers typically rely on centralized RPC servers, such as Alchemy, Infura, and Ankr, when using on-chain data.

2. Where is Blockchain Data? Trust Assumptions for Different Data Sources

There are two sources of computational data in blockchain: on-chain data and off-chain data. Computation is performed based on these two sources. For example, the previous article mentioned the need to adjust DeFi protocol parameters.

Data Access, computation, proof and verification

There are two notable characteristics of reading, writing, and computing on-chain and off-chain data:

1. In order to achieve decentralization and security, it is best to verify the data we obtain, that is, “Don’t Trust, Verify”.

2. Often involves complex and expensive computational processes.

If suitable technical solutions are not found, the above two points will affect the availability of blockchain.

We can illustrate different ways of obtaining data through a simple example. Suppose you want to check your account balance, what would you do?

One of the safest ways is to run a full node yourself, check the Ethereum state stored locally, and obtain the account balance from it.

Full Node Benchmark. Sync mode and client selection affect the required storage space.

However, running a full node yourself is costly and requires maintenance. To save trouble, many people may directly request data from centralized node operators. Although there is nothing wrong with doing so, similar to operations in Web2, and we have never seen these providers engage in any malicious behavior, it also means that we have to trust a centralized service provider, which increases the overall security assumption.

To solve this problem, we can consider two solutions: first, reduce the cost of running nodes; second, find a way to verify the credibility of third-party data.

It is better to only store necessary data. In order to access data more efficiently, reduce the cost of trust, and independently verify data, some institutions have developed light clients, such as Rust-based Helio (developed by a16z), Lodestar, Nimbus, and JavaScript-based Kevlar, etc. Light clients do not store all block data, but only download and store block headers, which are “summaries” of all information in a block. Light clients are able to independently verify the received data, so when you obtain data from a third-party data provider, you no longer need to fully trust the data provided by that provider.

https://medium.com/coinmonks/ethereum-data-transaction-trie-simplified-795483ff3929

The main characteristics of light clients include:

• Ideally, light clients can run on mobile phones or embedded devices.

• Ideally, they can have the same functionality and security guarantees as full nodes.

• However, light clients do not participate in the consensus process, or in other words, they are only protected by a partial consensus mechanism, namely the Sync Committee.

Sync Committee is a trust assumption for light nodes.

Before The Merge, starting from December 2020, the Beacon Chain underwent a hard fork called Altair, with the core purpose of providing consensus support for light nodes. Unlike full PoS consensus, this group of validators (512 in total) is composed of a smaller dataset, selected at longer intervals (256 epochs, approximately 27 hours) through random sampling.

This is why we need to validate the entire consensus layer of Ethereum, in order to usher in a future that is more secure, with stronger usability, more diverse protocols, and widespread adoption. Currently, the best solution appears to be zero-knowledge (zero-knowledge) technology.

III. The LianGuaith to Prove Consensus Using ZK

In order to build an environment without trust assumptions, it is necessary to address the issues of light node credibility, decentralized data access, and off-chain computation verification. In these aspects, zero-knowledge proofs are currently the most recognized core technology, which involves but is not limited to underlying solutions such as zkEVM, zkWASM, other zkVM, zk Co-processor, etc.

Proving the consensus layer is an important part of this.

The PoS algorithm is very complex, and implementing them in a ZK manner requires a lot of engineering work and architectural considerations. Let’s break down its components first.

1. Key Steps in Consensus Formation in Ethereum 2.0

(1) Validator-related algorithms

These include the following steps:

• Becoming a validator: Validator candidates need to send 32 ETH to the deposit contract and wait for at least 16 hours to several days or weeks for the Beacon Chain to process and activate them as formal validators. (Refer to FAQ – Why does it take so long for a validator to be activated)

• Exercising validator duties: Involves random number and block proof algorithms.

• Exiting the validator role: The validator can voluntarily exit or be penalized (slashed) for misconduct. Validators can initiate an “exit” at any time, but there is a limit to the number of validators that can exit in each epoch. If there are too many validators attempting to exit at the same time, they will be put in a queue and still need to fulfill their validator duties until their turn comes. After a successful exit, validators will be able to withdraw their staked funds after 1/8 of an eek.

(2) Random number-related algorithms

• Each epoch consists of 32 slots. Two epochs in advance, all validators are randomly divided into 32 committees, which are responsible for consensus on each block in the current epoch.

• Each committee has two roles: a proposer and builders, randomly selected. This separates the processes of transaction ordering and block construction (see proposer/builder seLianGuairation – PBS) .

(3) Block attestation and BLS signature-related algorithms

• The signature part is the core part of the consensus layer.

• The validation committee of each slot votes (using BLS signatures) and needs to achieve a 2/3 approval rate to build a block.

• In the Ethereum PoS consensus layer, BLS signatures use the BLS12-381 elliptic curve, which is LianGuaiiring-friendly and suitable for aggregating all signatures, reducing proof time and size.

• In proof of work, blocks may undergo reorganization (re-org). After the merge, the concept of “finalized blocks and safe head” is introduced on the execution layer. To create a conflicting block, an attacker needs to destroy at least 1/3 of the total staked Ether. In many ways, PoS is more reliable than PoW.

https://blog.ethereum.org/2021/11/29/how-the-merge-imLianGuaicts-app-layer

At the end of June 2023, “Puzzle Ventures Evening Study” introduced the zkPoS of Hyper Oracle (using zk methods to verify the Ethereum full consensus layer). For details, please see zkPoS: End-to-End Trustless

(4) Others: Weak Subjectivity Checkpoints

One of the challenges faced by trustless PoS consensus proofs is the selection of weak subjectivity checkpoints, which involves social consensus based on social information. These checkpoints are revert limits because blocks before weak subjectivity checkpoints cannot be changed. See: https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/weak-subjectivity/

Checkpoints are also a consideration in the zkification of the consensus layer.

2. Tech Stacks to Prove Consensus

In the proof consensus layer, proof signatures or other calculations themselves are very expensive, but compared to that, verifying zero-knowledge proofs is very cheap.

When choosing the method of using zero-knowledge proofs in the consensus layer, the protocol needs to consider the following factors:

• What do you want to prove?

• What are the application scenarios after the proof?

• How to improve the efficiency of the proof?

Taking Hyper Oracle as an example, for proving BLS signatures, Halo2 was chosen. They chose Halo2 instead of Circom used by Succinct Labs for the following reasons:

• Both Circom and Halo2 can generate zero-knowledge proofs for BLS signatures (BLS12-381 elliptic curve).

• Hyper Oracle is not only doing zkPoS, its core product is a programmable on-chain zkOracle. Directly facing users are zkGraph, zkIndexing, and zkAutomation. It also uses the zkWASM virtual machine to verify off-chain computations. Although Circom is easier for engineers to use, its compatibility is poor and cannot ensure that all functionalities can be used.

• Circom-LianGuaiiring is compiled into R1CS, which is incompatible with zkWASM and other circuit’s Plonkish constraint systems, while Halo2 LianGuaiiring circuits can be easily integrated into zkWASM circuits. In comparison, R1CS is not ideal for proof batching.

• From an efficiency perspective, the BLS circuit generated by Halo2-LianGuaiiring is smaller, has shorter proof time, lower hardware requirements, and lower gas fees.

https://mirror.xyz/hyperoracleblog.eth/lAE9erAz5eIlQZ346PG6tfh7Q6xy59bmA_kFNr-l6dE

Another key point in using zero-knowledge to prove the consensus layer is recursive proof, which is proof of proofs, bundling previous events into one proof.

Without recursive proof, the proof size will eventually output O(block height), which includes each block attestation and its corresponding zkp. Through recursive proof, apart from the initial and final states, we only need a proof of size O(1) for any number of blocks.

Verify Proof N and Step N+1 to get Proof N+1, i.e. you know N+1 pieces of knowledge, instead of verifying all N Steps separately.

Going back to the initial goal, our solution should target “light clients” with computational and memory constraints. Even if each proof can be verified in a fixed time, if the number of blocks and proofs accumulates, the verification time will become very long.

3. The End Game: Diversified Level 1 zkEVM

Ethereum’s goal is not only to prove the consensus layer but also to achieve zero-knowledgeization of the entire Layer 1 virtual machine through zkEVM and ultimately realize diversified zkEVM to enhance Ethereum’s decentralization and robustness.

For these issues, Ethereum’s current solution and roadmap are as follows:

“Lightweight” – smaller memory, storage, and bandwidth requirements

• Currently achieved by light nodes storing and verifying only block headers.

• Future developments require further efforts in verkle tree and stateless clients, involving improvements to the mainnet data structure.

“Trustless” – achieving minimal trust equivalent to full nodes

• Basic light node consensus layer has been implemented, namely Sync Committees, but this is only a transitional solution.

• Use SNARK to verify Ethereum Layer 1, including verifying Verkle Proof of the execution layer, verifying the consensus layer, and SNARKizing the entire virtual machine.

• Level 1 zkEVM is used to achieve zero-knowledgeization of the entire Ethereum Layer 1 virtual machine and to implement diversified zkEVM.

Potential Risks In an ideal scenario, when entering the zk era, we need multiple open-source zkEVMs – different clients have different zkEVM implementations, and each client waits for a proof compatible with its own implementation before accepting a block.

However, having multiple proof systems may pose some problems because each proof system requires a peer-to-peer network, and a client that only supports a certain proof system can only wait for the corresponding type of proof to be recognized by its verifier. Two main challenges that may arise include “latency challenge” and “data inefficiency”. The former is mainly due to slow proof generation, creating a time gap for malicious actors to create temporary forks when generating proofs for different proof systems. The latter is because you need to generate multiple types of zk proofs, which requires saving the original signatures. Although theoretically the advantage of zkSNARK itself is the ability to delete original signatures and other data, there are contradictions that need to be optimized and resolved.

4. Future Outlook

In order to bring more users to web3, provide smoother experiences, create higher usability, and ensure application security, we must build the infrastructure for decentralized data access, off-chain computing, and on-chain verification.

The proof of consensus layer is one important component. In addition to Ethereum PSE and the aforementioned zkEVM layer2, there are some protocols that are using zero-knowledge proof consensus to achieve their application goals, including the Hyper Oracle (Programmable zkOracle Network) which plans to use the entire consensus layer of Ethereum PoS for data retrieval; Succinct Labs’ TeleLianGuaithy, which is a light node bridge that achieves cross-chain communication through verifying the Sync Committee consensus and submitting state validity proofs; Polyhedra, which was originally a light node bridge but now claims to use devirgo to achieve full node full consensus with zk proof.

In addition to cross-chain security and decentralized oracles, this combination of off-chain computing and on-chain verification may also be involved in fraud proofs in optimistic rollup, integrating with OP L2; or in intent-based architecture, providing on-chain proofs for more complex intent structures, and so on.

Here we are discussing not only the off-chain ecosystem surrounding Ethereum but also the broader market beyond Ethereum.

There are still many aspects of this topic worth further research, such as the article published by a16z on August 24th last week, which argues that “stateless blockchains cannot be achieved”, weak subjectivity checkpoints, whether the security of the Sync Committee is sufficient mathematically, and other issues. Interested peers are welcome to contact the author (zoey@puzzle.ventures) to continue the discussion on this topic.

Once again, thank you to colleagues for their guidance and feedback, Alex @ IOBC (@looksrare_eth), Fan Zhang @ Yale University (@0xFanZhang), Roy @ Aki Protocol (@aki_protocol), Zhixiong LianGuain @ LianGuai (@nake13), Suning Yao @ Hyper Oracle (@msfew_eth), Qi Zhou @ EthStorage (@qc_qizhou), Sinka @ Delphinus (@DelphinusLab), Shumo @ Manta (@shumochu)

Reference

Annotated Ethereum Roadmaphttps://notes.ethereum.org/@domothy/roadmap#Annotated-Ethereum-Roadmap

Altair Hard Fork – The Beacon Chain

https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/fork.md

How will Ethereum’s multi-client philosophy interact with ZK-EVMs?, Vitalik Buterin

https://vitalik.eth.limo/general/2023/03/31/zkmulticlient.html

State of research: increasing censorship resistance of transactions under proposer/builder seLianGuairation (PBS), Francesco (Ethereum foundation)

https://notes.ethereum.org/s3JToeApTx6CKLJt8AbhFQ

How The Merge ImLianGuaicts Ethereum’s Application Layer, by Tim Beiko

https://blog.ethereum.org/2021/11/29/how-the-merge-imLianGuaicts-app-layer

Ethereum Developer Docs – Nodes and Clients, Ethereum Foundation

https://ethereum.org/en/developers/docs/nodes-and-clients/light-clients/

Building Helios: Fully trustless access to Ethereum, a16z

https://a16zcrypto.com/posts/article/building-helios-ethereum-light-client/

How I Learned to Stop Worrying and Love the Sync Committee, Uma Roy, Succinct Labs

https://blog.succinct.xyz/blog/sync-committee

zkPoS: End-to-End Trustless, msfew & Shuyang, Hyper Oracle

https://mirror.xyz/hyperoracleblog.eth/lAE9erAz5eIlQZ346PG6tfh7Q6xy59bmA_kFNr-l6dE

Proof of Consensus for Ethereum, Succinct Labs

https://github.com/succinctlabs/eth-proof-of-consensus

zkLightClient on LayerZero, Polyhedra

https://docs.zkbridge.com/zklightclient-overview/zklightclient-on-layerzero

Intent-Based Architectures and Their Risks, Quintus Kilbourn, Georgios Konstantopoulos, LianGuairadigm

https://www.paradigm.xyz/2023/06/intents#conclusion

RFP: OP Stack Zero Knowledge Proof, Optimism

https://github.com/ethereum-optimism/ecosystem-contributions/issues/61

We will continue to update Gambling Chain; if you have any questions or suggestions, please contact us!

Follow us on Twitter, Facebook, YouTube, and TikTok.