Author: Colby SerBlocking
Translation: DAOrayaki
Nostr 2.0 may be able to establish as Layer 2 on top of Bitcoin, providing secure off-chain data storage, just as the Lightning Network provides instant off-chain payments as Layer 2.
This article will explain how Nostr relays synchronize their data while maintaining their lightweight nature, allowing users to selectively delete data, which is not possible with Layer 1 blockchains. Additionally, because the capacity and speed of Bitcoin blocks are limited, using Nostr relays to store data may be cheaper than storing large amounts of data on the Bitcoin blockchain.
- Uniswap v4 unveils itself, what are the new features that spark imagination?
- LayerZero OFT: A Solution for “Unified Interoperability” Among Public Chains
- Osmosis welcomes a major update, what is Supercharged Liquidity?
The following simple computer science design improves the distributed properties of Nostr networks under the standardized CAP theorem criteria. CAP stands for Consistency, Availability, and Partition tolerance.
Nostr relays do not know when a configuration file is incomplete, and relays lack consistency (C in the CAP theorem)
Relays lack consistency (C in the CAP theorem)
Consistency means that the synchronized databases on various computers are the same. Nostr relays cannot perform trust-minimized synchronization in the same way as blockchain by synchronizing their data block by block. Unlike a Bitcoin full node, the database stored by a Nostr relay is usually incomplete. Except for blindly requesting all posts signed by a specific user, Nostr relays cannot discover missing data.
Consistency/synchronization issues with Nostr:
If two users upload their posts to different Nostr relays, these two users may not be able to see each other’s posts, as Nostr is not like a blockchain. In a blockchain, every time there is a new record, all full nodes will synchronize the blockchain. All full nodes will add these data as a block to their blockchain at the same time. Each full node on the Bitcoin blockchain has exactly the same blockchain.
If we want Nostr users to always be able to see each other’s posts, then all Nostr relays need a way to identify the missing data in the user’s configuration file so that they can request the missing part from other Nostr relays or users.
Synchronizing Nostr relays using weekly on-chain Merkle root and entire tree hash
- Approximately once a week, a user can organize all of their posts into a Merkle tree.
- Each leaf in the Merkle tree contains a hash of a post, just like each leaf in Bitcoin contains a hash of a transaction.
- Once a user organizes their entire profile into a Merkle tree, they will publish the Merkle root in an OP_RETURN on-chain, below a regular Bitcoin transaction. This is why Nostr 2.0 does not require a hard fork of the blockchain to work. OP_RETURN is a segment below a Bitcoin transaction that allows small comment information to be attached before the sender signs.
- Additionally, the user will hash the entire tree and upload it (along with the Merkle root in the OP_RETURN) to the chain. The Merkle root is just the hash of the top branch, not the entire tree. The entire tree hash is crucial for users and relays to detect if profile data is missing.
- To get the entire tree hash, put the Merkle root at the top of a text file. Then put the Merkle branches on the lines below the root. Then put the Merkle leaves on the lines below the branches. Once the tree is arranged as described, hash it all at once. Here’s an example of the entire tree hash, which is what the entire tree hash for the Merkle tree shown above looks like. Entire Tree Hash (hashes all the Merkle tree data at once)
The Merkle root and entire tree hash provide two key functions:
- The Merkle root allows users and relays to download a part of a profile in one go, just like they can download a transaction without having to download an entire block.
- The entire tree hash lets users and relays know if the profile they’re storing is incomplete. Unlike the Merkle root, only when all the data bits are in the Merkle tree does the entire tree hash match.
This cheap method can be used to update the entire profile every week or at a frequency a user customizes. Nostr can still work as it does now, but if a user wants all users to see their posts, they can occasionally pay some sats to sync their data between Nostr relays.
Users and relays can download a branch of posts at a time. After each branch, they hash the branch with another branch closest to the Merkle root to check if it matches the Merkle root on-chain (similar to SPV). If these branches hash together to match the Merkle root, then they know that the branch is part of the user profile even if they do not have the complete user profile yet. Users can download different branches of the same configuration file from many different Nostr relays while verifying the validity of each branch and ensuring that the downloaded configuration file is complete.
Downloading branches one by one can prevent delay attacks which can cripple many distributed networks, which is why Merkle root and branches are used to protect SPV light wallets in the Bitcoin whitepaper.
Why can’t the Merkle root function as the entire tree hash?
If Nostr relays only relied on the Merkle root, then they would not know when the Merkle tree is complete because each pair of branches closest to the Merkle root would hash to the same Merkle root.
To ensure that the user’s profile is complete, a relay or user will hash their updated entire Merkle tree and verify if it matches the entire tree hash on-chain. If the entire tree hash matches, then the user’s data is complete. If the entire tree hash does not match, then the relay or user can tell other relays their latest leaf number and request for missing branches until the entire tree hash matches. To keep track of all new Merkle roots added every week or so, Nostr relays must be a Bitcoin full node. Nostr 2.0 relays store the Bitcoin blockchain indirectly by being rewarded to enhance the security of both Bitcoin and Nostr.
Nostr storage limitations: User’s rule of thumb
Due to relays having the right to choose what they want to store, unlike a Bitcoin full node, Nostr relays may lose some user data. Therefore, users should only store data on Nostr relays if they can backup locally. Web5’s self-hosted service allows users to sync backups to all local devices, which reduces the risk for users who are concerned about using Nostr. Ultimately, only the blockchain is where data is truly immutable. Even so, Nostr is a fairly secure hybrid solution that still works well for many applications. The trade-offs are listed below:
Three-tier trust minimization
- Layer 1: Immutable and expensive data storage that is difficult to censor. (On-chain block syncs all Bitcoin full nodes.)
- Layer 2: Mutable and inexpensive data storage that is moderately difficult to censor. (Off-chain Merkle tree and on-chain hash, sync Nostr relays as needed.)
- Local data storage is synced to all local devices and is easy to censor. (Local centralization)
The fundamental trade-off between Nakamoto consensus blockchain and Nostr
The more Nostr relays that store specific address data, the more difficult it is to censor that data. This means that popular data hosted by many Nostr relays may be more difficult to censor than unpopular data that is downloaded infrequently.
On the other hand, the Nakamoto consensus blockchain can prevent censorship based on data age. The longer the data exists in the blockchain, the more difficult it is to delete the data with a 51% attack.
Using ZKCSP with searchable proofs-of-retrievability and Lightning Network payments to Nostr relays
Since we can verify that certain branches belong to a specific user, we can pay Nostr relays each time they hand over a small piece of data to users. To achieve this, users need to download the header of the blockchain (just like in SPV) so that they can perform typical light wallet functions. Users will use the SPV function of the light wallet to retrieve a specific transaction from the chain, which will include the Merkle root of the user’s configuration file and the entire tree hash in its OP_RETURN. Now, users can pay relays to download the user’s configuration file branch by branch and verify each branch by hashing it to match the Merkle root on the chain.
To send sats (the smallest unit of Bitcoin) to Nostr relays in exchange for providing data, we use an evolved version of Gregory Maxwell’s (well-known Bitcoin Core developer) ZKCP design (Zero-Knowledge Conditional Payment) [1], ZKCSP: Searchable Proof with Lightning Network.
According to the description in the ZKCSP white paper:
“…no third parties that need to be trusted…We also have implemented the ZKCSP protocol for the case of searchable proofs, where a client pays a server to provide a proof that the client’s data is correctly stored on the server.”
- The user initiated a lightning smart contract with several funders.
- The user sends a request to the surrounding funders. The funders sign the request.
- The user sends the signed requests by funders directly to the Nostr relay connected to these funders.
- The user now launches the ZKCSP construct to ensure that payment is only received from the funders after the correct request data has been passed.
Once step 3 occurs, the user modifies the original request signed by funders in step 4. The user adds additional content on top of the original request, specifying the amount to be deducted from both the user’s and the funder’s balances (which must be the same amount plus the funder’s fee), and then signs the content added to the original message.
If the user specifies sats to be sent that exceed the amounts they own, or exceed the amounts frozen by funders on that Nostr relay, the Nostr relay will reject the request as it will not be able to get paid.
In this way, users can connect to many Nostr relays while only locking funds with a few funders. Lightning Network can also use a similar approach where trust-minimized funders are intermediaries between users and merchants without permission. Ordinary P2P lightning hops can also be used in Nostr 2.0, but this method may be useful if routing and channel balance failures occur frequently.
Whitelist Unlock Paid Nostr Relay
If a Nostr relay wishes to store all the data that these users view, they can whitelist certain keys. If a Nostr relay cannot whitelist users who wish to store data, they will store any data sent to them. If users can always send data to relays for free, users will never need to pay the Nostr relay. Nostr can only provide paid options when relays have the option to refuse to store data that cannot be paid for. Free relays still exist, but currently there is no option for paid relays.
Paid Nostr relays can use the whitelist to selectively store all data viewed by their paid users each day instead of attempting to store all Nostr data for free. Some Nostr relays will continue to operate in free mode, but at maximum scale, this is unsustainable as most free relays are just enthusiastic hobbyists. The ability of Nostr relays to decide which data to store with the whitelist (even if we could safely assign a public key to each Nostr profile) will not be possible to achieve.
Each configuration file has a whitelist function unlocked by a public key: a Bitcoin address becomes your Nostr public key. This ultimately allows us to assign a public key to each configuration file. There is no benefit for users to create new public keys for each post… because they are all associated with their configuration file! This is different from Bitcoin. Unlike Bitcoin, having multiple public keys for users in the same application does not increase privacy. The Nostr profile public key must match the public key of the Bitcoin transaction that contains the weekly hash value (Merkle root and entire tree hash of all user posts). Unlike Nostr users who use Schnorr signatures, they will need to use a Bitcoin wallet (mobile wallet/lightweight wallet or full node) to sign.
The beauty of this is that every Nostr account will be represented by its Bitcoin address, which means that users can send payments directly to Nostr accounts without having to request two different public keys. This reduces the cognitive cost for new users in the system. Users still need to send each other public keys or DNS to find each other on Nostr, rather than using usernames.
If other Nostr applications use different public keys, they can still be attached to the same decentralized identity (DID) – so the way to identify your account remains consistent across applications. However, this Nostr consensus rule will limit each configuration file on each Nostr application to only use one public key.
DHT as a peer discovery leaderboard.
Relays can use a distributed hash table (DHT) to find other relays. Relays can share their whitelist in the distributed hash table by listing the public keys that store the data. In this way, relays that lack branches for a public key’s data can scan the DHT and directly connect to the IP addresses of other relays that claim to store those missing branches. Then, relays can download the missing branches directly from these other Nostr relays.
Relays will also be able to find the most active relays by checking how many previous ZKCSP transactions – recent and all-time – those relays have solved on the chain. In this system, all Nostr relays become full nodes, so auditing previous transactions of other relays will be easy. This will make the cost of forging trust expensive, as on-chain transactions always require transaction fees. If Nostr relays open many channels to themselves to gain trust from other relays, he will have to pay a lot of transaction fees to maintain weekly false reputation. After the attacker is unable to provide the missing branches, timeouts will cause the relays to disconnect – so this is a temporary and expensive attack (just like a 51% attack is a temporary and expensive attack).
Unlike mining, DHT is not anonymous because each Nostr relay’s public key will be listed next to their IP address in the DHT, but it will avoid the need for relays to blindly send requests on the network – at a large enough scale, this would lead to network overload. Nostr relays that want higher privacy can use VPNs or other IP protection services.
Users will not have access to this same trust system because they are not full nodes. However, users can rely on them.
Financiers indirectly connect to hundreds of Nostr relays
Because users automatically store all block headers in their light wallets, users can see which miners are the most active. Having miners become financiers will allow users to filter out the most popular miners, so they don’t have to blindly bind funds with random financiers that are not survival-capable on the network.
Financiers (miners) only need to lock their sats with Nostr relays, without passing the data itself between users and relays. Financiers (miners) only need to sign user requests so that users can directly interact with all Nostr relays connected to financiers – the 4 steps of ZKCSP+Lightning as described above.
Conclusion
If there were no Nakamoto consensus blockchain for Bitcoin, the Lightning Network would not exist because users would have nowhere to store their off-chain transaction bundling proofs.
Just as users bundle all of these Lightning Network transactions together and place small proofs into the blockchain, we will bundle all Nostr data and place small proofs into the blockchain. Similarly to how the Lightning Network provides instant payments on Layer 2, Nostr may be able to provide data storage on Layer 2 without the risk of insecure sidechains.
It uses the same approach as the Lightning Network, where the Nakamoto consensus blockchain of Bitcoin is on Layer 1 and Nostr+ZKCSP Lightning is on Layer 2.
Like what you're reading? Subscribe to our top stories.
We will continue to update Gambling Chain; if you have any questions or suggestions, please contact us!
