This contest was organised by Catered Content on a revolutionary sharding blockchain system called HARMONY
INTRODUCTION
The Blockchain
It would not be wrong to tag the blockchain as one of the biggest innovations of the twenty-first century considering the tremendous effects it has had on the global economy. Application ranges from finance, technology, education, health, gaming and so on.
Its major breakthrough was in 2008/2009 but like every major breakthrough of science and technology, it has its little beginnings, which can be traced back to 1991, but, history is not really the aim of this article.
In 2008, it started to gain grounds, thanks to the work of Satoshi Nakamoto (still uncertain whether it is a person or a group of revolutionaries) and the first decentralized ledger.
The major application of the blockchain is to ensure that individuals or users can carry out transactions laissez faire (with freedom/ without restrictions). There will be no central authority overseeing the transactions ensuring a certain level of security, privacy and complete anonymity. The network participants will have distributed to them, a transparently shared ledger or database containing blocks of data where each created block has a reference to the previous block thus forming a chain of blocks. The name blockchain was no misnomer after all.
BTC and ETH
The original Bitcoin blockchain was designed as a peer-to-peer (P2P) payment system that allows people transfer value over the internet without any intermediaries. The gimmick of anonymity pooled in a lot of users but the bitcoin had a serious issue with performance and cost. Bitcoin carried out just 7 transactions per second (7Txps) and it was prohibitively expensive to operate as a payment system.
In 2014, another blockchain was proposed called Ethereum (ETH) which brought about electronic contracts called smart contracts and offered the users some level of leverage but it was not all smooth sail. Just like its Bitcoin colleague in the blockchain, it had its fair share of problems. The most noticeable was scalability. Although the transaction rate increased to 15Txps, it failed to support high-throughput applications such as gaming or decentralized exchanges.
whitepaper pg 2.
Consensus
Looking at the blockchain in general and how new blocks are appended i.e. before blocks are assigned to the chain, the process of MINING occurs. This involves a distributed consensus system that is used to confirm pending transactions by including them in the blockchain. Consensus is basically where the participants in the network reach an agreement on what block to be assigned to the blockchain. There have been various approaches towards achieving a credible and safe consensus mechanism and they include:
[a] Proof of Work (PoW): computational cryptographic puzzles are solved by miners and the miners get rewarded. Used by Bitcoin blockchain.
[b] Proof of Stake (PoS): a certain amount is staked by participants.
[c] Decentralized PoS (DPoS): participants involved in creating new blocks elect a leader by voting rather by some software algorithm.
All of these consensus mechanisms could not truly address the issues faced by the blockchain without having certain trade-offs such as speed for security or security for complexity and so on.
The goal of any network, is to scale well i.e. to grow bigger. Since 2009, blockchain technologists have been able to create several ways to do this. However, as the network grows, so does the computational power required to process and analyse the decentralized database/ledger. Energy is expensive and computing power is not easy to come by. The blockchain technology is a viable and cutting edge technology of the modern world and so a new method was devised to ensure that blockchain continues but with lesser computational power. This is where the process of sharding comes in.
In light words, sharding is the process of dividing the database computational responsibility into smaller segments thus reducing the overall burden of computational power.
Source
Ziliqa is the first blockchain that proposed the concept of sharding which should address scalability problems of the blpckchain. Ziliqa’s approach did not turn out so well because:
[i] there was no state sharding i.e. the storage blockchain was divided making it difficult for machine with limited resources to participate.
[ii] There was also the problem of single shard take-over attack due to Ziliqa’s reliance on PoW as its randomization.
whitepaper pg 2
It should be noted that for any blockchain to meet fully, the desires of the public there MUST be a level of:
[i] Security: how secure is the system; how best does the system guard against attackers.
[ii] Scalability: can the network grow well without any glitches or other performance trade-offs.
[iii] Consensus: is there integrity in the mechanism and participants whom will be signing a new block of data
HARMONY HAS COME WITH THE AIM OF MAKING A BETTER BLOCKCHAIN
HARMONY
Harmony is the next generation sharding-based blockchain, that is fully scalable, provably secure, and energy efficient and addresses the current problems of existing blockchains leveraging exisiting technologies in the blockchain and best research results and engineering practice.
These are some of the aspects Harmony will be making headway:
[i] full scalability: making a better sharding than Ziliqa.
[ii] security: a provably secure random number generation that is unbiased.
[iii] A better proof of stake that is adjustable.
[iv] consistency in cross-sharding communication.
[v] efficient and faster consensus mechanism
Aim of harmony
Harmony provides the world with a scalable and secure blockchain system that is able to support emerging decentralized economy. Applications which were thought not feasible on the blockchain such as interactive fair games, visa-scale payment systems, and internet of things (IoT), will be leveraged by Harmony
BETTER CONSENSUS WITH FBFT
Validators on the blockchain, help reach a consensus on the next block and they do so based on the already existing consensus mechanism of the bockchain system. As mentioned earlier mechanisms such as PoW, PoS and DPoS are the common ones. Another consensus mechanism is the Practical Byzantine Fault Tolerant (PBFT). A node is elected as a leader and the rest of the nodes are validators. The leader broadcasts a block proposal and validators in turn broadcast their votes to everyone else. The broadcast is to ensure that the votes are counted. When 2f + 1 voters are consistent and the total number of validators sum the leader us 3f + 1 is given as the number of malicious voters. Its order of operation is O(N2) and this provides some level of complexity and makes scalability difficult.
Harmony’s approach to consensus is to achieve scalability, speed, and a less complex communication. This is why their choice of consensus protocol is the Fast Byzantine Fault Tolerance (FBFT). This is how it works:
the validators need not broadcast to everyone else, their votes. The leader generates a multi-signature contract which is signed by each of the validators and ultimately reducing the communication complexity. The magnitude of operation is reduced to
O(N).
The process for Harmony’s FBFT protocol is as follow:
[i] The leader constructs a new block and broadcasts the block header to all validators.
(less complex communication)
[ii] The multi-signature contract is signed by the validators and sent to the leader.
[iii] When the valid signature threshold is met, the leader broadcasts a bitmap indicative of all signers to the contract.
[iv] The validators will check for consistency in the validity of signature threshold and signs the broadcast contract from the leader back to the leader.
(consistency and security)
[v] the leader waits for another valid signature threshold at [iv] before committing the new block.
These validators are chosen by stake and so the signing process also has an alternative. The leader need not wait for the 2f + 1 signatures he can go ahead if the signatures from validators that have voted surpass the 2f + 1voting shares.
HARMONY AND SHARDING
Against the background information on what sharding is and its role to creating a more scalable blockchain system without outrageous power expenditure, it is important to note that certain solutions are already on ground. Ziliqa for example assigns different transactions to shards and these shards are processed separately and results from respective shards are culminated at the directory-service committee. Its solution as earlier stated in this article is not sufficient. Other solutions such as Omniledger and RapidChain looks into the drawback of Ziliqa and implements STATE SHARDING. Here, each shard holds a subset of the blockchain state.
Apart from the issue of scalability addressed inherently by these sharding solutions, the issue of security is also addressed by ensuring that the allocation of nodes to a given shard is as random and verifiable and unbiased as possible and also, that shards are not easily corrupted during the time it takes to sign the new block.
Harmony after carefully looking at these solutions (Ziliqa, Rapidchain, Omniledger), has designed a proof-of-stake (PoS) based sharding scheme that is linearly scalable and provably secure. The solution will make use of two other concepts called BeaconChain and ShardChain which together, ensures security, randomness and blockchain state save realization.
whitepaper pg 6
[i] DRG
There are three (3) approaches to sharding: random, location-based and centrally controlled and Harmony chooses to go with random. In this choice of sharding, a mutually agreed upon random number is used to determine which nodes get assigned shard. This random number in order to maintain is name “random” has to be unpredictable, unbiased, verifiable and scalable.
From Omniledger, they use the RandHound protocol that divides participants into multiple groups of size c but the speed was not encouraging even though it satisfied the first three out of the four criteria for randomness of a number for sharding. RapidChain took a simpler swing at the ball of randomness and it was total miss; the system was not secure enough.
Harmony’s approach combined the strengths of the aforementioned solutions and reduced complexity. Their approach has a random number that is verifiable and unbiased and also leverages BFT in attaining the final random number. This feat was achieved using Verifiable Random Function (VRF) and Verifiable Delay Function (VDF). The former is used to select the group of consensus validators and the latter delays the revelation of the number so as to prevent last-revealer attack.
whitepaper pg 7
This whole process of sharding and random number generation all happen within a time frame called EPOCH. The epoch is a predetermined time interval during which the sharding structure is fixed and each shard continuously runs consensus with the same set of validators. The epoch begins, the random number is used to allocate shards to nodes and validators who desire to vote or partake in the next block in epoch (e) must place their stakes before the start of that epoch (e - 1).
Another way Harmony looks at the issue of security is in the aspect of validators registration. There are ways to prevent Sybil attacks but Harmony takes a different approach. The prospective validator must stake a certain amount of tokens for him to be eligible and the number of voting shares is proportional to the amount of tokens staked. The VOTING SHARE is a virtual ticket that allows the validator cast one vote in the consensus period. The amount for voting share is algorithmically adjusted and at the beginning of each epoch shards will be randomly assigned to new validators. As an extra layer of security, the amount of voting shares owned by malicious voters is kept below one-third of all the voting shares in that shard. The voting shares are randomly assigned to shards rather than to individual validators.
The reason behind sharding by validators being shunned in the Harmony system is that a single malicious voter can possess more than a quarter of the total voting shares in the shard and this is passed the acceptable amount which is pegged at one-third. Even in large scale attacks by malicious voters or malicious validator, Harmony has a backup plan. The random number generated which helps in allocating shards to the validators, once revealed at the start of the epoch, is used to carry out a permutation on all the voting shares and these permuted list of voting share will be divided into the number of shards.
whitepaper pg 8-9
Harmony ensures that users with large stakes a validators enjoy their rewards appropriately. A single validator may be assigned to multiple shards if he has voting shares assigned to those shards. The leader is chosen by a first-come-first-serve rule i.e. the owner of the first voting share after permutation. The larger the stake, the higher the chances of being selected as leader and the greater the risk, should they flaunt the protocol of the harmony system.
[ii] RE-Sharding
Sharding on its own is not sufficient especially when it will be fixed. Attackers can overtake a single shard. The attackers can achieve this overtaking using any of three methods:
[-] corrupting a subset of nodes at a predetermined state. (State Round Adaptive)
[-] corrupting the subset of nodes over time during the epoch. (Slowly Adaptive)
[-] corrupting the subset instantaneously at any time. (Fully Adaptive)
Omniledger assumes model of attack number 2 and tries to prevent it by replacing validators in all shards at the start of every epoch. This approach, however, had two problems: cost of bootstrapping and security during consensus.
Harmony approaches resharding with a different rule called the CUCKOO-rule. At the end of every epoch, those validators who decide to withdraw their stakes are evicted from the network and those who keep theirs stay. Shards will be randomly assigned after the voters get new shares. The shares will be assigned to shards whose existing voting shares are more than the median of the total voting shares. For the rest shards whose voting shares are less than the median, a random amount will be allotted. The voting shares are balanced and security is maintained.
[iii] Fast State Synchronization
One of the major drawbacks of the Ziliqa’s approach to achieving scalable sharding mechanism is the inability for the system to save states of the blockchains.
Having to download the whole blockchain history and reconstructing current state is computationally heavy and time consuming. The current state is by a great deal small than the whole history. Harmony takes a better approach to achieve a faster state synchronization.
[iv] Beacon and Shard
Sounding so much like the name of sitcom, is the two basis for the better sharding approach that Harmony has brought to the blockchain.
A shard chain is a blockchain that processes and validates its own transaction and stores its own state. This means that everything concerning the shard is within that shard i.e. restricted. However, harmony’s system also allows for cross-shard communications.
In cross shard communication, the functionality of the shard is extended beyond its own scope. There are three catergories of shard communication:
[-] Main Chain [-] Client Driven [-] Shard Driven
Harmony adopts the shard driven shard because the messages sent and received between nodes that are assigned shards need no external help to do so. Agreed that there are some drawbacks to this approach, but its merits outweigh the drawbacks. Complexity in communication is also reduced by the choice of routing protocol chosen by Harmony (KADEMLIA). The protocol shall be discussed later.
The beacon chain on one hand serves more purpose than the shard chain. The beacon is also a shard but with extra functionalities. One is to accept the stakes by intending validators and generating random number for permuting in allocating shards.
In the matters of security and consistency, the beacon chain helps Harmony by adding the block header to the shard chain and also verifying that the correct validators have signed the multi-signature contract. This addition of block headers serves two puposes:
[-] Increase difficulty of attacking a single shard
[-] Reduce network cost of broadcasting block headers among shards. Each shard already has a valid block header for all the shards this makes validation easier.
HarmoNetwork
Network performance is a major bane of blockchain systems and Harmony focuses on creating an efficient networking solutions for real world application.
Internetworking in Harmony could be seen as the cross-shard communication and the choice of routing protocol is the KADEMLIA. In Kademlia, each node maintains a routing table that contains nodes from different shards. When a message is to be sent from a node in shard A to another shard B, the nodes in A will check to see which network ID has the closest hop to B. the operation of routing is O(logN) which is considerably lesser in complexity and significantly reduces the network load.
Nodes that carry out computations for theses shards in the Harmony network especially those in residential areas cannot be reached from the outside unless specified by the router employing a networking concept known as Network Address Translation (NAT). the devices vary and with each device a specific method have been designed to work around them. The problem here is if there are n-number of nodes with different routers, then there are n-number of methods for these routers.
Harmony presents a solution called Interactive Connectivity Establishment (ICE). the P2P layer of the Harmony system will be able to detect the NAT and the right mechanism to go around it. One could say the solution ”is as cool as ICE”.
Another network utilization approach presented by Harmony is a system that supports locator mobility. Nodes (users) are bound to change IP addresses and usually when this happens, all communication breaks down and if a new IP is detected, the connection has to be re-established using the new IP. Such a handover (semantic for interruption) is very difficult to implement because it can complicate application layer protocol. Harmony leverages the Host Identity Protocol version 2 (HIP v2.0) to present a solution. With HIP2.0, node identity (crypto key) and node locator (where the node can be reached) is kept separately. Even in the event of a change in locator, the node identity is kept intact. Pretty HIP huh?!!!
REWARDS
[a] consensus reward
After the successful commitment of a block, the validators who obviously signed the block will be rewarded commensurate with their voting shares. The same goes for transaction fees.
[b] slashing
For any misbehaviour on the network, a certain amount of token will be slashed. If a validators are found to be dishonest, all the stakes under the same shard will be slashed. A proof of misbehaviour can be two signed blocks of conflicting data. Once a validator tenders this proof, the culprit’s token will be slashed and the distributed to the “provers”.
SYNOPSIS OF HARMONY’S CONTRIBUTION
Transparency
Using the Cuckoorule system, Harmony is able to keep a balance between old and new validators and their voting shares at the start of every epoch.
Users who possess a high number of tokens and indicate interest, are made validators.
Validators with the highest number tokens staked have a higher chance of becoming the leader of a shard. NO RISK NO REWARD!!!
Security
- The DRG approach of harmony ensures that no single person can predict or manipulate the random number for shard allocation that will be generated.
- Beacon chain help to strengthen another of Harmony technology called shardchain to maintain the true randomness other generated number.
- Validators who go against the protocol of Harmony will have their stakes slashed if the apt evidence is provided.
Speed
- With Fast State Synchronization in the Harmony system, users need not download the whole blockchain history to fully reconstruct the whole chain. The Beacon chain and Shard chain and the ability to save block headers will do this. Large number of users can now join faster.
Utility and Efficiency
- Different routers may have different technologies in address translation but that will not be a problem with Harmony’s interactive connectivity establishment (ICE).
- The HIP 2.0 used by Harmony will ensure that even when the nodes change their IP address, it will not affect the node identity on the blockchain.
- KADEMLIA routing protocol provides a better means of sending information from one shard to another.