There is a steady development of innovation in technology sector in different forms and the new technologies are advancing at a quicker rate than the human may ever have thought of. Out of such a large number of technologies, just a couple have gone on for quite a long time which really offered a genuine solution for the different issues/problem interface. One such innovation in technology is Blockchain technology. Over the most recent two decades, the physical world is endeavoring to advance into a digitized world and that, in fact, is a need to oblige the different needs of new ages. Blockchain innovation is fundamental to understand an advanced world, to manage the administration of digitized resources, to value transfer in a trustless environment and through a distributed system.
Blockchain is a technology which is fundamentally based on encryption and peer to peer distributed ledger. This technology is such a powerful technology, which disrupts the traditional third party based centralized technology. It works in a way similar to that of worldwide internet connection and the value transfer can happen in a trustless environment. We are sending so many things over the internet through a centralized server, where the data is routed through a centralized server before being delivered to the desired beneficiary. Blockchain technology removes this centralized server and makes the transaction happens in a peer to peer distributed ledger.
In this context, we can say, the future is trustless with Blockchain technology but the application of Blockchain technology has not yet converged to a point where the users/enterprises/large scale economies/industries will find it scalable. It is not like that scalable blockchain solutions are not available as on date. When we talk about it in 2019, we can say that we have some scalable blockchains as well. For example EOS. But there is a point to consider here. While we have BTC or ETH which are not scalable but relatively more secured and sufficiently decentralized. Both BTC and ETH have PoW consensus mechanism. On the other hand we have EOS which is a highly scalable blockchain and it has DPoS consensus mechanism but the extent of decentralization is not at par with BTC or ETH.
Note - The above figures are average figures and it must be noted that VISA has a capacity of 56000 t/s and the daily peak rate for VISA is roughly 4000 t/s.
Therefore it is apparent to note that there is tradeoff between scalability and the extent of decentralization. When you make the blockchain scalable, you have to compromise with the decentralization to some extent. On the other hand when you focus on the decentralization aspect, the transaction throughput remains an issue. Security is another aspect which varies from one blockchain to the other.
In order to enhance the performance of applications on the top of blockchain technology, it has to be sufficiently scalable just like the "centralized electronic payment systems like VISA, PAYPAL, etc". But at the same time it has to be sufficiently decentralized and secured too. One should not be at the cost of other. While scalability is a key requirement in order to encourage the implementation of Blockchain technology in various domain, "security & decentralization" are critical and fundamental to Blockchain technology. Therefore there is a need of blockchain solution which can preserve scalability, security & decentralization" simultaneously.
Since 2009, many blockchain infrastructures evolved. With the passage of time, the blockchain world realized that scalability is a major issue and needs to be addressed. Since 2017, serious efforts have been put to make the blockchain scalable. While some new blockchains evolved with high transaction throughput(for example EOS), the existing blockchains focused on second layer scaling solution. The different second layer scaling solution are Lightning Network, Sharding, etc.
Harmony is the future generation blockchain solution which is fully scalable, provably secure and sufficiently decentralized. Its scaling capacity is based on optimized sharding technology. Zilliqa was the first blockchain project with sharding technology. Harmony is also a blockchain project which is based on sharding technology but it is provably a better & efficient sharding solution than what Ziliqa does. In Zilliqa the sharding is at the transaction layer where as Harmony implements deep sharding, means it implements state sharding along with sharding at transaction layer.
Therefore with optimized sharding technology(state sharding), Harmony offers a future generation blockchain which can support the performance of new & emerging dApps on the top of it. Now along with the core fundamentals of blockchain technology(such as security & decentralization), the blockchain can be feasible & scalable to a variety of real-world implementation. High volume exchanges with low latency, almost instant transactions like centralized electronic payment, games, IoT, large scale industries can become a reality now with Harmony blockchain solution.
Harmony approaches a way which has never been approached before(i.e. state sharding). The potential is huge as it takes the sharding deep (i.e. sharding at the consensus layer) without compromising the security and decentralization of the network.
The key features of Harmony state-sharding blockchain are:-
Scalability
Harmony implements optimized sharding(deep sharding), which shards the state of blockchain, so the different nodes can keep different states, can make cross-shard communication to reach consensus and concurrently process transaction, which ultimately makes the blockchain exceptionally scalable to an extent which has never been achieved before.
Secure Sharding
Harmony employs DRG protocol to generate random number which is unpredictable, unbiaseable, verifiable and scalable. Re-sharding further makes the entire sharding arrangement more secure and efficient.
Consensus
Harmony uses PoS to enable validators to stake and partake in the network and further the consensus is achieved with linearly scalable FBFT algorithm which simplifies the complexity of PBFT. So Harmony is both energy efficient and linearly scalable.
Adaptive Threshold PoS
The threshold stake required by nodes to join a shard is adjusted based on the total volume of stakes in the entire network, which prevents the attackers to concentrate their stake on a single shard. Further, the threshold is low enough so as to ensure that anyone can participate in the network.
Scalable Networking Infrastructure
Harmony sharding architecture enables block to propagate quickly within the shards or across the network. Further it scales logarithmically to facilitate the communication between the shards.
Cross-shard communication
Harmony uses "Kademlia routing" for cross-shard communication, which simplifies the communication complexity.
There are many types of consensus mechanism and the security & scalability of a particular blockchain is largely dependent upon the consensus mechanism. The notables consensus mechanism are PoW(BTC, ETH, etc), PoS, DPoS(EOS, STEEM), PBFT(Ziliqa), etc. Harmoney uses a modified consensus mechanism which is known as FBFT.
In PBFT, one node is elected as a "leader" and all other nodes are elected as "validators". The consensus process has two phases namely: "prepare" & "commit". In both the phases, the proposal is broadcasted by the leader to all other validators and the validator nodes further broadcast it to all other nodes. Both the phases finishes when more than 2f+1 consistent votes are seen, where f is the number of malicious validators. In PBFT, the process of rebroadcasting leads to communication complexity O(NxN), which is not scalable for a blockchain containing hundreds, thousands or even more number of nodes.
Harmony solves this complexity by making it O(N) in stead of O(NxN). That means it makes the communication protocol linear. That is why it is called as FBFT(Fast Byzanitine Fault Tolerance). Here the validator nodes need not have to rebroadcast their votes to other nodes, rather the leader collects the validator's vote by multi-signature signing. Therefore the communication complexity gets simplified from O(NxN) to O(N).
Furthermore, Harmony adopts BLS signature in stead of schnorr signature for multi-signature signing. In schnorr signature, it requires two round-trips for a single multi-signature where as in case of BLS signature, it requires only one round-trip. Therefore Harmony with FBFT & BLS signature speeds up the block broadcasting process.
The validators can partake in the network by staking a certain amount of tokens(PoS method). Those who will stake largest amount of tokens have higher chances of being selected as a "leader" node. PoS is definitely a better option in the context of saving energy.
FBFT consensus process in Harmony happens in the following ways:-
Prepare Phase
- The leader node constructs a new block and broadcasts it to all other validator nodes.
- The validator nodes sign the block header using BLS signature.
- The leader waits for 2f+1 valid signatures and aggregates that to BLS multi-signature with a bitmap which contains a log of those who have signed.
Commit Phase
- The validators make sure that the multi-signature has at least 2f+1 signers and then verify the transaction in the block. The validators sign the message received in the "prepare phase" and send it back to the leader node.
- The leader again waits for 2f+1 valid signatures and aggregates that to BLS signature with a bitmap. So now the leader commits the new block with all the details such as multi-signatures, bitmap, etc and broadcast it to all the validator nodes to commit.
Note- The voting shares is proportional to the amount of tokens staked. Therefore the validators have more voting shares who have higher amount of tokens staked.
In Harmony, the network is divided into a number of shards which contains hundreds or thousands of nodes. The assigning of nodes to shards is critical from the consideration of security and the potential risk of any attack, because malicious actors can easily corrupt a single shard if the same nodes are assigned to it. Therefore the point is that the attackers should not be able to choose a shard of their choice and the attackers should not be able to know or guess ahead of the schedule, which shard will work. In order full proof this, Harmony employs DRG protocol which is known as Distributed Randomness Generation Protocol.
Under DRG protocol, a random number is used to assign nodes to shards. The random number is unpredictable, unbiased, verifiable and scalable. That means no one can predict it, it is not biased of any participant, it is verifiable by any observer and it can scale to a large number of participants.
The DRG protocol leverages on the strength of Verifiable Random Function(VRF) & Verifiable Delay Function(VDF) for randomness generation. VRF is a cryptographic procedure to select the validator nodes where as VDF delays the revealation of actual random number in order to prevent last-revealer attack.
The DRG protocol is linear, that means it is O(n). It consist of the following steps:-
- Each validator node creates a VRF based random number and sends it to the leader node.
- The leader node collects all the valid random number and generates the final random number from this data set.
- Then by using VDF, the revealation of this final random number is delayed so as to prevent any attack and to make it more secure. VDF can prevent a malicious leader from biasing the randomness.
By employing DRG protocol, the validator nodes are assigned to a particular shard for a predetermined time interval. This predetermined time interval is known as Epoch. The validators who partake in the in epoch "e", needs to stake their tokens during epoch "e-1". Before the beginning of each epoch the random number is generated and sharding structure is determined.
Re-sharding makes Harmony more secure and more balanced. It uses Cuckcoo-rule in this re-sharding mechanism. It happens in the following ways:-
- For a validator to partake in epoch "e", needs to stake tokens in epoch "e-1".
- At the end of the epoch, those who withdraw their stake are evicted from the network.
- The new validators who stake get new voting shares.
- These voting shares are randomly assigned to the shards which have greater than the "median of the total voting shares".
- A constant number of voting shares from all the shards are then randomly redistributed to the other half of the shards which have less than "median of the total voting shares".
The sharding architecture of Harmony consists of "Beacon Chain" & "Shard Chain". A shard chain validates a subset f total transaction and it stores its own state. In essence, Beacon chain is also a shard chain. However it is special, as it has additional roles & functionalities:-
- Beacon chain generates random number using DRG protocol.
- Beacon chain accepts stakes from validators.
The Beacon chain helps in maintaining the security & consistency of the shard chain states. After every new block committed by the shard chain, its block header is sent to the Beacon chain (via Kademlia routing). The beacon chain checks the validity of the block header by checking the hash of its previous block & the signers of the block multi-signatures and then finally broadcasts it to the network.
In any sharding based blockchain it is extremely important to ensure cross-shard communication, as it extends the utility of the shards beyond itself. But most importantly, the communication must be simple. Harmony implements shard-driven approach. The communication complexity in this approach has O(N) network cost. Harmony further uses Kademlia protocol to simplify the communication complexity from O(N) to O(log(N)).
Kademlia routing
Each node in the network of Harmony maintains a "routing table" where the distance of all other shards of the network are defined. This distance is a function of the shard ID. When it comes to cross-shard communication using Kademlia protocol, it does not require to broadcast the message to all the neighboring nodes, in stead the nodes can send a message to another shard with closest shard ID. The message travels across O(log(N)) nodes before it reaches the destination shard. Therefore it simplifies the communication complexity and greatly reduce the network load.
There are different types of blockchain solutions available in the market. Some of them focus on enterprise solution, modularity, etc while some are focused on performance, scalability, etc. The use-case may slightly vary from one blockchain to other. But Harmony as a blockchain infrastructure is very unique in its own way and it has addressed the problem and issues of the existing blockchain very precisely. It utilizes deep sharding technology which preserves both scalability and decentralization, apart from that it has made the arrangement of sharding architecture in such a way that any attack vector would look less effective. The DRG protocol makes it unpredictable for the malicious actors.
The decentralized applications and various projects before choosing a particular blockchain infrastructure basically look at its security features, degree of decentralization and most importantly scalability(which is key for the next generation dApps). Some of the available blockchain solutions may offer scalability but the security feature may not be robust, may not be sufficiently decentralized. Similarly some of the solution may offer security and adequate decentralization but scaling capacity may be below par. Therefore Harmony can be that much needed infrastructure for the next generation dApps and projects, where the projects can leverage on the robust security features, exceptional scaling capacity of Harmony state-sharded blockchain to achieve their respective business goal.
There are many other blockchains which implements sharding technology in their chains. Zilliqa was the first ever blockchain based on sharding technology. Other blockchains which implements sharding technology in their chains are Quarkchain, Multivac, Harmony, etc. But there is a big difference in sharding of the chain between Ziliqa and Harmony. In Ziliqa, the sharding is implemented at the transaction layer, where as in case of Harmony the sharding is implemented at the consensus layer in addition to transaction layer.
In Zilliqa, the network is divided in to a number of segments(known as shards) and the transactions are processed by these shards concurrently which effectively improves the transaction throughput. However, in Zilliqa, each node in the shard has to store the entire blockchain state to be able to process the transaction. So the problem here in Zilliqa is that as the blockchain size grows, it will become difficult to maintain and handle the large size ledger. Furthermore, the participation in this type of network may not be general, because not all the machines are sufficiently resourceful and hence they can not participate in such a network. Therefore the degree of decentralization may not encouraging in such a network. Harmony solves this dilemma with its optimized sharding(deep sharding).
Harmony not only shards at the transaction layer, but also at consensus layer( means the state of blockchain or storage or record keeping of the blockchain). That is why it is also known as deep sharding or state sharding. All the nodes do not need to mainatin and hold the entire state of blockchain. Each shards only need to hold a subset of the blockchain state. Nodes can map the other nodes in the network which are relevant to a particular transaction and can transact & participate in the consensus process.
As it does not require to hold & maintain the entire state of blockchain in Harmony, it lowers the barrier to entry and even a simple computer can join such a network, that means the degree of decentralization can further be encouraged with Harmony. Therefore with Harmony's optimized sharding, not only the transaction throughput significantly improves but also it improves the degree of decentralization. The parallel processing makes the blockchain scalable to an extent which has not been achieved till date. That is why it has been attributed as future generation blockchain.
Harmony's state sharding blockchain employs sharding at transaction layer as well as consensus layer. That means the multiple shards don't need to keep the entire state of blockchain and they can keep a subset of it. Further, such an arrangement of sharding improves scaling capacity as parallel processing by different shards makes the blockchain sufficiently scalable. Therefore the implications of having such a network are:-
- It encourages the degree of decentralization. Simple & general machines(for example a simple computer) can participate in the network, as it does not require to store the entire state of blockchain . So it can be a more general public blockchain as anyone one with simple resources can participate.
- The new machines can sync with the network very quickly.
- It lowers the barrier to entry.
- As the network preserves security, decentralization, scalability, efficiency concurrently, it can be implemented with large scale economies, it can handle heavy traffic and transactions.
Harmony is a state-sharding blockchain which consists of a number of shards and different shards keep a subset of the blockchain state. That means the state of one shard is different than the state of other shards. However there is cross-shard communication to send message from one shard to another. In its sharding architecture it has Beacon chain and shard chain. Beacon chain and shard chain are similar, but Beacon chain has additional roles and it broadcasts the block header after checking its validity.
Each shard keeps a chain of valid block headers for all other shards, which is used to check the transactions from other shards. The adding of shard chain's block header into the Beacon chain significantly improves security. It makes difficult for the attacker to attack such an arrangement of sharding on multiple layers, as in such a case the attacker has to corrupt both the shard chain and the beacon chain in order to include an invalid block in the chain.
Furthermore, it is efficient and does not involve any complexity in network communication. In the composition of Beacon chain and Shard chain, the beacon chain broadcasts the block headers and the shard chains do not have to separately broadcast the block headers to all other shards. So the communication complexity is simplified from O(NxN) to O(N) in this arrangement of sharding.
With the combination of PoS and deep sharding, Harmony essentially becomes the first ever blockchain which can optimize the consensus layer as well as the networking layer, thereby making the blockchain exceptionally scalable, yet secure and decentralized. This opens up a huge potential of its implementation & use-cases in many existing and gray areas of businesses.
For example, in developed countires like Europe GDPR is in place, so now the users can trade their data and monopoly will end and the blockchain has given a new home for data trading, but Harmony can truly offer that solution to the use-case like data sharing and trading. In a similar way, gaming industry can utilize the Harmony blockchain infrastructure, high volume exchanges where low latency is critical can now utilize Harmony blockchain as it is exceptionally scalable. Enterprises and large scale economies will be happy to use the Harmony blockchain as the infrastructure as Harmony preserves all the three key aspects of blockchain: scalability, security, decentralization.
After evaluation of both the chains, the CEO and PMIC team got to know the following:-
The scaling capacity of ETH chain is 20 tps, whereas the reported testnet scalability of Harmony chain is 118000, and the potential scalability is even more in Harmony.
ETH chain has chain-based consensus, whereas Harmony chain is PoS based with Fast BFT algorithm which is linear.
ETH chain uses peer to peer gossip protocol, whereas Harmony chain relies on Kademila protocol which does not broadcast the message to all nodes and only broadcast to those nodes which fall in the shortest route, so communication is faster & saves the network cost significantly.
ETH chain does not have sharding structure, Harmony chain employs optimized state sharding technology and innovate the network at its root to achieve a scalability which is exceptional.
ETH chain is not resource efficient as compared to Harmony chain, because the new node has to download the full state of the chain in ETH, where as in Harmony the new node need to download only subset of the state of the chain. So in Harmony not only a new node can quickly synchronize with the network, but also resources like simple computers can join the network.
The CEO checks the comparative analysis of the two short listed chains and finally took a call and decides to integrate with Harmony chain so as to meet their requirement of real time reporting of different project sites.
What Harmony offers is really huge as a general general public blockchain. The whitepaper of Harmony suggests the extensive ground work made by the team of this project. They have really identified the key aspects of different projects and worked on them to find an optimal solution and offered that in Harmony blockchain. In the testnet, the scaling capacity has been reported as 118000 transaction per second with 44000 nodes. It has solved the trilemma of security, decentralization and scalability and offered a solution which preserved all of them.
The proven experience of Harmony team in large companies like Google, Apple, etc have a better understanding on what large enterprises, large economies, gaming industry expect from a blockchain solution. It has also anticipated the mass adoption of blockchain technology in future and it has done all the ground work to make Harmony as a general public blockchain which can really handle billions of people, heavy traffic, fast transactions like centralized electronic payment and last but not the least it is cheaper.
Roadmap
Hello @crypto-wisdom thanks for using the Realityhubs tag, Harmony blockchain is truly one of a kind, it has really solved the trilemma. The rate and pace of development of existing technology is very astonishing, for every 100 problem an existing tech has, there are 200 developers waiting to fix it and spearhead a solution, also the fact that most of these technologies are open source makes it more accessible, as a result, Vitalik of Ethereum can be on top of the world today with a cutting edge technology, and by tomorrow, another cuttimg edge technology is built that surpasses Vitaliks Ethereum.
Your review is exceptionally informing, detailed and extensive.
We look forward to your next review.
Realityhubs Mod
Hello, @crypto-wisdom
Thank you so much for sharing a detailed review. Keep up your great work.
Regards,
@anggreklestari
[RealityHubs Curator]
Congratulations @crypto-wisdom! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
You can view your badges on your Steem Board and compare to others on the Steem Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPVote for @Steemitboard as a witness to get one more award and increased upvotes!
Hey Garlam - Head of Marketing at Harmony here - really love the article and was wondering if you'd be open to more work as one of our writers!
Message me on telegram - @garlamw (and yes it is the real me - check my twitter to verify my telegram id - https://twitter.com/GarlamWon)