Ethereum: Sharding Explained

The scalability is at the front and center of the crypto community. So what are the approaches the community has decided to take? The solution is two-fold.

1)The first approach is to improve scaling through off-chain solutions, also known as layer-2 scaling.
2)The other approach is to modify the design of the protocol altogether to fix the fundamental problems with parallelizability the blockchain faces. Unfortunately, many of us protocol devs often look at these problems and instantly feel put off by the immense difficulty they pose.

This post will break down the current approach the Ethereum core team is taking towards sharding and expose its current limitations and paths for improvement. By the end of this post, you’ll know enough to explore this problem on your own and who knows, maybe you’ll be the one to build the first sharding client!

What is Sharding?

Currently, every single node running the Ethereum network has to process every single transaction that goes through the network. This gives the blockchain a high amount of security because of how much validation goes into each block, but at the same time it means that an entire blockchain is only as fast as its individual nodes and not the sum of their parts. Currently, transactions on the EVM are not parallelizable, and every transaction is executed in sequence globally. The scalability problem then has to do with the idea that a blockchain can have at most 2 of these 3 properties:

Decentralization
Scalabiltity
Security

If we have scalability and security, it would mean that our blockchain is centralized and that would allow it to have a faster throughput. Right not, Ethereum is decentralized and secure.
Well can’t we just increase the block size, or in Ethereum’s case, the GAS_LIMIT, to increase throughput?

A smarter approach

is the idea of blockchain sharding, where we split the entire state of the network into a bunch of partitions called shards that contain their own independent piece of state and transaction history. In this system, certain nodes would process transactions only for certain shards, allowing the throughput of transactions processed in total across all shards to be much higher than having a single shard do all the work as the main chain does now.

Before we dive into how a sharded blockchain actually works, let’s go over some important vocabulary:

State: the entire set of information that describes a system at any point in time. In Ethereum, this is the current account set containing current balances, smart contract code, and nonces at a given moment. Each transaction alters this state into an entirely new state.

Transaction: an operation issued by a user that changes the state of a system

Merkle Tree: a data structure that can store a large amount of data via cryptographic hashes. Merkle trees make it easy to check whether a piece of data is part of the structure in a very short amount of time and computational effort.

Receipt: a side-effect of a transaction that is not stored in the state of the system, but is kept in a Merkle tree so that its existence can be easily verified to a node. Smart contracts logs in Ethereum are kept as receipts in Merkle Trees, for example.

With this in mind, let’s take a look at how Ethereum 2.0 would work. We will create a sidechain known as a random beacon chain that stores hashes to main chain blocks in its own blocks. This sidechain will be a full Proof of Stake system implementing Casper FFG and will provide a source of distributed randomness that will allow us to build a sharding system on top of it.

The problems with sharded blockchains become more apparent once we consider that possible attacks on the network. A major problem is the idea of a Single-Shard Takeover Attack, where an attacker takes over the majority of collators in a single shard to create a malicious shard that can submit invalid collations. How do we solve this problem?

https://github.com/ethereum/wiki/wiki/Sharding-introduction-R&D-compendium
for more answers for sharding FAQ in github. Screen Shot 2018-06-07 at 4.53.15 PM.png

An initial idea is use the concept of receipts for this system to work.

Raul (Address on Shard 1) Wants to Send 100 ETH to Jim (Address on Shard 10)

1)A transaction is sent to Shard 1 that reduces Raul’s balance by 100 ETH and the system waits for the transaction to finalize
2)A receipt is then created for the transaction that is not stored in the state but in a Merkle root that can be easily verified
3)A transaction is sent to Shard 10 including the Merkle receipt as data. Shard 10 checks if this receipt has not been spent yet
4)Shard 10 processes the transaction and increases the balance of Jim by 100 ETH. It then also saves the fact that the receipt from Shard 1 has been spent
5)Shard 10 creates a new receipt that can then be used in subsequent transactions

This Sounds So Complex for Solidity Devs and Ethereum Users to Understand! How Will We Educate Them on Sharding?

They don’t need to. Sharding will exist exclusively at the protocol layer and will not be exposed to developers. The Ethereum state system will continue to look as it currently does, but the protocol will have a built-in system that creates shards, balances state across shards, gets rid of shards that are too small, and more. This will all be done behind the scenes, allowing devs to continue their current workflow on Ethereum.
other feauters of Sharding
*Beyond Scaling: Super-Quadratic Sharding and Incredible Speed Gains
*Resources and Where to Get Started

Ok so now you want to start coding a sharded blockchain! How do you begin? At the most basic level, the proposed initial implementation will not work through a hard fork, but rather a sidechain known as a random beacon chain that will serve as a proof of stake + sharding system.

This beacon chain will manage validators and their sampling from a global validator set and will take responsibility for the global reconciliation of all shard states. Vitalik has outlined a fantastic reference doc for implementing sharding here: https://ethresear.ch/t/convenience-link-to-full-casper-chain-v2-spec/2332/4

To get explore this beacon chain architecture in detail and to learn more about how the system works, check out the following resources:

Sharding FAQ: https://github.com/ethereum/wiki/wiki/Sharding-FAQ