Image Source: https://hive-engine.com
Hive Engine Light Nodes
The Hive Engine sidechain has been running for quite some time now and since its start a lot of new smart contracts were added. This and the increasing usage has led to the sidechain becoming more heavy over time. Currently running a full node for the sidechain requires 4-8GB of ram and a minimum of 100GB of storage. This doesn't sound like a lot when compared to HIVE itself, but it's increasing a few hundred MB per day and even witnesses have to run full nodes currently, even though they don't really require all data for verifying blocks.
In the last week I have been working on making changes to the Hive Engine node software to enable running light nodes, which store sufficiently less data, including the data required by witnesses to verify blocks.
As a result it will be much cheaper to run Hive Engine nodes, if not the full history of blocks / transactions is needed and witnesses won't be required to run full nodes anymore. This will hopefully make the Hive Engine network more stable in the long run, encourage people to run more nodes and make the process of restoring in case of an error much faster.
The changes are still experimental, but if you want to test out a light node, feel free to do so and report back to me if having any issues. You will find more info on how to do so further down.
Pull Request
I have created a pull request 2 days ago, which I put up for discussion and review in the witness channel. In this post I will be outlining the most important parts of it and describe some of the benefits / downsides of light nodes and what exactly I changed.
Here is a link to the pull request, if you would like to have a look at the code yourself: https://github.com/hive-engine/steemsmartcontracts/pull/144
Description
The pull requests adds light node functionality to hive-engine. Instead of storing all data of the hive-engine blockchain, light nodes only store smart contract data and data required for block verification, but not the full chain (blocks, transactions). This will allow for faster restoring / synchronizing and make the network / the witness block signing process more stable.
How to enable
If you don't know how to run a Hive Engine node, you will find information on how to do so here: https://primersion.com/he-links
If you already have a node you just need to:
- Stop the node
- Set
lightNode: truein theconfig.json - Start the node and it will initially delete all transactions from the database and all blocks <
lastVerifiedBlockNumber - 1(ATTENTION: this can't be undone and you can't switch back to a full node)
This process may take a few minutes - if you want to be faster you may want to use a lightnode snapshot and restore the database first. Ask me on discord for a recent snapshot or check out the following url:
https://snap.primersion.com/light/
Restoring process is the same as for full nodes:
- Download snapshot
- Drop database
- Restore snapshot using
mongorestore - Set
startHiveBlockinconfig.json - Set flag
lightNode: trueinconfig.json - Start node
- If switching from a full node you may need to run
db.runCommand({compact: 'chain', force: true})to free up storage.
Differences of light nodes
Light nodes do not store transactions and do not store blocks, which means that the transactions collection is empty and the chain collection will only contain the genesis block and blocks >= lastVerifiedBlockNumber - 1. As a result the following changes were made to the rpc interface:
getStatusrequest will return an additional parameterlightNode: truegetBlockInforpc call disabled (throws an error)getTransactionInforpc call disabled (throws an error)
I already made a change to the witness monitoring site https://primersion.com/he-witnesses, which will now show a small green circle for witnesses only running a light node:
Cons
- Transactions are not stored in the database (transactions collection is empty)
- Blocks are only stored as long as >=
lastVerifiedBlockNumber - 1 - Can't switch back from light node to full node (need to restore from full node snapshot)
Pros
- Fast restore and dump (~10 minutes, compared to > 2 hours for full node)
- Small dump size (400MB currently, compared to 40GB for full node)
- Nodes get less load
- 2GB of storage compared to 90GB of a full node
- Less RAM usage
The long term goal will be to create a good mix of full nodes and light nodes, which will make the network more stable and still provide enough full nodes for services requiring the data.
In any case I will continue to run a full node with my witness account @primersion.
That's it - if you have any questions about the code or something else, feel free to ask below.
The light node is exactly what I was waiting for to test some of the stuff I am trying in terms of mongo snapshots and the Cloudflare front end stuff. =) Huge thanks for this.
Great work, congrats!
Light node is an equivalent of witness node for Hive, isn't it?
Yeah, that was the idea, since the main function of witnesses is verifying blocks and they shouldn't really require to have a full state of the blockchain in my opinion :)
Yay! 🤗
Your content has been boosted with Ecency Points, by @primersion.
Use Ecency daily to boost your growth on platform!
Support Ecency
Vote for new Proposal
Delegate HP and earn more
Congratulations @primersion! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s):
Your next target is to reach 56000 upvotes.
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPCheck out the last post from @hivebuzz:
Support the HiveBuzz project. Vote for our proposal!
Dear @primersion, we need your help!
The Hivebuzz proposal already got an important support from the community. However, it lost its funding few days ago and only needs a few more HP to get funded again.
May we ask you to support it so our team can continue its work this year?
You can do it on Peakd, ecency,
https://peakd.com/me/proposals/199
Your support would be really helpful and you could make the difference! Thank you!
Nice post