Yet another Hive mirror instance
I’ll use this post for on-chain coordination on the progress of its deployment.
It will evolve for the next few days as the new instance of the mirror is brought to life.
Three months ago I started the first public mirror net instance for Hive.
There were three more instances running and it was of great help to all core devs involved in making Hive better. Now I’m in the process of updating it to the release candidate version for HF26.
When?
Soon(TM).
As you can see in my posts… I mean, by the lack of them, there’s a lot of work going on.
RTFM
You can find some more info on the mirrornet (a.k.a. fakenet) in my
previous post
I strongly recommend you read it before doing anything that involves Hive Mirrornet.
Logo reveal video that I’ve featured in my previous post converted to a fancy animated GIF
Notes
The "Recipe for Copy&Paste Experts" will currently not work- Conversion process is time consuming and resource hungry (like everything that runs on such a huge amount of data), fortunately it needs to be done only once for the whole mirror. Other participants will just download the converted one.
- To have a better idea about the amount of data, we are processing
629GBof data, just getting md5 checksum for the input file could take a couple of minutes (on a regular HDD it might take more than an hour). With a 1Gbps network you will need roughly two hours to just download it. - I've used a trick to speed things up - knowing that there were no significant changes in the blockchain converter itself, I’m reusing converted
block_logthat was used for a previous instance of the mirror and resume feature so I could just convert blocks in range66000000-66755355. That saves us two days if the replay will succeed. - It wasn't possible to just replay previous instance because the gap between the blocks that was too big and caused unexpected issues. (In the real world scenario it is very unlikely for Hive to be stopped for more than 7 days!)
- Sleeping is such a waste of time.
- Replay on my node took unexpectedly long time so I used plan B (that is I borrowed @blocktrades resources) - before my node reached 50M blocks, I was able to move data there, do the replay, do the snapshot, get the snapshot back to my infrastructure, load it and start production).
- Mirrornet (converted)
block_logand binaries are already uploaded to my server (the usual place where you can get the useful Hive stuff from), so those who are willing to run their own nodes can start downloading it. Before you download it, I will have a snapshot ready. - The "Recipe for Copy&Paste Experts" should work again (see my
previous post) - The original "Recipe for Copy&Paste Experts" used
block_log.indexfile instead of the new fancyblock_log.artifacts, so those who would use that would have to recreate artifacts on their own. Now with the updated recipe it's downloaded which saves some time (assuming fast Internet connection). (IO+CPU) vs (Bandwidth) trade-off. - Please note, that if you want to build a
hivedbinary yourself, you need to configure it properly to work with the mirror, i.e. with cmake's-DHIVE_CONVERTER_BUILD=ON. - New instance started, please make sure that you are using
hivedat9b1e913acafd42ef1fe30e97310fa2dab8241ea7or later, because otherwise you will fork out (due to witness schedule change) - Providing a fully functional Mirrornet API node is far more tricky than just running a bunch of consensus nodes, I already have an Account History node up, and Hivemind sync in progress. Once it's done I will be able to patch all the pieces together.
Changelog
- Using current latest
developi.e.5885515c2e99064213b3b2b33708ada28a8702e0. - Using the current state of the mainnet, i.e. as the input for the converter.
- Mainnet uncompressed
block_logat block66755355- size
674947358456 - md5
0a686f863ab32cc8d6265b3c82384994
- size
- Mainnet uncompressed
- Conversion started (incremental, see Notes above)
- Conversion finished
- Mirrornet compressed
block_logat block66755355- size
368370305268 - md5
0e7d66af3c757f1c71bb34b96ea3180e
- size
- Production on Mirrornet started
Generated block #66755356 with timestamp 2022-08-06T06:39:09 at time 2022-08-06T06:39:09
- Mirrornet
block_logand binaries uploaded to https://gtg.openhive.network/get/testnet/mirror/ - Updated
mirror-consensus-bootstrapsnapshot is now available. - Attached
config.inifile that's compatible with the snapshot. - Updated original "Recipe for Copy&Paste Experts" to include the
block_log.artifactsfile instead of the obsoleteblock_log.index. - Processing Mainnet transactions (through node based converter) starting at block
66866000(that's 66840234 on the Mirrornet). HARDFORK 26 at block 66840644- I've found some issues #350, #351, #352 related to different versions with different witness schedules, which forces me to reinitialize the whole Mirrornet.
- New instance of the Mirrornet started, same
block_logheight as previous one, withHIVE_HF26_TIMEunchanged.- Using current latest develop i.e.
9b1e913acafd42ef1fe30e97310fa2dab8241ea7
- Using current latest develop i.e.
HARDFORK 26 at block 66755388- Processing Mainnet transactions (through node based converter) starting at block
66906969(that's 66756449 on the Mirrornet). - Another node with extra validation was started.
- Account history node that's needed for Hivemind was revived from the snapshot.
- Hivemind sync from scratch survived replacing the AH node.
- Apparently a bug in RocksDB based account history node requires an update, so replaced binaries with current latest develop i.e.
0ace05ebfbcdf8ac887e1ad5c5b2a2dcf082b5fd
Work in progress...
This post will be updated during the next few days until the mirror consensus will be fully functional and other participants could join it. That of course will include a "starter pack" to download and bootstrap your nodes. So please, pay attention, and then, once it's up and running, please participate.
(That's a mirror so you will participate to some extent even if you don't know it ;-) )
As I guess the available to download block_log is a compressed one, your "Recipe for Copy&Paste Experts" should be updated to download
block_log.artifactsinstead ofblock_log.indexGood point, thanks, I just updated the recipe!
Always those mirrors! :D
When I read this type of posts it makes me want to study programming, both to understand more about what they talk about and to be able to contribute to such great projects.
What are you waiting for? :-)
Finish studies in Project Management 😋
Studying programming is not something I want to spend time on right now. I have a life project more important than that, but without adoubt it is something that I will learn. 😊
Do it in the meantime. That will help with computer science related project management :-D
Me after reading what Mirrornet is and not understanding a single goddamit thing
It's a separate testnet that copies over all the blocks and transaction traffic (posts, comments, votes, transfers, etc) from the mainnet.
Nothing on the mirrrornet is real (there's no economic value for tokens on the mirrornet), but it makes for a great environment for testing Hive nodes and Hive apps, because it is closely mirroring what is happening on the real network. Because it is a close copy to the real network, it allows us to detect potential bugs that we might not see with simpler testing methods and increases the chance for us to detect bugs that might occur on the real network once we deploy the new code.
The mirror i was thinking is is quite different in this regard. I think I grab now. Thanks for this short piece. Practically have zero knowledge about programming.
That's an explanation that I can get behind! Thanks <3
Keep up the good work. Are you coming to Hivefest? I'd like to meet you again.
Thank you, that's my plan (for the HiveFest too!), hopefully see you there :-)
😎💪 so awesome! That's some coos S.. stuff there! :P
Congratulations @gtg! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s):
Your next target is to reach 150 posts.
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPCheck out the last post from @hivebuzz:
is it another hive similar network ?
No, it's a tool to help with testing Hive and improve its development processes.
Hive is such a wide platform and I am having hard time exploring everything. Sorry I'm just new here and I have to remember my posting keys.
Keys are not to be remembered, they are to be saved in a secure manner and not only private posting key, even more important is to keep active and owner keys saved securely.
True, Hive has it's learning curve, but it's worth to spend some time and learn about how it works.
This is quite ingenious. I once witnessed a website that was attacked. They called it a Ddos attack and they advise that people login with the mirror site. Was quite confused, I think i understand better now, the importance of mirror sites.
Well, same name, quite different purpose.
Hive has such functionality built-in, you can see my post in:
hive.blog, ecency.com, peakd.com, or even in many block explorers such as hiveblocks.com, or hive.ausibit.dev
Every Hive node has a copy of the network.
However, a Mirrornet (a.k.a. fakenet) is a concept of a disposable clone, that can be used for experiments without the risk of affecting the original.
Who am I personally? Other than a person who knows how Hive and votes (up and down) actually works.
So I'll try to explain. There's no reason to be upset for the downvotes. It's not about violating rules. All the votes, whether up or down are here to split rewards pool among the content creators. Everyone is free to post whatever they wish, and downvotes can't change that.