Hardfork 24 progress last week

in HiveDevs4 years ago (edited)

Below is a summary of HF24 work by BlockTrades team in the past week:

Hived work

Last week we made one final update to hived, this time to allow eclipse nodes that haven’t yet processed hardfork 24 to connect to nodes that have processed the hardfork. This change was necessary because the chainid (the special code that identifies network nodes for the hive blockchain) will change when hardfork 24 takes place. The chainid change posed a potential problem after hardfork 24 took place, because someone starting an eclipse node for the first time that didn’t have blockchain data would only connect to nodes with the old chainid, so they wouldn’t be able to get blocks created after the hardfork. To fix this problem, we now allow eclipse nodes to communicate regardless of whether they’ve processed the hardfork or not yet.
https://gitlab.syncad.com/hive/hive/-/merge_requests/110
https://gitlab.syncad.com/hive/hive/-/merge_requests/111

At a later date, sometime after the hardfork, we’ll release a version of hived that always broadcasts the new chainid (although it will still validate blocks before hardfork 24 using the old chainid).

We’ve tagged v1.24.0 which has the hardfork date set to be triggerable after Sept 22nd (whenever a super-majority of the top 20 witnesses upgrade to the new version). We’re also notifying exchanges to begin updating to v1.24.0, so that they can continue to operate without any interruption whenever the hardfork is triggered.
https://gitlab.syncad.com/hive/hive/-/tags/v1.24.0

Hivemind work

Most of our work last week continued to focus on hivemind.

We continued work on the reputation algorithm. The second attempt at a SQL-based implementation was also too slow, and we’re about to try a 3rd one. If that one is too slow, we’ll implement an in-memory solution using python that periodically commits the computed reputations to the database.
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/121

We’ve mainly completed the new notification algorithm that generates notification information on demand instead of constantly updating it for all accounts. The last thing that needs to be implemented is notifications from communities, which do need to be created as objects rather than on-demand. We’re hoping to complete the implementation tomorrow.
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/123

During our “full sync” tests of hivemind, we found a couple of problems, including one problem where indexes that were dropped for fast processing during “bulk syncs” (a fast processing mode used to process many blocks at once when hivemind is far behind hived’s head block) weren’t always probably restored during normal “live sync” (when hivemind is in sync with hived and receive blocks one at a time). We made a fix today and we’re waiting for results from that test now (this particular test is time intensive). Related commits:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/125
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/128
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/144

Finally, we continue to create more API tests for hivemind, fix small discrepancies between the old version of hivemind and the Eclipse version as we find them, and make various small cleanups to the code:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/129
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/118
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/130
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/133
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/134
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/137
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/140
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/141
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/145
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/146
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/147

Hardfork release date and plans for upcoming week

We’re still working on hivemind, but we have setup one preliminary API node for eclipse that has hivemind support (minus reputation and notification support, which still needs testing). See https://hive.blog/hive-160391/@gtg/how-to-survive-eclipse-cheat-sheet for more details.

There’s still a chance that we can have the hardfork on the 22nd, but it will depend on us being able to setup a full hivemind node for apps testing this week (i.e. we’ll know in the next few days if we can meet that deadline). If we don’t, we will tag a new version of hived with an updated hardfork date. This should not pose any major inconvenience to witnesses or exchanges who’ve already updated to hived v1.24.0 if it happens, since no blockchain replay would be required (exchanges wouldn’t even be required to update to the later version, as there would be no consensus changes).

Sort:  

Thank you for all the hard work team BT et all has done. Really curious how Hivemind progress finishes, as I think it's imminent for the continuation of the ecosystem as you have indicated yourself too. Rest assured I feel the devs are sane and on top of things, so maybe what I'm saying here is echo'ing in a chamber errrr... hive:

Although hiveD(aemon) would work & fork with a v1.24 fork on 22nd September, if that results in Hivemind being broken because of forking changes, leaving the hive-swarm with a less than optimal (or broken) user experience, I'm all for waiting until the whole stack is reliably working on HF24.

If users need to wait for fixes in the Hivemind (blockchain viewer optimalization software), and although the chain keeps running, it will still give a bad taste. Better to prevent that situation with a temporal HF date delay.

I've just had a chance to catch up on HF24 and its super impressive from a technical standpoint.

All the old pain points regarding RAM and slow replays seem to have been resolved with far more efficiently programmed system.

Its amazing that what Stinc couldn't do in 3 years, Hive Devs, working in a decentralised environment have managed in 6 months.

I'm following the work that still is being done on Hivemind and I want to congratulate the whole team for this impressive effort! We are going to have a powerful and much more flexible tool at our disposal.

However, I think it would be unwise to perform the hardfork when all of these modifications have not been exhaustively tested. Having migrated all data and non-consensus APIs out of hived to hivemind makes the latter crucial for layer-2 applications. If it did not function properly, the entire ecosystem would be weakened.

So, although I am eager to deploying HF24, I don't think September 22 is still realistic. Waiting another week won't make a big difference but that could save us from a negative experience for users who would see their UI and (d)Apps stop working.

Better safe than sorry!

Nice, thanks for the update @blocktrades. So if something will go wrong upon testing it could be possible that 22 will not be an exact date? Well, it's fine as long you will make things sure without fail. There's no need to rush when this will be the benefits for the hive.

By the way, what do you mean about notification from the community? About new things will come in hive?

Yes, 22nd isn't an exact date, for two reasons: 1) it depends on us getting a version of hivemind ready for apps testing in the next few days and 2) witness don't have to upgrade right away, in which case it could take longer. So what it really means is that Sept 22nd is earliest possible date that we will hardfork, but not necessarily the day it will happen. Testing and witness upgrade cycle will set the actual date.

Regarding notifications, it's a low-level coding thing related to hivemind itself. Front ends like hive.blog have a feature called "notifications" which allows you to know when someone has, for example, "mentioned" you, rehived your post, or upvoted one of your posts. There can also be notifications as a result of actions performed in a community. These latter types of notifications need special handling in hivemind, compared to the other ones.

Ah, so we will expect amazing things in hive for the hardfork to come.

But I think mentioning, rehive are already going on in hive? Although it's not that active because sometimes it won't notify me. So this time it will be 100% sure that if someone does an activity that needs mentioning me, it will notify me right away? That's nice, good luck.

Yes, notifications work right now, but they were broken by required changes in new Eclipse code, so we're fixing them again (and improving efficiency of that code in the process).

Ah okay, so that's what you meant. Thanks for answering my simple questions by the way. Keep safe and thanks for the hard work.🤗

Hello, will a new token be created after the hardfork?

No, this is system upgrade HF. No split of chain and no new coins.

Congratulations @blocktrades! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :

Your post got the highest payout of the week

You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP

To support your work, I also upvoted your post!

Looking forward to September 22nd

Thank you for the update

Thank you for the update, hopefully the notifications issues all get resolved, which I am sure it will. When you mention Reputation algorithm, is that the same as our reputation score like mine being displayed as 67? On peakd? and if so will the changes make it easier or more difficult to increase a persons reputation? I know the old bid bot system was pretty useful for some in building a rapid high reputation, compared to those that took a slower reputation route growth.

It's the same algorithm, so no change, except it's being done in hivemind instead of hived (which is so far a bit more challenging to code in a performant way).

Thanks for clarifying for me.

I have sugestion to update the terminology im Whitepaper of Hive. Because in the https://developers.hive.io/resources/#resources-whitepaper there are many word "Steem" and I think tat can be confuse for me as a beginner. So I think must build the Whitepaper of Hive 😁

Whitepaper is being written here if you want to contribute: https://gitlab.syncad.com/hive/hive-whitepaper

Any idea how long the HIVE / HBD gates will be under maintenance? My BTS wants to turn into LEO. 😁

Should be up in 15 minutes or so.

Nice. I'll check back in. Enjoy the day. 👍

Hello, thank you for the update. Is there documentation somewhere on how to set up a node? and how to upgrade it to v1.24 when it goes out?

Hello thanks, @blocktrades, as far as I know, there are no changes on API? So Dapps won't need to update? am I correct?

There are minor changes to the API. Most of those changes have been made at the level of the API libraries used by the apps. So, for most apps, the major change will be to update to later versions of their API library. But there could be issues that require direct changes. There's also some new API calls that apps may want to use (APIs for decentralized lists).