Update on BlockTrades Hive work as of Nov 9th, 2020

[blocktrades]

Last week we continued work on post-HF24 optimizations. Below is a summary of the work done last week and our plans for the upcoming week.

Hived work (blockchain node software)

We’re currently finishing up a major optimization to the get_block_api plugin that should likely provide a big boost in performance for the get_block API call (this will likely enable us to speed up the hivemind “full sync” process as well). The old implementation used an overly pessimistic mutex locking scheme that severely degraded potential performance under loading.

We’ve also added a new API call to fetch a consecutive set of blocks in one disk read operation.

We’ll likely commit those changes late today or tomorrow.

Hived status

We plan to tag v1.24.7 as soon as we complete the optimizations to the get_block_api plugin. V1.24.7 will be a recommended upgrade for API node operators, but it doesn’t contain changes needed by witness nodes or exchanges.

Hivemind (2nd layer microservice for social media applications)

Performance optimizations and code cleanup:






https://gitlab.syncad.com/hive/hivemind/-/merge_requests/339 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/329 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/347 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/357 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/337 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/365 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/366

Significant speedup in unread_notifications code:

https://gitlab.syncad.com/hive/hivemind/-/merge_requests/348 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/353

Various bug fixes:





https://gitlab.syncad.com/hive/hivemind/-/merge_requests/346 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/358 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/359 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/362 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/360 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/354

We added API tests for a hivemind “full sync” test run. Full sync test runs are very time-consuming, so these will only be run periodically (e.g. once a week).
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/293

We also introduced performance benchmarking into the automated build-and-test (CI) system. This will help prevent performance regressions as changes are made to the implementation of API calls and/or changes to hivemind’s database schema.


https://gitlab.syncad.com/hive/hivemind/-/merge_requests/339 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/350 https://gitlab.syncad.com/hive/hivemind/-/merge_requests/352

We’ve also been making further improvements for monitoring the performance of hivemind on our production system using various public domain tools for postgres.

Hivemind status

We deployed most of the above changes to hived and hivemind to our production node, api.hive.blog, and they’re working well.

Condenser + Condenser wallet (open-source code for hive.blog)

There’s a known issue with editing comments. We thought we had fixed this issue, but we’re still seeing it in production. It’s a high priority issue, so we’ll likely fix it tomorrow.

What’s the plan for next week?

• Finish re-merge of muting changes related to decentralized lists (some of this work needs to be re-incorporated due to two large divergences in the hivemind code base as part of the work to move notifications and reputation out of hived). I expect we will complete this in the next couple of days. This is one the largest remaining performance issues for hivemind (use of the old global mute lists can have a very negative impact on performance serving up posts).
• Deploy some pending merge requests and analyze performance.
• Enable reputation calculation code.
• Continue creating hivemind tests.
• I still haven’t had time to write up plans for new post-hardfork features, but I’m hopeful I will have time in the next couple of days to get to that.

Comments

[bashadow]

It is a slow process to check and recheck, I for one am glad you guys are taking the time to get it down solid and are not pushing for HF 25 before all the issues are resolved. We have seen in the past how thinking "we can fix that in the next fork" has only ever caused issues.

[zekepickleman]

I think have been noticing get_block errors on services like @hivetasks but figured it was just fork stuff.

It is always a refreshing surprise when you guys are so attentive and transparent with your development for the blockchain and the community.

I am looking forward to the post-hardfork features and other neat things in store for us.

Thanks @blocktrades!

[marki99]

Thanks for the post, always nice to read.

Hivemind work is infinite at some point it will be better to leave it even though it is not optimized.

[tobetada]

thanks for all the work!

[jacobtothe]

This is why you earned one of my witness votes. Keep up the good work! You're one of the main reasons HIVE is alive!

[steevc]

Good stuff. With any complex system there are always things to fix and improve. Hive is getting better all the time. Thanks for caring.

[footballbg]

Nice!

[minnowspower]

$tangent

[tan.bot]

^{Congratulations, @minnowspower You Successfully Trended The Post Shared By @blocktrades.
You Utilized 1/3 Daily Summon Bot Calls.}

---

^{TAN Current Market Price : 0.280 HIVE}

[mima2606]

Thank you very much for this comprehensive informations.

Liebe Grüße Michael

!invest_vote
!jeenger

[investinthefutur]

@mima2606 denkt du hast ein Vote durch @investinthefutur verdient!@mima2606 thinks you have earned a vote of @investinthefutur !

[jeenger]

Keep up the good work!Your contribution was curated manually by @mima2606

[preparedwombat]

It’s a high priority issue, so we’ll likely fix it tomorrow.

If only the rest of the world worked like that.

[knowhow92]

Thanks for the update @blocktrades!

[drynemer]

Thanks for the post, it's always good to read from you and know that hive is still alive and getting better every day.

Thanks for your job!

[tan.app]

^{0.578 TAN & Curators Made 0.578 TAN.Congratulations @blocktrades, You Earned}

---

^{Join CORE / VAULT Token Discord Channel or Trade TANGENT Token
TAN Current Market Price : 0.300 HIVE}

[brittandjosie]

Got my answer 😉 Enable reputation calculation code. thanks for the work