6th update of 2024: Wax release planned at end of June, hived/HAF 1.27.7 at end of July

[blocktrades]

Below are a few highlights of the Hive-related programming issues worked on by the BlockTrades team since my last report.

Hived: core blockchain node software

We’re planning to release 1.27.7 sometime in July (probably near the end of July). As part of this release, we’re also setting what we expect will be the final “trigger date” for hardfork 28: September 24, 2024 at 14:00 UTC (to allow a little time after the upcoming HiveFest in Croatia).

Most of the changes for 1.27.7 are already done, but we’re giving about a month for our production testing. Both hardfork-related changes and some other changes associated with this release can be found in links below (and/or discussed in subtopics below):

• https://gitlab.syncad.com/hive/hive/-/milestones/6#tab-issues
• https://gitlab.syncad.com/hive/hive/-/milestones/7#tab-issues (unstarted issues described here won’t be part of the release)

Split block logs (enables “lite” hived nodes for services)

We are in the middle of implementing the last changes associated with the split block logs functionality that I discussed in my previous report and we expect to complete that work this week. Then we plan to devote a week to production testing of the new various new features associated with block log splitting.

We added another new feature based on the new split block log technology: hived now opens block log files in read-only mode and then switches to read/write mode when the node begins syncing. This change allows old block log parts to be shared across multiple instances of hived, further reducing storage space required to support multiple hived nodes capable of supplying sync blocks.

Other hived changes

• Cleanup of signature generation code: now hived uses the bip0052 format everywhere instead of the fc::legacy code. Old legacy signatures are still verified correctly.
• Slight redefinition of virtual operations holding assets: where possible they were changed to use specialized versions of tiny_asset to limit the runtime C++ object size and more importantly, the size of their binary serialized form (this reduces HAF database size).

• Fix for bug in P2P layer associated with block compression changes that caused some hived nodes to crash (not sure why this bug took so long to appear, however).

BLS signatures research

We did some initial research on BLS signatures, the report is here: https://gitlab.syncad.com/hive/hive/-/wikis/Research-of-BLS-signatures

We plan to first implement support for BLS in the new Wax API library, where it will initially be useful for 2nd layer apps, then assuming we find no problems, add support for BLS in the 1st layer (in HF 29, which is scheduled for a relatively quick release after HF 28).

Beekeeper (cryptographic key management software)

Current we are working to finish up the python wrapper for BeeKeeper, which is needed to support Wax’s transaction builder for Python.

HAF API node

Added support for optional Prometheus and Grafana monitoring.

HAF (Hive Application Framework)

We’ve just about completed the redesign of the “app main loop” for HAF to make it more difficult for app developers to make an error when designing their app’s processing logic. That work is here:
https://gitlab.syncad.com/hive/haf/-/merge_requests/504

Other changes to HAF

• We’ve implemented a new approach for HAF application schemas that allows the same application to be deployed multiple times (to different schemas and contexts): https://gitlab.syncad.com/hive/haf/-/merge_requests/484
• Storage space optimizations (reduces current storage requirements by around 300GB):

1. Removed hive_rowid indexes for irreversible applications (i.e. hivemind)
2. Elimination of block_num, op_type_id columns from hive.operations and hive.account_operations tables. They are replaced with function calls decoding this data from operation_id
3. Re-using new virtual operation serialization to limit vops storage size in the database (this feature was discussed in the hived section above)
4. Elimination of operation timestamp column from operations table and replacing it with referenced block creation time.

2nd layer “Lite accounts” that are transportable across the Hive ecosystem

As mentioned in my last report, we are creating a new HAF app to manage the creation and maintenance of “Lite” accounts that can be used by any 2nd layer app to sign 2nd layer transactions. The specification for this also includes documentation for how we plan to support 2nd layer transactions. https://gitlab.syncad.com/hive/haf/-/issues/214.

Most recently we worked out how we will handle the namespaces for lite accounts (using decentralized ids):

The default namespace of la (for light account) can be used by any hive account. So any hive account will be able to create a lite account called did:la:null for example, as there are no special requirements to claim any account name in the la namespace. Names in this namespace are strictly first-come, first-served.

We’ll also have a special namespace for hive accounts called hive so that all hive accounts have one “verified” associated light account. For example, the light account called did:hive:justme can only be edited using the 1st layer keys of the Hive account called justme. So hive accounts don’t need to worry about someone grabbing their unique 2nd layer lite account in the hive namespace.

2nd layer accounts of this type are useful if you want to associate additional 2nd layer specific keys with your Hive account so that everyone knows who you are. For example, you might want to add a “battle” key to play a 2nd layer game.

We also have a plan for how to create namespaces for other blockchains and also non-blockchain apps such as google/facebook, etc, but I’ll go into that more in a later report.

HAF Block Explorer and Balance Tracker APIs

• Adjusted for new HAF version
• Support for account authority data
• Support account reputation info in API calls
• Optimizations of get_account API call performance (time reduced from 1s to 50ms)
• Auto-disable block/comment search APIs when optional indexes are not built
• OpenAPI rewrite support: This is part of work I described in my previous post to auto-generate some useful standards-conforming REST calls via rewrite rules AND generate interactive swagger pages for HAF apps (HAF block explorer is our “test case” for the new technology). Most recently we worked out how to best handle REST calls that take array parameters.

Wax (New multi-language Hive API library)

We’re preparing the first official release of the Wax npm package at the end of this month (2 weeks).

Latest improvements to Wax

• Implemented TS operation builder and properties serializer for witness_set_properties operation
• Implemented witness_set_properties text formatter to make it readable for humans
• Implemented API healthchecker component
• Package size reduced from 5.4MB to 2.3 MB (compressed size)
• New methods for key utility functions (suggest_brain_key, generate private key from password)

Clive: command-line and TUI wallet for Hive

• Published release: https://gitlab.syncad.com/hive/clive/-/releases/v1.27.5.11
• New features in progress (planned for June 19th release): https://gitlab.syncad.com/hive/clive/-/merge_requests/399
  UI changes cover account alarm management. CLI changes include bugfixes to recurrent transfer and custom_json support.

Imagehoster

• Fix for URL handling code that could crash imagehoster
• Updated docker images to use newer version of node.js
• Deployed version now supports SVG images

Some of our work in progress (or planned for near future)

• Creation of HAF-based Lite Accounts application (implementation in progress). This also includes support for 2nd layer transactions which should be useful for many types of HAF apps.
• Developing spec for 2nd layer smart contract processing engine (most of docs are here: https://gitlab.syncad.com/hive/smarc/-/tree/abw_smarc_design/doc?ref_type=heads )
• Many hived improvements: https://gitlab.syncad.com/hive/hive/-/issues/675
• Official release of wax and workerbee npm package
• Finish new OpenAPI documentation of existing REST APIs (balance tracker and HAFAH are next)
• Create a release candidate for Denser (replacement for Condenser)
• Continue adding new commands to Clive
• Release new reputation tracker app
• More hivemind performance improvements (continued replacement of Python code). Replacement of python web server with PostgREST server should be completed in time for July release.
• Integrate reputation_tracker as a sub-app inside hivemind. This should improve space optimization (now hivemind must collect all votes to recalculate reputation) and sync speed. We also hope to have this in July release.

Comments

[andablackwidow]

Slight redefinition of virtual operations holding assets

While there was no reason not to do it, only the trivial part of the task was done so it took very little time, and it also serves as a protection against passing wrong assets to virtual operations, the impact was extremely disappointing despite hundreds of millions of vops in database. Therefore even though we could still continue with more complex cases, we most likely won't because there would be no gain. It also puts into question somewhat similar but separate task of merging vops related to comment payouts - at the very least we'll have to estimate the impact before making any changes.

[blocktrades]

Since the vast majority of the operations are custom_json, it's not surprising the impact on the size of the operations table was small.

[sanjeevm]

So what is our final goal that we want to achieve ? And how far we are from that ? It's good to see we are working, but its equally important to figure out, how it aligns to the goal.

[blocktrades]

We have several goals for our current work:

• Ensure we have a completely scalable decentralized blockchain. This means we need to continue lowering the costs to operate the entire infrastructure and make it as parallel distributable as possible so that it can handle a huge number of transactors, social media users, etc. Hive can already easily match the transaction capabilities of the fastest other blockchains, but we need to think beyond that to create a truly viable large-scale solution. As to where are at currently with this goal, we'll need to do more testing to see where we are at now, as it has been a while since we have done such testing and many improvements have been made. But some of the improvement can be seen now in terms of the lowered costs of running Hive API nodes that can serve more users. Right now I think a single API node with one computer can handle all of Hive's traffic: compare this to the 70K per month that Steemit was spending to support their fleet of servers a few years ago.
• Create a scalable 2nd layer solution for smart contract processing. This is needed to enable Hive to move beyond just the social media space into the larger world of distributed computing and a larger user base for Hive. Note that such features will also make Hive more attractive to social media users too, because it means Hive will offer more capabilities to social media users than they have with traditional social media. As to progress in this area, there's several solutions under development by other Hive developers, and we're making promising progress on our own solution (which in many cases will provide technology that other devs will be able to re-use in their own solutions, such as the lite account tech we're creating).
• Create a best-in-class development environment for blockchain app development. This is in many ways one of the most challenging tasks, as it requires a lot of work to create a powerful, intuitive, and well-documented framework for app developers. But IMO we are making good progress in this area, and I expect we will start to see more of the fruits of this effort near the end of this year as new apps embrace the technology.

[sanjeevm]

Create a best-in-class development environment for blockchain app development. This is in many ways one of the most challenging tasks, as it requires a lot of work to create a powerful, intuitive, and well-documented framework for app developers. But IMO we are making good progress in this area, and I expect we will start to see more of the fruits of this effort near the end of this year as new apps embrace the technology.

I smell the confidence, thank you so much for the detailed response. I understand the challenge, but who dares wins and 2025 will be ours. Kudos to you and the team.

[resiliencia]

Thanks for the work @blocktrades !

[hivebuzz]

Congratulations @blocktrades! Your post has been a top performer on the Hive blockchain and you have been rewarded with this rare badge

Post with the highest payout of the day.

_{You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP}

Check out our last posts:

LEO Power Up Day - June 15, 2024

[mdasein]

Keep up the great work guys 👍

[michealb]

👍

[thisnewgirl]

this is always good to know, some good work getting done.

[cryptoyzzy]

!LOL
!ALIVE

[lolzbot]

lolztoken.com

Bruce Lee was fast.
But not as fast as his brother Sudden Lee.
_{Credit: manuvert}
@blocktrades, I sent you an $LOLZ on behalf of cryptoyzzy

(8/10)
Delegate Hive Tokens to Farm $LOLZ and earn 110% Rewards. Learn more.

[forykw]

😱 I need to catch up, Soooo many changes!

[samgiset]

It is exciting to know that we already have a date for the next HardFork. I see each of these updates shaping the future of the Hive blockchain. 🚀

[encuentro]

Hola @blocktrades , es muy importante el trabajo que estás realizando, felicitaciones. Leyendo tu publicación, me di cuenta que mucho me falta por aprender de este mundo blockchain y su lenguaje técnico, gracias por los apoyos que he recibido de tu parte, me han ayudado mucho.

[thelivingworld]

This projects are really good, important and helpful to all hive users and also to the Platform. 👍👏

[offgridlife]

Dear @blocktrades … Why are you downvoting all my posts on Day 7 ? For over a year now ? How can I get you to stop ? What do I need to do ? I have brought hundreds of very good bloggers to Hive. You have effectively deleted over $ 2000 HBD from my rewards. I am a Long time blogger (June 2017) and investor in Hive and HBD,

[devferri]

wow