We’ve recently added five new programmers that were previously working on another project to our Hive team: three frontend devs, one frontend tester, and one backend dev. We now have over 30 devs working on Hive projects.
Below are a few highlights of the Hive-related programming issues worked on by the BlockTrades team since my last report.
Hived: blockchain node software
- Support for resuming artifact file generation. Also fixed flushing problems on long-running Hived nodes. This will be supplemented soon by: https://gitlab.syncad.com/hive/hive/-/merge_requests/974
- Load snapshot clears shared memory file automatically
- Unit testing improvements - developed new fixture much better emulating real Hived environment - most of Hive startup code has been reused:
https://gitlab.syncad.com/hive/hive/-/merge_requests/948
https://gitlab.syncad.com/hive/hive/-/merge_requests/963
https://gitlab.syncad.com/hive/hive/-/merge_requests/965 - Fixed bugs in RC related condenser_api methods, to return legacy form of operation/transaction
- Changes were made to the Hive requests/response verification schema library as required by clive. There were also changes in test-tools to allow clive to share code.
https://gitlab.syncad.com/hive/hive/-/merge_requests/959
https://gitlab.syncad.com/hive/hive/-/merge_requests/970
https://gitlab.syncad.com/hive/hive/-/merge_requests/972
https://gitlab.syncad.com/hive/test-tools/-/merge_requests/152 - Reduced the size of proposal_pay_operation (unneeded data members removed). Resolves issue: https://gitlab.syncad.com/hive/hive/-/issues/498
- Hive docker/run_hived_img.sh script improvements: https://gitlab.syncad.com/hive/hive/-/merge_requests/916
https://gitlab.syncad.com/hive/haf/-/merge_requests/300
https://gitlab.syncad.com/hive/hive/-/issues/498
https://gitlab.syncad.com/hive/hive/-/issues/548
https://gitlab.syncad.com/hive/hive/-/issues/547
Denser: modern replacement for Condenser
Although I am just announcing it now, the Denser project has actually been under way since the beginning of this year. The idea behind Denser is to replace the rather messy codebase of condenser (code used for hive.blog and several other hive-based front ends) with a cleaner, more maintainable codebase that uses the latest web technologies.
At this point, the Denser prototype has reached the stage where it is clear the project is very viable, so we’ve opened up the repo for general inspection by the community. Denser has an automated deployment system so I'm able to preview and test each new deployment of the site (but unfortunately that site isn't yet accessible to users outside of our local network).
Clive: new Hive wallet with a text-based user interface
Clive is a Hive wallet written in Python that runs on your own computer (it is not a web-based wallet where the code comes from a remote server) so it is inherently more secure than web-based wallets. Currently there are two such wallets available and supported in the Hive ecosystem: 1) a command-line interface wallet (aka the CLI wallet) written using C++ and 2) a graphical interface wallet called Vessel (a JavaScript-based wallet).
Clive is designed to be easier to use than the existing CLI wallet, but at the same time it doesn’t require a terminal that supports graphics like Vessel does. For most people, Clive should provide a more friendly interface for performing Hive operations in a high security environment compared to using the CLI wallet.
I saw the first demo of Clive on Friday and I’ll make a later post with some screenshots of it. Currently it only supports transfer operations, but the project is getting to the point where it should be easy to add support for many additional operation types soon.
Mostly work is focused on preparing a version for public demo purposes and bugfixing:
- changing exit shortcut to avoid accident kill while exiting in prepared testnet version
- display network where wallet is connected to (Mainnet, Alt-chain)
- header shows blockchain state info (head block number, time, elapsed time from last refresh)
- implemented external cli support: https://gitlab.syncad.com/hive/clive/-/merge_requests/94 https://gitlab.syncad.com/hive/clive/-/merge_requests/92 https://gitlab.syncad.com/hive/clive/-/merge_requests/96
- refactor of operations screen to get better UI experience and
https://gitlab.syncad.com/hive/clive/-/merge_requests/113 - Many bugfixes, error handling improvements:
- https://gitlab.syncad.com/hive/clive/-/merge_requests/114
- https://gitlab.syncad.com/hive/clive/-/merge_requests/118
- https://gitlab.syncad.com/hive/clive/-/merge_requests/111
- https://gitlab.syncad.com/hive/clive/-/merge_requests/110
- https://gitlab.syncad.com/hive/clive/-/merge_requests/101
- https://gitlab.syncad.com/hive/clive/-/merge_requests/100
- https://gitlab.syncad.com/hive/clive/-/merge_requests/98
- https://gitlab.syncad.com/hive/clive/-/merge_requests/97
- https://gitlab.syncad.com/hive/clive/-/merge_requests/99
Wax repo: glue code for C++, Python, and Javascript
Additional work related to Clive is also being done in the wax repo. Wax acts as a glue between the direct blockchain code (written in C++), clive-foundation (written in Python), and (soon) Javascript, providing a shared codebase for transaction serialization.
Beekeeper: tool for managing encryption keys and generating/signing Hive transactions
Along with the creation of Clive, we created a new C++ program that can be used to store encryption keys and sign transactions with those keys. The purpose of this new program is to separate the high-security aspects of encryption key management from other wallet operations. For example, Clive doesn’t directly store keys or sign transactions. Instead it communicates with a separate beekeeper process and requests these operations to be performed. For those familiar with the EOS ecosystem, beekeeper is based on the ideas embodied in keosd.
New WASM-based version of Beekeeper
The first version of Beekeeper was written in C++, but now we are creating a WASM (emscripten) toolchain based image to allow cross compiling beekeeper to a WASM target. The idea is to spawn beekeeper in a web browser to provide a common way to safely store keys, perform transaction serialization/deserialization and transaction signing.
The wasm version of beekeeper is in this branch currently: https://gitlab.syncad.com/hive/hive/-/tree/mt-wasm-beekeeper A merge request will be made soon (probably Monday).
WASM-based beekeeper is part of broader work related to creating a common authentication component for Denser, integrated chat, and all other apps where Hive authentication is needed.
The new WASM-based code will also eliminate the need to duplicate transaction serialization implementation in any Javascript library which wants to support Hive. The transaction generation code will soon be moved from beekeeper to the wax library to reduce the amount of code in beekeeper (so as to reduce the amount of security analysis required).
Hive Application Framework (HAF)
HAF is a SQL-based framework for creating highly scalable and robust 2nd layer apps that operate using data from the Hive blockchain.
New HAF features and bug fixes
- provided common function to calculate HivePower from vests
- fixes to query supervisor (due to bugs detected at using public haf instance): https://gitlab.syncad.com/hive/haf/-/merge_requests/291
https://gitlab.syncad.com/hive/haf/-/merge_requests/294 - explicit error when custom tables have too long identifiers (and creating a shadow tables can fail silently)
- unified way of creating forking and nonforking apps
- Improved JSON conversion performance
- Improved textual dump of hive.operation (to bytea literal) to speedup pg_dump/pg_restore processes
- fixed bugs caused to HAF node crash while processing a fork
- fixed database object ownership management for automatically created items (by HAF)
- Fixes to account metadata state provider due to bugs detected while integrating it in the block explorer app
HAF query supervisor to prevent rogue queries consuming too many resources
One of the more important tasks we’ve been working on for HAF recently is the creation of a “query supervisor”. This is a set of code that is used to monitor the queries executed on a SQL server and limit how many resources can be used by that query before the query will be terminated.
The query supervisor is necessary to prevent potential denial-of-service attacks that might be launched against a HAF server, especially a HAF server that allows the execution of semi-arbitrary SQL code (e.g. the code of SQL code that would likely be used for a smart contract system running on a HAF server).
Publicly accessible read-only HAF server testing
Recently we setup an instance of a HAF server that is publicly accessible with the query supervisor enabled to rate limit loading on the database. By publicly accessible, I mean it is possible to make direct read-only SQL queries to this HAF database that are not mediated by any API interface.
Based on what we learned from that test, we’ve made further improvements to the query supervisor and plan to deploy a new version of the publicly accessible HAF server with the latest HAF version and the latest query supervisor in the coming week.
HAF bug fixes and test improvements
- Fixed bug that could lead to a constraint violation while processing forks (community reported bug)
- Eliminated hived node hung while closing
- Fixed bug when different SQL role (user) owns application schema and different role creates context
- Support for creating testnet HAF environments to simplify writing testcases
HAF-based block explorer
We’ve been working for a while on new HAF-based block explorer for Hive. There were two main reasons for creating another block explorer for Hive: 1) we wanted an open-source block explorer than could easily be deployed by any of the existing API node operators without adding much overhead to their servers as a means of further decentralizing this critical functionality and 2) we wanted a “heavy-duty” HAF app that could help us identify and fix potential weaknesses and programming difficulties that might arise when developing a complex HAF-based app.
Block explorer backend
- Massive verification of account balances. This was done to verify the new features and recent fixes made to the HAF balance_tracker application which can provide historical data about various account balances.
- Endpoints for saving balances
- New endpoint for Block explorer UI
Block explorer UI
For the original prototype of the Block explorer UI, we only had one relatively new frontend programmer assigned. Now we’ve added several more experienced programmers to the project, and we’ve embarked on a major code rewrite to bring this project inline with the frontend coding guidelines we’re establishing for all our frontend projects in order to use a common set of web technologies.
We hope to complete the rewrite by the 3rd week of August. Here’s some of the work planned and the devs assigned to each piece:
https://gitlab.syncad.com/hive/block_explorer_ui/-/issues/7
https://gitlab.syncad.com/hive/block_explorer_ui/-/issues/8
https://gitlab.syncad.com/hive/block_explorer_ui/-/issues/9
https://gitlab.syncad.com/hive/block_explorer_ui/-/issues/10
Some upcoming tasks
- Integration of keyauth state provider provided by HAF into HAF block explorer.
- Continue work on Consensus State Providers (for more powerful HAF apps).
- Continue work on HAF-based block explorer backend and GUI
- Add support for more operations to Clive wallet.
- Collect benchmarks for a hafah app operating in “irreversible block mode” and compare to a hafah app operating in “normal” mode (low priority).
- Publish more documentation for various new tools (beekeeper, Clive, consensus state providers) and separate HAF documentation into smaller, more easily digestible chunks.
- Create docker compose scripts to ease deployment of API node infrastructure.
- Benchmark some of the recent performance improvements in HAF and HAF-based apps.
- Deploy an updated version of HAF to our publicly-accessible HAF server.
A lot of things done here! Great one.
Any news on a L2 haf smart contracts :)
The query supervisor is one of the key technologies for the L2 haf smart contract system (it's the reason it is being created, although it can also serve other uses, such as for a publicly-available HAF server). And the updates to HAF in terms of performance and some of the feature improvements we're making are also important to the smart contract implementation.
Ok, now in addition to Clive and Vessel, that's gonna be awesome. Thank you!
Thanks!
Thank you for everything, some really special new stuff coming I can see.
I know I should be using HAF for what I'm doing with Podping but it was just too heavy when I first looked at it. I will try to take a new look at it sometime.
Yes, HAF can be extremely light nowadays, because you can just filter out operations you're not interested in, and that's the largest portion of the database by far.
Great to see all the active updates!
There has been lots of development, I am new here but this is really great
Great development to us.
A lot of things done here!
Thanks for the effort to continue improving our blockchain.
Can communities be made to work with account claim tokens?
Currently we have to burn hive.
Communities are a 2nd layer function, so they can't perform 1st layer functionality such as account creation.
So, communities aren't just another account with special function?
I mean there should be an account on HIVE @hive-(insert community ID) that allows you to both manage the community and hypothetically create accounts with account claim tokens too.
Example https://hiveblocks.com/@hive-135001
I created a community, @hive-104940, with this account, but did not see an option to claim it with a claim token.
I used peakd's interface.
Did I miss something?
I think it might be a UI thing then. Afaik and what I can see on hiveblocks it looks like a normal account. Maybe there is a 3 Hive fee for other parts of the community not just the account itself for it to be active.
Normal looking hive account creation ^ just using liquid hive.
Yes, that is what I thought.
I'll ask peakd about it.
No, communities are more like special "tags" that one or more accounts create and can control.
Keep up the great work guys 👍 !PGM
BUY AND STAKE THE PGM TO SEND A LOT OF TOKENS!
The tokens that the command sends are: 0.1 PGM-0.1 LVL-0.1 THGAMING-0.05 DEC-15 SBT-1 STARBITS-[0.00000001 BTC (SWAP.BTC) only if you have 2500 PGM in stake or more ]
5000 PGM IN STAKE = 2x rewards!
Discord
Support the curation account @ pgm-curator with a delegation 10 HP - 50 HP - 100 HP - 500 HP - 1000 HP
Get potential votes from @ pgm-curator by paying in PGM, here is a guide
I'm a bot, if you want a hand ask @ zottone444
Honestly I haven't been keeping up, but I did have a question?
Can you use Blocktrades in the US now?
Unfortunately, no. We shut down our cryptocurrency exchange business recently, see here for more details: https://hive.blog/blocktrades/@blocktrades/blocktrades-ending-its-cryptocurrency-trading-service-as-of-june-30th-2023-today
Feature Request: allow to remove reblogs from one's timeline again, at the moment all blogging related functions are editable except reblogs.
That would only require a change to hivemind, since it is 2nd layer functionality. I'll ping @howo since he's working in this area to see what he thinks.
Thanks for all your hard work
Long live to the web3 King
Congratulations @blocktrades! Your post has been a top performer on the Hive blockchain and you have been rewarded with this rare badges
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPCheck out our last posts:
@blocktrades, you guys are doing a good job but could you please bridle thugs of Hivewatcher and cancel their funding from DHF? As I believe they are doing more harm than good to Hive.
It's not only me but several other users as well being viscously attacked by Hivewatcher.