This post has been updated from 1.24.4 to 1.27.2
Used With Permission From @someguy123
Because @someguy123 hasn't written a post about using HIAB I decided to do it, mainly because I use it and always end up referring to discord chats on how to set it up. Much easier to just have it all in one place, and what better place than the chain itself? Im just going over basic setup using p2p sync but theres even more functionality(you can replay the server instead of using p2p, or use a snapshot) which you can read about here: https://github.com/someguy123/hive-docker.
Server
First thing that you'll need is a server for this. Thanks to the efforts from Blocktrades and team, its become quite a bit easier to run I would recommend a Privex server for this because you can pay with Hive and HBD, but because we have a lot of witnesses already on Privex and its important to have our chain be distributed between different datacenter and providers, I would recommend that you find your own. I do run a company which rents out servers so you could potentially use that, check out Ryamer.
I run my main witness(@hextech) node on a server with 32 gigs of RAM and 8 CPUs for the witness along with 1TB(about 400 GB being used right now) on a NVME on Ubuntu 20. You could possibly get by with less threads and 16 gigs of RAM, but please use a fast disk or else it'll take a long time to get caught up.
Setting Up HIAB
I like to update my server as the first thing to do as well as install git and screen.
sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install git -y
sudo apt-get install screen -y
Then we clone the repo.
git clone https://github.com/someguy123/hive-docker.git
And then use it to install docker for you. This shouldn't take too long.
cd hive-docker
./run.sh install_docker
We can then install the latest docker image for the witness with. It will grab whatever is the latest from dockerhub(https://hub.docker.com/r/someguy123/hive/tags).
./run.sh install
CLI Wallet And Keys
Now we are going to want to generate our witness signing keys. You can reuse some keys that you already use, but it's recommended to generate new ones just for signing blocks. To do that, we use the cli_wallet.
We'll be using the remote wallet as our local instance isn't ready yet.
./run.sh wallet
You should get presented with something like this on a fresh install. If it says locked instead of new then you already have a wallet setup(you won't unless you are updating) and you can unlock it and skip the init step.
You can type in set_password to set your password. For example, if I wanted my password to be 123, I'd type in set_password 123 Remember the password as you'll need this to unlock the wallet in the future. You'll want to unlock your wallet after that. Thats as simple as unlock 123. Replace 123 with the password you set.
Once you are unlocked you can use suggest_brain_key to get a public private keypair generated. You'll want to save that as you'll need it in the future. 
You can exit the wallet by pressing control + D. If you ever forget your password, you can wipe the wallet clean and use a new password. A wallet.json file is generated and stored within the data directory. Just delete that and it'll start the wallet fresh again.
Config
First let's increase the shared memory file size.
./run.sh shm_size 24G
Next we'll need to modify the config file to work with our witness. Its located at data/witness_node_data_dir/config.ini directory. We'll want to modify that.
nano data/witness_node_data_dir/config.ini
Most of the values there are already good. If you want to modify anything go ahead if you know what you are doing.
The place we want to modify is witness = and private-key =. They are already commented out(line begins with #) so we can just go find them and delete the # at the start of the line. For the witness, you'll want to type in the username of the account that the witness will be running under. This value must go in quotes. The private key should be the private key that we generated from the cli wallet. This should not go in quotation marks. Use the image below as an example.
Starting Up
We are going to start the node up, and to do that, run the following command
./run.sh start
This does take some time, on my machine I mentioned above, it took about a day to sync to head. You can monitor what step of the process this is at by using ./run.sh logs.
Once the older blocks are synced up, it'll automatically start syncing up to live. You can check the sync progress using ./run.sh monitor. It'll tell you how long it'll take to get synced up. Again this is a time to wait. You'll know that you are live when you see individual blocks being produced in ./run.sh logs. That'll look like this:
The block numbers will be going up by one. Nows the big time, time to enable as a witness.
Enabling Your Witness
Let's enable the witness, you can do so using the wallet. Run the following to open it up:
./run.sh wallet
Unlock the wallet with the password set before using unlock 123 where 123 is the password. We'll want to import in your active key to sign the transaction that we'll be broadcasting. Find the private active_key using your preferred choice. If your private active key is 5KNrHHigj3PRurkm5moaV1XM3gWGrDZJJZCgPRL2FpBLjWy1obg you'd import it by typing in import_key 5KNrHHigj3PRurkm5moaV1XM3gWGrDZJJZCgPRL2FpBLjWy1obg
Now we want to broadcast enabling the witness. Modify the following command to match what you want and type it into the wallet. Replace username with the name of your witness, the https://example.com part with the url you want to use for your witness and STM1111111111111111111111111111111114T1Anm with the public key generated for your private signing key when we first got into the wallet. Following that are your witness parameters, modify them to your liking, but these are the current standard ones being used by witnesses and so it's wise to follow along.
update_witness "username" "https://example.com" "STM1111111111111111111111111111111114T1Anm" {"account_creation_fee":"3.000 HIVE","maximum_block_size":65536,"hbd_interest_rate":0} true
After that gets broadcast, you are up and running as a witness. You can check that you are enabled somewhere like https://peakd.com/me/witnesses. The green ball means that you are enabled. Your version will not report to the version that you are using until you sign your first block so don't worry if it says 0.0.0. Just get the word out there that you are being a witness and get more votes.
I was following along on a fresh server and it took me 2 hours and 15 minutes to get to the point where I could broadcast the enable witness transaction. The latest updates and the work from someguy to make HIAB exist has made it a lot easier to be a witness.
Future & Questions
You'll want to setup a pricefeed and failover script as well. For pricefeed, I recommend someguy's hivefeed-js (can be found here https://github.com/someguy123/hivefeed-js) as well for failover script(watcher by therealwolf works https://github.com/therealwolf42/hive-witness-essentials/tree/master/essentials/watcher)
As time goes on, you'll need to update. The update process might involve a replay or it might not. Hard forks involve a replay and soft forks usually don't. You can get information and ask questions about that in the Privex discord which can be found at http://discord.privex.io/. The server requirements might increase as well and you might need to upgrade that.
If you like what I do, feel free to vote for my witness https://vote.hive.uno/@rishi556
Looking for a server to host on? Check out Ryamer
Heh, you beat me to it.
Great guide, I've reblogged it so most HIAB users will find it :)
Greetings @rishi556 ! Thank you very much for sharing this post. I have a question, I hope you can help me.
By this, do you mean the private active key of the witness account, or the same one generated in the previous step, when we did "suggest_brain_key"?
The witness already has more than 24 hours synchronized and printing the lines that I indicate in the image
And he's active on the witness list.
Do you think everything is fine?
You'll want to import your private active key first to be able to broadcast the operation to enable your witness, I also recommend importing the key from
suggest_brain_keyas some CLI commands use it(witness_update actions I believe is what they are called).That's normal. It's just how your node is spending most of its, time, it's mostly idle, which is good since it's not getting overloaded.
As long as your key is correct in the config.ini(most common issue I've seen people do), you are good to go. Now go out there and get more votes, best of luck to you.
I already imported the private active key of the witness project, I have not imported the ones I generated with suggest_brain_key, although I did write it in the config.ini file. So import the one from suggest_brain_key, and everything is ready, or is it not necessary now?
Friend, thank you very much for your prompt response, that is very valuable.
Not necessary if you aren't going to use the CLI. I honestly never use the CLI anymore, it was basically only used for enable/disable which worked with just the active key, then I've used ausbit's site to manage which key is active or enabling/disabling ever since. Check it out here: https://hive.ausbit.dev/witnesss
Brilliant! thank you so much!!!
I am trying to figure it out and so far am not having all that much luck.
Here's the latest update on that experience.
https://peakd.com/hive-169321/@borepstein/hiab-hive-in-a-box--20-april-update
I wonder what I may be doing wrong.
Made a comment on the post for anyone else wondering.
This is honestly awesome and very tempting for me to finally pull the trigger on launching a node myself
Posted using Dapplr
Setting up a node isn't too hard. Its maintaining it and campaigning for votes. Good luck if you decide to take the plunge.
Well, I know for a fact I wouldn't win the popularity contest for votes here (at least not yet) considering the amount of tight-knit mega whales there are - but, I would do it solely for the benefit of the community. I know it'd be a resource drain, but I'd simply ask for donations or just upvotes on posts/client posts to make it work.
Otherwise, I pay and it's cool - that's that. Sometimes, people have to act selflessly for the greater good, and I feel this is one of them (even if in just a very small way).
This is a good run down for HIAB - it's an easy starting point for anyone setting up their own server. I started using @drakos price feed it's simple and reliable https://github.com/Jolly-Pirate/pricefeed
I recommended the one from someguy because I work on it from time to time and so I'm more familiar with it.
Sweet, Ill prob set up a witness sometime soon .. just snowballing myself there
Good luck. Setting up is very easy :) Getting the votes is soooooooo much harder.
Ive heard talk about resetting the witness votes in a future HF
Yup, I would love that since it would push me up since I'm newer :) But it would also mean witnesses actually have to work for their spot and can't sit back and relax like some of the top 50 are doing right now.
might be something to do on some rolling .. I kinda feel the same about delegations
Thanks for this. I'm in the process of running HIAB using Docker Desktop on a Windows 10 Home machine using WLS 2 (thanks @themarkymark).
I'd do a post when its all working.
My machine is quite a beast - 112 Gb RAM with 6/12 core Intel i76800k CPU, 500 Gb NVME SSD & 500 Gb SATA SSD - so I'm thinking of turning this into an API node and then running a witness on a second, lighter machine.
What changes do I need to make to do that?
What is the network traffic requirements for a witness node on a monthly basis?
Haven't really measured that. Setting up you'll eat up like 350 GB downloading everything you need. Day to day running is a lot less.
I've attached my network traffic graph for the last hour. I can't really tell how often its being transferred but I'm guessing per second, and using that as an estimate, its about 70 GB/month which really isn't bad at all. (30 kb/sec * 60 sec/min * 60 min/hour * 24/hour/day * 30 day/month).
Do you have a transaction log for how many requests you get during that timeframe.
Witness nodes don't normally handle requests(that'd be the job of a fullnode, but it is possible to take some specific requests which I have my witness node doing for some very small private projects). Witness nodes just get and process blocks and sign blocks when they get assigned to produce a block.
Now that I've got my witness node running I want to be able to take snapshots so I can restart easily in case of computer reboot. How do I do that?
I think you have to stop running it first(I haven't tried it yet) then run
./run.sh dump_snap SNAPSHOTNAME. Then it'll be located withindata/witness_node_data_dir/snapshot/with the name that you chose.@rishi556
I followed the steps, but getting this error after the steps of import_key.
error.
./run.sh: line 1836: STEEM_RUN_ARGS[@]: unbound variable
Why the heck steem still exist?
Any idea why?
Thats a thing from hive-docker and @someguy123 being lazy and not renaming everything to Hive. For importing key, it should be done once you start wallet (
./run.sh walletor./run.sh remote_walletif your local node isn't synced up yet). Then unlock it and then doimport_key 5XYZ....I think I will start over.
There’s no need to start over, it only looks like you are doing something wrong in the wallet step. Just try it again.
Glad I found this. Incredibly helpful! 🙌
Hi,
my blockchain version is 1.25.0
./run.sh dlblocks --> is up to date
./run.sh replay --> last step is database.cpp:5884 apply_hardfork HARDFORK 11 at block 3276913
./run.sh start --> return the following error :
1174662ms database.cpp:170 open ] Opened a blockchain database holding a state specific to head block: 3348714 and last irreversible block: 3348691
1174670ms database.cpp:183 operator() ] Blockchain state database is AT IRREVERSIBLE state specific to head block: 3348714 and LIB: 3348691
1174672ms database.cpp:248 open ] 10 assert_exception: Assert Exception
revision() == head_block_num(): Chainbase revision does not match head block num.
{"rev":0,"head_block":3348714}
database.cpp:192 operator()
1174672ms database.cpp:248 open ] args.data_dir: /steem/witness_node_data_dir/blockchain args.shared_mem_dir: /shm/ args.shared_file_size: 26843545600
1174675ms chain_plugin.cpp:538 open ] Error opening database. If the binary or configuration has changed, replay the blockchain explicitly using
--replay-blockchain.1174675ms chain_plugin.cpp:539 open ] If you know what you are doing you can skip this check and force open the database using
--force-open.1174675ms chain_plugin.cpp:540 open ] WARNING: THIS MAY CORRUPT YOUR DATABASE. FORCE OPEN AT YOUR OWN RISK.
1174675ms chain_plugin.cpp:541 open ] Error: {"code":10,"name":"assert_exception","message":"Assert Exception","stack":[{"context":{"level":"error","file":"database.cpp","line":192,"method":"operator()","hostname":"","timestamp":"2021-12-16T19:19:34"},"format":"revision() == head_block_num(): Chainbase revision does not match head block num.","data":{"rev":0,"head_block":3348714}},{"context":{"level":"warn","file":"database.cpp","line":248,"method":"open","hostname":"","timestamp":"2021-12-16T19:19:34"},"format":"rethrow","data":{"args.data_dir":"/steem/witness_node_data_dir/blockchain","args.shared_mem_dir":"/shm/","args.shared_file_size":"26843545600"}}]}
any idea ?
Thanks
I just ran into this. You need to invoke
./run.sh shm_size 25Gor maybe even30Gto avoid the hive node from crashing and corrupting the database this way.Thank you for this great guide, but you're missing an important line item: users need to invoke
./run.sh shm_size <size>to prevent the hive node from crashing. I'm not sure what the necessary size is nowadays but I believe it's at least 25GB. Can you please add this to the guide?I haven't run into any shared memory size issues during my time running HIAB.Update, did notice this happen on Privex NIAB, which came with 19G SHM by default, and we resolved that by upping to 25G, and HIAB had been upped to that at some point in the past.Hi @rishi556, @hextech or @someguy123,
I started the remote wallet, but the set_password command is "not found" - what should I do?
Try using the local wallet if that doesn't work(./run.sh wallet). Can I get a screenshot of the error message as I haven't seen that before.
And when I tried the local wallet, this happened:
Ah, there's some issues with the remote wallet not working with the docker build. For now, you are going to want to use the local wallet. The issue with the local wallet is that you need HIAB to be running. Did you start up the node using
./run.sh start? Once you get the node running and fully up to sync, you can then use the local wallet,./run.sh wallet, generate some keys and insert them into your config file, and then finally restart the node./run.sh restartand register using the local wallet again. I'll poke @someguy123 to take a look at the wallet issues.I did make ./run.sh start,
but when I afterwards did the ./run.sh wallet, it shows this:
Or do I need each time sync the block.log first?
No need to fetch the block_log, it's pretty fast with just using p2p to sync. It should be syncing up right now, just might take a while(it took about a day for me on a pretty powerful machine). You can check the progress using
./run.sh logsOnce it's counting the block numbers up by 1 each time, you can try running the local wallet.logs currently shows this, not sure if this is one block each time, doesn´t look like it
./run.sh wallet leads to this
Sorry for my ignorance and bothering you. I am not an IT-guy.