HF16 and post-HF16 cause a lot of trouble for witnesses, so I would like to update my witness as follow, with the hope that it could help other fellows witnesses in some ways.
Disclaimer: It's a collection of wisdom from other witnesses, not my own.
For hardware I think it's overkill and not worth for backup witnesses, but I love overkill ;)
- 1 dedicated Dual Xeon, 32GB RAM, 200GB SSD
- 2 VMs at Azure and Google Cloud, 2-4 Cores, 10-15GB RAM, 60GB SSD
- 1 private server Core i7, 32GB RAM, 250GB SSD, behind a corporate-grade firewall, no direct ssh allowed
- 1 seed node Azure
All five machines were compiled with low memory mode, syncing share_file to shm (thanks to @abit and @gandalf) and zswap trick from @smooth to reduce IO as much as possible.
To use shm, please do following (10-12GB is the recommend size)
sudo mount -o remount,size=12G /dev/shm
Remember to backup your /dev/shm before reboot because it will be clean and you will have to resync from scratch again. On the good side that the syncing time is much faster now, 10mins for reindex and 70mins for resyncing.
Suggestion from @abit:
No need to resync but need a replay, if you have --data-dir set to a location on a disk.
To use zswap, please change this line
GRUB_CMDLINE_LINUX="zswap.enabled=1
in /etc/default/grub, then run update-grub and reboot (credit goes to @smooth, thanks).
The first three machines are running with only database_api and witness plugin (no cli connection allowed).
Recommend config.ini:
shared-file-size = 12G
shared-file-dir = /dev/shm
flush = 0
public-api = database_api #login_api
enable-plugin = witness #account_by_key
The flush parameter doesn't actually have any effect so disable it now for less logs.
The private server is hosting the price feed and failover script which automatic switching between 4 machines. This server is running with login_api and account_by_key to allow connection from cli_wallet.
If you're interested in my failover script please get it here. Automatic switching between unlimited nodes, no dependency, just a small python script but it requires you to have cli_wallet running as daemon.
Basically for my setup, the script running at the private server, if the current witness node misses blocks for any reason, after a threshold (2 blocks in my case), the script switch the witness signing key for the next node. By design if all the keys were iterated then the script will stop. If it switches, you will get an email notification through mailgun API.
Just drop me a line if you have something to ask.
Thanks for reading.
Aizen
No need to resync but need a replay, if you have --data-dir set to a location on a disk. So actually for low mem nodes I'd recommend just let the shared file go if the disk is slow, as it's annoying to copy a 10G file from memory to disk before reboot, then copy it back afterwards.
I didn't pay attention to that flag. Thanks for correcting. Will add it now.
Thank you. This is great information as I am learning to maintain my witness.
Can you please fix your price feed?
I fully understand now @aizensou ... "Witness"
This post has been linked to from another place on Steem.
Learn more about and upvote to support linkback bot v0.5. Flag this comment if you don't want the bot to continue posting linkbacks for your posts.
Built by @ontofractal
Congratulations @aizensou! You have completed some achievement on Steemit and have been rewarded with new badge(s) :
Click on any badge to view your own Board of Honnor on SteemitBoard.
For more information about SteemitBoard, click here
If you no longer want to receive notifications, reply to this comment with the word
STOPIf you want to support the SteemitBoard project, your upvote for this notification is welcome!
why do you not post? :) greetings Lena
Congratulations @aizensou! You have completed some achievement on Steemit and have been rewarded with new badge(s) :
Click on any badge to view your own Board of Honnor on SteemitBoard.
For more information about SteemitBoard, click here
If you no longer want to receive notifications, reply to this comment with the word
STOPBy upvoting this notification, you can help all Steemit users. Learn how here!
Nice post ! I followed you.
Congratulations @aizensou! You have completed some achievement on Steemit and have been rewarded with new badge(s) :
Click on any badge to view your own Board of Honor on SteemitBoard.
For more information about SteemitBoard, click here
If you no longer want to receive notifications, reply to this comment with the word
STOPCongratulations @aizensou! You have completed some achievement on Steemit and have been rewarded with new badge(s) :
Click on any badge to view your own Board of Honor on SteemitBoard.
For more information about SteemitBoard, click here
If you no longer want to receive notifications, reply to this comment with the word
STOPexcelente post
Hello. You have a seed node
104.40.230.35:2001which is currently showing as "Offline" on https://status.steemnodes.com/. Are you planning to bring this node back online, or should we remove it from the list of seed nodes?