As I mentioned in my previous post https://peakd.com/dev/@howo/hive-core-developper-meeting-8 we are going to setup multi nodes testnets and we want as many people participating as we can.
Testnets won't be using the main net chain data, so they won't be taking that much resources, (a node currently takes about 45mb of ram and since those testnets will be short lived I don't expect them to use more than 500mb).
We'll create two testnets sequentially:
First testnet will be set without a supermajority support for hf24 to see how the transition goes at the hf24 time (nothing should happen and we should continue on hf23). We are just interested in the transition, so as soon as the test is successful, we'll kill it and switch to the second one. This testnet is already live and the hardfork date for it is 2020-08-27T20:00:00.000Z UTC (1598558400 unix time)
Second testnet will be the same but one where the super-majority of witnesses run hard fork 24, we want to see how the transition from hf23 to hf24 goes (we should fork to hf24). This one is not live yet, it will be after the 27th when the first one has hard forked. We will hardfork on the 29th at 20:00 UTC
How to run your own testnet node
First you need to pick wether you want to be running a hard fork 23 or hard fork 24 node. If you don't know, pick hard fork 23 for the first testnet, and hard fork 24 for the second hard fork.
Hard fork 23 node setup
Do this if you want to run a hard fork 23 node, if you want to run a hard fork 24 node, skip to the next section.
I feel like this info is getting copy pasted a lot, but I need to write it here again because there are some slight variations.
Install the dependencies:
So first of all install all of the dependencies :
# Required packages
sudo apt-get install -y \
autoconf \
automake \
cmake \
g++ \
git \
libbz2-dev \
libsnappy-dev \
libssl-dev \
libtool \
make \
pkg-config \
python3 \
python3-jinja2 \
doxygen
# Boost packages (also required)
sudo apt-get install -y \
libboost-chrono-dev \
libboost-context-dev \
libboost-coroutine-dev \
libboost-date-time-dev \
libboost-filesystem-dev \
libboost-iostreams-dev \
libboost-locale-dev \
libboost-program-options-dev \
libboost-serialization-dev \
libboost-signals-dev \
libboost-system-dev \
libboost-test-dev \
libboost-thread-dev
# Optional packages (not required, but will make a nicer experience)
sudo apt-get install -y \
libncurses5-dev \
libreadline-dev \
perl
build hive
in hf23, hive still uses the steem naming, so that's why you're seeing a lot of steem things all around. But it is hive
git clone [email protected]:hive/hive.git
cd hive
git checkout testnet-0.23.0
git submodule update --init --recursive
mkdir build
cd build
cmake -DENABLE_COVERAGE_TESTING=ON -DBUILD_STEEM_TESTNET=ON -DLOW_MEMORY_NODE=OFF -DCLEAR_VOTES=ON -DSKIP_BY_TX_ID=ON -DCHAINBASE_CHECK_LOCKING=OFF ..
make -j$(nproc) steemd cli_wallet
if you got an error while doing the git clone try with this url:
https://gitlab.syncad.com/hive/hive.git
if you got an error while doing the git clone try with this url:
https://gitlab.syncad.com/hive/hive.git
Then wait a bit, depending on your machine, this process can take a while (it takes about 5 minutes on my machine)
running it
Run the node a few seconds and then exit steemd like so
./programs/steemd/steemd -d testnethf23/
this will create a testnethf23 directory with the default config file.
then open the config.ini file
nano testnethf23/config.ini
The default config is okay, but you need to add these fields
# tells the node which seeds he should listen to to get blocks
p2p-seed-node = 95.216.25.162:2001
p2p-seed-node = 95.216.25.162:2002
p2p-seed-node = hf24.mintrawa.com:2001
# more seed nodes will be added later
# Local http endpoint for webserver requests.
webserver-http-endpoint = 127.0.0.1:8090
# Local websocket endpoint for webserver requests.
webserver-ws-endpoint =127.0.0.1:8091
# Tell which port should be used to broadcast blocks as a seed node
p2p-endpoint = 0.0.0.0:2001
And edit the plugin field to this:
plugin = webserver p2p json_rpc witness account_by_key reputation market_history
plugin = database_api account_by_key_api network_broadcast_api reputation_api market_history_api condenser_api block_api rc_api
It's a bit overkill to put all these, but it'll make for a better experience if you want to mess with the cli on your own later.
Then run steemd again, this time we want it to stay up so setup a way for it to persist even if you log out, like screen. I personally use pm2 for the testnets just because it's convenient. This is not optimal for a production setup but it's fine for those testnets. (see https://pm2.keymetrics.io/docs/usage/quick-start/)
I run steemd like this with pm2:
pm2 start "./programs/steemd/steemd -d testnethf23 " --name hf23
It will take some time (a few minutes) before you get your blocks, but at some point you should be synched with the testnet and you should see messages like this:
3|hf23 | 1526617ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4415 by howo -- Block Time Offset: -382 ms
3|hf23 | 1532618ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4416 by howo -- Block Time Offset: -381 ms
3|hf23 | 1538612ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4417 by howo -- Block Time Offset: -387 ms
3|hf23 | 1544620ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4418 by howo -- Block Time Offset: -379 ms
3|hf23 | 1550615ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4419 by howo -- Block Time Offset: -384 ms
now your node is running ! So that's great, but it's not producing.
So you need to setup your witness.
Setting up your witness
cli_wallet
Open up the cli wallet:
./programs/cli_wallet/cli_wallet --server-rpc-endpoint="ws://127.0.0.1:8091"
it'll ask for a password, so execute:
set_password yourpassword
then unlock it using:
unlock yourpassword
Now execute those commands:
import initminer's key to do various actions
import_key 5JNHfZYKGaomSFvd4NUdQ9qMcEAC43kujbfjueTHpVapX1Kzq2n
create your account
create_account "initminer" "yourusername" "" true
power up from initminer:
transfer_to_vesting "initminer" "yourusername" "100.000 TESTS" true
then generate keys for your witness:
suggest_brain_key
it'll be something like this:
{
"brain_priv_key": "STRANGE ROADWAY TWASOME MOLER CENTIMO THOFT COMPORT VARIANT OUTSWIM EGGER SCRIBE GLAVER CROWDED DOLLIER AWNED DOPPER",
"wif_priv_key": "5KJUxvUSdAV6H7LiutRJMVwEVptmoaDhM73Cg1itzYjG8zs7tWF",
"pub_key": "TST71oeHrdBZEKDG1GaC75VKDaRGcXk2R8htD8J6wkP6p1zDRPzFZ"
}
Save these for later !
Then set yourself as a witness (notice how I used the pub_key generated above)
update_witness "yourusername" "http://example.org" "TST71oeHrdBZEKDG1GaC75VKDaRGcXk2R8htD8J6wkP6p1zDRPzFZ" {"account_creation_fee":"0.000 TESTS","maximum_block_size":65536,"sbd_interest_rate":0} true
and finally vote for yourself so you get scheduled for blocks
vote_for_witness "yourusername" "yourusername" true true
and we are done with cli_wallet ! but not with the setup.
final config
Open the config one final time to set yourself as a witness by adding these fields:
I use the private key that you generated with the cli above not the one that I put here. I put one here to show that you don't put quotes around the private key.
# name of witness controlled by this node (e.g. initwitness )
witness = "yourusername"
# WIF PRIVATE KEY to be used by one or more witnesses or miners
private-key = 5KJUxvUSdAV6H7LiutRJMVwEVptmoaDhM73Cg1itzYjG8zs7tWF
restart the node and you should be producing blocks, the logs should look something like this:
3|hf23 | 2687621ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4595 by howo -- Block Time Offset: -378 ms
3|hf23 | 2690613ms witness_plugin.cpp:343 block_production_loo ] Generated block #4596 with timestamp 2020-08-25T19:44:51 at time 2020-08-25T19:44:51
Hard fork 24 node setup
Do this if you want to run a hard fork 24 node, if you want to run a hard fork 23 node, see the previous section.
this may feel like a copy paste from above but there are some slight variations.
Install the dependencies:
So first of all install all of the dependencies :
apt-get install -y \
autoconf \
automake \
autotools-dev \
build-essential \
cmake \
doxygen \
git \
libboost-all-dev \
libyajl-dev \
libreadline-dev \
libssl-dev \
libtool \
liblz4-tool \
ncurses-dev \
python3 \
python3-dev \
python3-jinja2 \
python3-pip \
libgflags-dev \
libsnappy-dev \
zlib1g-dev \
libbz2-dev \
liblz4-dev \
libzstd-dev
build hive
git clone [email protected]:hive/hive.git
cd hive
git checkout master
git submodule update --init --recursive
mkdir build
cd build
cmake -DENABLE_COVERAGE_TESTING=ON -DBUILD_HIVE_TESTNET=ON -DLOW_MEMORY_NODE=OFF -DCLEAR_VOTES=ON -DSKIP_BY_TX_ID=ON -DCHAINBASE_CHECK_LOCKING=OFF ..
make -j$(nproc) hived cli_wallet
if you got an error while doing the git clone try with this url:
https://gitlab.syncad.com/hive/hive.git
if you got an error while doing the git clone try with this url:
https://gitlab.syncad.com/hive/hive.git
Then wait a bit, depending on your machine, this process can take a while (it takes about 5 minutes on my machine)
running it
Run the node a few seconds and then exit hived like so
export HIVE_HF24_TIME=1598558400 && ./programs/steemd/hived -d testnethf24/
note the env variable HIVE_HF24_TIME. It's very important that you put it every time you run hived. It's what states at which time the fork will execute.
For the first testnet its value should be 1598558400, for the second testnet its value should be 1598731200
this will create a testnethf24 directory with the default config file.
then open the config.ini file
nano testnethf24/config.ini
replace the existing config this one (stolen and edited from @gtg: https://peakd.com/hive-160391/@gtg/witness-update-release-candidate-for-eclipse-is-out)
# tells the node which seeds he should listen to to get blocks
p2p-seed-node = 95.216.25.162:2001
p2p-seed-node = 95.216.25.162:2002
p2p-seed-node = hf24.mintrawa.com:2001
# more seed nodes will be added later
# Local http endpoint for webserver requests.
webserver-http-endpoint = 127.0.0.1:8090
# Local websocket endpoint for webserver requests.
webserver-ws-endpoint =127.0.0.1:8091
log-appender = {"appender":"stderr","stream":"std_error"}
log-logger = {"name":"default","level":"info","appender":"stderr"}
backtrace = yes
plugin = webserver p2p json_rpc
plugin = database_api condenser_api
plugin = witness
plugin = rc
plugin = market_history
plugin = market_history_api
plugin = account_history_rocksdb
plugin = account_history_api
plugin = transaction_status
plugin = transaction_status_api
plugin = account_by_key
plugin = account_by_key_api
plugin = reputation
plugin = reputation_api
plugin = block_api network_broadcast_api rc_api
account-history-rocksdb-path = "blockchain/account-history-rocksdb-storage"
shared-file-size = 20G
shared-file-full-threshold = 9500
shared-file-scale-rate = 1000
flush-state-interval = 0
market-history-bucket-size = [15,60,300,3600,86400]
market-history-buckets-per-size = 5760
p2p-endpoint = 0.0.0.0:2001
transaction-status-block-depth = 64000
transaction-status-track-after-block = 46000000
webserver-http-endpoint = 127.0.0.1:8091
webserver-ws-endpoint = 127.0.0.1:8090
webserver-thread-pool-size = 256
It's a bit overkill to put all these, but it'll make for a better experience if you want to mess with the cli on your own later.
Then run hived again, this time we want it to stay up so setup a way for it to persist even if you log out, like screen. I personally use pm2 for the testnets just because it's convenient. This is not optimal for a production setup but it's fine for those testnets. (see https://pm2.keymetrics.io/docs/usage/quick-start/)
I run hived like this with pm2:
pm2 start "export HIVE_HF24_TIME=1598558400 && ./programs/hived/hived -d testnethf24" --name hf24
It will take some time (a few minutes) before you get your blocks, but at some point you should be synched with the testnet and you should see messages like this:
3|hf23 | 1526617ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4415 by howo -- Block Time Offset: -382 ms
3|hf23 | 1532618ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4416 by howo -- Block Time Offset: -381 ms
3|hf23 | 1538612ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4417 by howo -- Block Time Offset: -387 ms
3|hf23 | 1544620ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4418 by howo -- Block Time Offset: -379 ms
3|hf23 | 1550615ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4419 by howo -- Block Time Offset: -384 ms
now your node is running ! So that's great, but it's not producing.
So you need to setup your witness.
Setting up your witness
cli_wallet
Open up the cli wallet:
./programs/cli_wallet/cli_wallet --server-rpc-endpoint="ws://127.0.0.1:8091"
it'll ask for a password, so execute:
set_password yourpassword
then unlock it using:
unlock yourpassword
Now execute those commands:
import initminer's key to do various actions
import_key 5JNHfZYKGaomSFvd4NUdQ9qMcEAC43kujbfjueTHpVapX1Kzq2n
create your account
create_account "initminer" "yourusername" "" true
power up from initminer:
transfer_to_vesting "initminer" "yourusername" "100.000 TESTS" true
then generate keys for your witness:
suggest_brain_key
it'll be something like this:
{
"brain_priv_key": "STRANGE ROADWAY TWASOME MOLER CENTIMO THOFT COMPORT VARIANT OUTSWIM EGGER SCRIBE GLAVER CROWDED DOLLIER AWNED DOPPER",
"wif_priv_key": "5KJUxvUSdAV6H7LiutRJMVwEVptmoaDhM73Cg1itzYjG8zs7tWF",
"pub_key": "TST71oeHrdBZEKDG1GaC75VKDaRGcXk2R8htD8J6wkP6p1zDRPzFZ"
}
Save these for later !
Then set yourself as a witness (notice how I used the pub_key generated above)
update_witness "yourusername" "http://example.org" "TST71oeHrdBZEKDG1GaC75VKDaRGcXk2R8htD8J6wkP6p1zDRPzFZ" {"account_creation_fee":"0.000 TESTS","maximum_block_size":65536,"hbd_interest_rate":0} true
and finally vote for yourself so you get scheduled for blocks
vote_for_witness "yourusername" "yourusername" true true
and we are done with cli_wallet ! but not with the setup.
final config
Open the config one final time to set yourself as a witness by adding these fields:
# name of witness controlled by this node (e.g. initwitness )
witness = "yourusername"
# WIF PRIVATE KEY to be used by one or more witnesses or miners
private-key = 5KJUxvUSdAV6H7LiutRJMVwEVptmoaDhM73Cg1itzYjG8zs7tWF
Use the private key that you generated with the cli when doing suggest_brain_key. Note you don't put quotes around the private key.
restart the node and you should be producing blocks, the logs should look something like this:
3|hf23 | 2687621ms p2p_plugin.cpp:212 handle_block ] Got 0 transactions on block 4595 by howo -- Block Time Offset: -378 ms
3|hf23 | 2690613ms witness_plugin.cpp:343 block_production_loo ] Generated block #4596 with timestamp 2020-08-25T19:44:51 at time 2020-08-25T19:44:51
Final words
And done ! I hope you'll consider joining us on this testnet campaign.
I'd like to remind you that the first testnet is due to hardfork (and be killed an hour later on the 27th at 20 UTC) and the second testnet will be hardfork on the 29th at 20:00 UTC.
Seed node
Go the extra mile and setup a seed node ! this is quite important as the testnet have very few seed nodes, so it helps a lot if you make one.
Your node is already configured to be a seed node so you just need to open the corresponding port:
(note that this command may vary depending on the firewall setup of your computer)
sudo iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 2001 -j ACCEPT
And now you can communicate the new seed node to everyone. Please post your ip as a comment of that post so that I can update this link to use your seed as well :).
Thank you for reading.
If you liked this writeup, please consider voting for @steempress it's the witness that I co-manage with @fredrikaa:
Is MIRA always available as an option when use
cmake? there is aHIVE BLOCK EXPLORERfor the testnet?Witness FR - Gen X - Geek 🤓 Gamer 🎮 traveler ⛩️
Mira is still available, but you shouldn't use it. The testnet takes far to little memory for it to be relevant.
There is no block explorer, you have to see things for yourself using the cli_wallet, (a tool which is often forgotten but very useful if you know how to use it. Try the help command to see it's capacities).
I see you're running a node, thanks you !
OK thanks, I wasn't sure if it was kept or not. I use it on my production server but not on the Testnet server I made.
It's OK for the block explorer, I use the cli_wallet and postman to query my node (lazy to make a small Angular application for the little time it will be used)
I have opened my TCP ports, if you need, it's at the end of my post that I made a few hours ago
Witness FR - Gen X - Geek 🤓 Gamer 🎮 traveler ⛩️
Hallo i have a problem with authorization "git clone [email protected]:hive/hive.git" gives a manual link where i can download ?
Ah, you could try with http: https://gitlab.syncad.com/hive/hive.git
Hello i have tested but HF23 stopped at this point and HF24 don't started :( i will check this problem at the evening
Hf24 shouldn't start, that's the point of the first testnet, this looks like the first time you ran steemd.
looks like you are on track, now you need to update the config etc
I online but i think the note don't start , have you any idea why ?
Hi @cervisia, I don't see any error on your screenshot,
Testnet's blockchain is now at block 32115 so it will take some time to replay it before your node comes online.If you have opened your http port, can you give me the TCP port you put and the IP of the node to make some queries with
postman(have you enabled plugins?)Did you do all the stuff with the
cli_walletas @howo put it in his post?Witness FR - Gen X - Geek 🤓 Gamer 🎮 traveler ⛩️
and the ip =
Very fascinating! It’s been a long time since I did any programming (40 years ago in college), but the technical side of blockchains is interesting. I might take a crack at it.
Wow! What was programming like back in the 1970s?
Scratch that question. What were computers like back in the 1970s?
I'd say big IBM machine with punch cards or 2400-foot magnetic stripes for a hundred mega 😄
Witness FR - Gen X - Geek 🤓 Gamer 🎮 traveler ⛩️
Well, 1980s actually. It was all BASIC and COBOL. And I dabbled in FORTRAN and Pascal. If you asked me to remember them now, I couldn’t. As lost to me as my high school Spanish.
The computer was in a room all by itself and the printers were dot matrix. Everything was hard wired, no WiFi back then. In the late 80s my first PC was a portable I bought from a coworker. It had 640k of Ram and two 5 3/4 inch floppy drives definitely a different time.
title typo,
tesntetstestnetsthanks !
What does this all mean?
And.... did you attach a photograph of a fork from google? LOL
Coding is easy.... photography not so much : D
If you don't know what it means you don't have to bother with it :) it's mostly a technical guides for witnesses.
As for the fork picture illustrating deeply technical articles are often quite hard haha, so I improvised, it's a reference to the term "hard fork"
@cmplxty that's the post :)
I just saw it, thank you! Appreciate it!
Whoa, really nice here thanks. I am going to need to take some time to process it! If I can figure it out, I will get it running!
Congratulations @howo! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :
You can view your badges on your board And compare to others on the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPTo support your work, I also upvoted your post!
Do not miss the last post from @hivebuzz:
加油
Oh yeah! I am going to be setting one of these up to help with testing.
Thanks for the copy-paste tutorial, it will help a lot.
I'm totally confused I haven't learnt that far yet
Ok, thanks ;(
I understood, that I must go to study Linux ))))
Every time I run
make -j$(nproc) steemd cli_wallet, I get this error:[ 4%] Linking CXX static library libchainbase.a [ 4%] Built target chainbase CMakeFiles/Makefile2:4873: recipe for target 'programs/steemd/CMakeFiles/steemd.dir/rule' failed make[1]: *** [programs/steemd/CMakeFiles/steemd.dir/rule] Error 2 Makefile:1100: recipe for target 'steemd' failed make: *** [steemd] Error 2I need a bit more logs than that, are you sure that the cmake command didn't output any errors ?
It was a problem with my server, it ran out of memory.
Aah yes, I forgot to mention that compiling requires a lot of memory.
Try to increase swap :)
Is it possible to compile on one machine and then run it on another? I can compile on my personal laptop, but it's not online 24/7, but I have some 1GB RAM servers.
I am not 100% sure, but you could try building and then porting the hived and cli_wallet binaries.
But have you tried increasing swap and building ? It will be slow but it should compile.
If not it's fine to not be online 24/7 you mostly need to be there at the moment of the hard fork (the 30 of august at 20 UTC).